Slip detection plays a pivotal role in the dexterity of robotics, improving the reliability and precision of manipulations but also contributing to safety, efficiency, and adaptability. Deep learning-based slip detection algorithms commonly difficult to concentrate on key features when faced with dense 3D shape data obtained by visuo-tactile sensors. Data from noncontact locations can interfere with slip judgements and the ignorance of interframe linkage can also lead to slip detection failure. In this paper, a new spatio-temporal sequences fusion-based self-attention, STNet, is proposed to perform slip detection by allocating more attention to the object-sensor contact area when processing complex 3D shape data. A binocular visuo-tactile system (BVTS) is designed and fabricated for dataset construction. The entire 3D shape dataset containing 4 motion patterns, including stationary, pressing, rolling and slipping. Self-attention architecture with and without spatio-temporal sequences fusion mechanism (denoted as STNet and TemNet, respectively) are trained based on the same dataset. The experiments show the validity of STNet, which can reach 98.91% slip detection accuracy. Meanwhile, the ablation studies confirm the effectiveness of the spatio-temporal sequences fusion mechanism.

Accurate perception of the surrounding environment stands as a primary objective for robots. Through tactile interaction, vision-based tactile sensors provide the capability to capture high-resolution and multi-modal surface information of objects, thereby facilitating robots in achieving more dexterous manipulations. However, the prevailing GelSight sensors entail intricate calibration procedures, posing challenges in their application on curved surfaces and requiring the maintenance of stable lighting conditions throughout experimentation. Additionally, constrained by shape and structure, current vision-based tactile sensors are predominantly applied to measurements within a limited area. In this study, we design a novel cylindrical vision-based tactile sensor that enables continuous and swift perception of large-scale object surfaces through rolling. To tackle the challenges posed by laborious calibration processes, we propose a self-supervised photometric stereo method based on deep learning, which eliminates pre-calibration requirements and enables the derivation of surface normals from a single image without relying on stable lighting conditions. Finally, we perform surface reconstruction from normal and point cloud registration on the multiple frames of images obtained by rolling the cylindrical sensor, resulting in large surface reconstruction. We compare our method with the representative lookup table method in the GelSight sensors. The results show that the proposed method enhances both reconstruction accuracy and robustness, thereby demonstrating the potential of the proposed sensor in large-scale surface reconstruction. Codes and mechanical structures are available at: https://github.com/ZhangZhiyuanZhang/GelRoller

Digital image correlation (DIC) is an image-based deformation measurement method, which has been widely used in multiple cutting-edge fields, benefitted from its noncontact, low cost and high robustness. However, achieving higher accuracy and overcoming the tradeoff between resolution and measurement uncertainty have been recognized as bottleneck problems for DIC. This article proposes a new method by assimilating sparse observations into DIC, constituting “Assimilated DIC,” to reconstruct deformation fields with high fidelity. In the proposed method, the subpixel registration process in the traditional DIC procedure is viewed as a dynamic model, which constitutes the basis for assimilation. Meanwhile, the sparse observation provides highly accurate guidance for correcting the dynamic model, thus facilitating the update of model data towards the ground-truth. In addition to effectively improving the measurement accuracy without lowering the computational efficiency, Assimilated DIC can obtain a higher resolution without compromising the accuracy performance. And the boundary regions of multiconnected structures, which are generally subject to large errors by traditional methods, can be precisely reconstructed by Assimilated DIC. In this article, the proposed method with two types of observations on strain and displacement has been experimentally validated under various deformation cases, including a rigid-body translation, a heterogeneous deformation, and an extension of a holed plate. These excellent properties of Assimilated DIC have been confirmed in comparison with those of a traditional DIC method.

With the development of computer technology and management science, decision support systems have emerged that can improve the quality and effects of decision‐making. This study mainly examined the application of the wireless communication of an embedded microprocessor and computer vision in the decision support system of martial arts competition referees. Using the embedded microprocessor’s characteristics of a small size, high precision, high reliability, and high efficiency, a decision support system for martial arts competition referees was designed. In the experiment, the similarity between the target field and the source field could be controlled by adjusting the mean value. To better extract the target, this study used the time domain changes of the three adjacent frames, before, middle, and back, to detect the moving target to extract the change detection template. The Canny edge detection method was used to extract the edge information of the image and eliminate the nonmotion area; then, morphology was used to correct the image to complete the connection of the broken edge to obtain the final initial segmentation mask image. In the process of calculation, there were some noises and small fragments. In this study, morphology and background difference were used to optimize the segmented image. Experimental data show that the algorithm detection accuracy rate was high—between 70% and 100%—and the effect was relatively ideal. The results indicate that the proposed algorithm can effectively reduce matching noise, improve the matching accuracy of the edge area and the low‐texture area, and achieve a fast matching speed.

This paper presents a new method that combines a physics-based model and finite-element (FE) based digital image correlation (DIC) with an interfacial continuity condition for simultaneous deformation measurements of strain field and material mechanical properties. The dual-domain global DIC method incorporates a model-based kernel that offers useful physical insights into the complex deformation and constrains the solutions to the iterative minimization of the DIC criterion, which leads to a faster convergence. Unlike local subset-based and global FE-based methods where the DIC is formulated to solve for the nodal displacements, the model-based DIC that accounts for complex deformation in the specific region is formulated to identify system parameters. Both domains, along with their interface and the continuity condition enforced on the global displacement field, are discretized and solved simultaneously in a single calculation. The dual-domain global DIC method has been investigated by comparing reconstructed strain fields and estimated parameters against targets simulated using the commercial FEA software COMSOL and published experimental data. Unlike FE-DIC which is noise sensitive, the model-based DIC is robust against the Gaussian noise effects.

This paper presents an efficient method, which reconstructs the temperature field around the tool/chip interface from infrared (IR) thermal images, for online monitoring the internal peak temperature of the cutting tool. The tool temperature field is divided into two regions; namely, a far field for solving the heat-transfer coefficient between the tool and ambient temperature, and a near field where an artificial neural network (ANN) is trained to account for the unknown heat variations at the frictional contact interface. Methods to extract physics-based feature points from the IR image as ANN inputs are discussed. The effects of image resolution, feature selection, chip occlusion, contact heat variation, and measurement noises on the estimated contact temperature are analyzed numerically and experimentally. The proposed method has been verified by comparing the ANN-estimated surface temperatures against “true values” experimentally obtained using a high-resolution IR imager on a custom-designed testbed as well as numerically simulated using finite-element analysis. The concept feasibility of the temperature monitoring method is demonstrated on an industrial lathe-turning center with a commercial tool insert.

This paper presents the design and analysis of a passive body weight (BW)-support lower extremity exoskeleton (LEE) with compliant joints to relieve compressive load in the knee. The biojoint-like mechanical knee decouples human gait into two phases, stance and swing, by a dual snap fit. The knee joint transfers the BW to the ground in the stance phases and is compliant to free the leg in the swing phases. Along with a leg dynamic model and a knee biomechanical model, the unmeasurable knee internal forces are simulated. The concept feasibility and dynamic models of the passive LEE design have been experimentally validated with measured plantar forces. The reduced knee forces confirm the effectiveness of the LEE in supporting human BW during walking and also provide a basis for computing the internal knee forces as a percentage of BW. Energy harvested from the hip spring reveals that the LEE can save human walking energy.