Tiny crack detection plays a vital role in ensuring the safety of critical industrial components and infrastructure, providing early intervention to prevent crack propagation and mitigate potential damage. Although machine vision-based defect detection has been greatly utilized in safety inspection and maintenance, tiny crack detection remains a big issue due to the low visibility and weak features with large background noise. To address this challenge, this paper presents an end-to-end trainable hierarchical constrained neural network for tiny crack detection. Firstly, we present a hierarchical loss constraint Filter (HLCF) module based on a region-level attention mechanism to capture the holistic vein structure of cracks and enhance the faint feature extraction of tiny cracks. In order to balance local high contrast and holistic crack features, a feature fusion loss constraint is designed to reduce noise interference and refine the boundary details of cracks by learning different receptive fields in each layer. Besides that, we created a dataset consisting of tiny crack samples collected through image processing and manual labeling from industrial production, named TinyCrack. The proposed HLCF model is evaluated on TinyCrack and seven public databases. The experimental indices show that the model achieves Precision over 0.62%, Recall over 0.12%, F-score over 0.53%, and IOU over 1.12% in TinyCrack, demonstrating the best accuracy compared with other public crack detection models. The results indicate that the HLCF detection model has better performance in identifying low-visibility tiny cracks.

Few-shot classification aims to develop a classifier that adapts to new tasks using only a limited number of labeled images. To overcome the limitation of lacking training images in few-shot image classification, dense features have been extensively utilized to represent images by providing more subtle and discriminative clues. However, dense feature based methods are still facing challenges despite leveraging local details in images. Primarily, these methods deal with the support set images in each category independently, which ignores the information across different categories. Furthermore, dense features suffer from background noise, when performing similarity calculations based on a large number of dense feature pairs, these methods are susceptible to interference from task-irrelevant feature pairs. In this paper, we propose a cross dense feature learning with task guidance method to address the aforementioned issues. The key components of our method include two aspects. Firstly, a dense feature extraction approach based on transformer is proposed, aiming to better utilize inter-class information within the support set. We design two types of cross-attention mechanisms to get the across information among different categories for a better representation of dense features, named Support-Support Attention (SSA) and Support-Query Attention (SQA). Secondly, a task-relevant model is trained for dense feature pairs similarity calculating, aiming to filter out feature pairs that contribute more effectively to classification. Then we can get the final similarity to predict the label of query image through summarizing weighted local similarity. The experimental results prove that our method achieves a promising improvement for few-shot classification by taking information across different categories and task attention similarity into consideration.

In[1], inaccuracies in several critical equations along with their accompanying descriptions appear in the article. Furthermore, some references are missing, and certain analyses of experiments are flawed.

Hardness is one of the most critical tactile properties for robots to recognize objects. Machine learning methods have shown superior performance in object hardness classification. However, existing machine learning methods for tactile hardness classification cannot use the ordinal information between hardness classes because the one-hot encoding only cares about the correct class and ignores the inter-class relationship. To solve this problem, we propose to generalize the one-hot encoding using unimodal distributions including the Poisson and binomial distributions for tactile ordinal classification problems, resulting in two tactile ordinal networks (TacONet): TacONet-p and TacONet-b. Furthermore, we collect a tactile hardness dataset on the silicone samples with three different shapes (Shapes A, B, C), and each shape samples have thirteen hardness classes ranging from 0A (Shore A scale) to 60A at 5A intervals. We validate the resulting method for tactile hardness classification using a real robot. Experimental results demonstrate that compared with state-of-the-art methods, the proposed method achieves better classification performance in terms of accuracy and quadratic weighted kappa (QWK) on the tactile hardness dataset, reaching a classification accuracy up to 99.5% and a QWK up to 99.9% on Shape C. Note to Practitioners—In the field of robotics tactile recognition, hardness classification is one of the most important and common tasks for robots to accurately recognize objects, particularly when the environment is dark or visual sensors are not working. In this paper, we propose a novel tactile ordinal network for tactile hardness classification tasks. The existing machine learning models for tactile hardness classification are trained by minimizing the cross-entropy loss between predicted vectors and one-hot encoding vectors of true classes, which makes the models only care about the correct classes and ignores the inter-class relationship of hardness classes. In other words, these models have the same probability to misclassify a hardness class with any other hardness class. To tackle this problem, we propose to generalize the one-hot encoding method using a unimodal distribution method to encode the true classes. The unimodal distribution encoding vectors can make the model learn the ordinal information between classes. It is proved that the proposed method is able to effectively improve the classification accuracy and QWK in a tactile hardness classification task.

The purpose is to optimize the Beetle Antenna Search (BAS) algorithm and apply it to the Intelligent Transportation System (ITS) to process traffic data in time and solve traffic congestion. This work studies the development status of ITS and the application status of the BAS algorithm. It optimizes BAS to converge to local optimization prematurely in high-dimensional space, affecting the prediction accuracy. Then, combined with the Least Squares Support Vector Machine Algorithm (LSSVM), the algorithm with quadratic interpolation optimization is proposed. The proposed algorithm is named the Quadratic Interpolation Beetle Antenna Search (QIBAS). On this basis, a traffic flow prediction model based on QIBAS-LSSVM is established. Finally, the improved QIBAS algorithm and Traffic Flow Prediction (TFP) model are verified. The results show that the test Mean Square Error (MSE) of the TFP model based on QIBAS-LSSVM increases by 4.28%, 7.38%, and 18.23%, respectively compared with the other three models. The test Mean Absolute Percentage Error (MAPE) increases by 0.09%, 0.06%, and 0.36% respectively. The proposed QIBAS algorithm has a good effect and high accuracy in short-term TFP. The research has important reference value for the digital transformation of transportation systems in modern smart cities.

This work aims to improve the transmission and sharing efficiency of intelligent transportation data and promote the further development of intelligent transportation and smart city. In this work, an EEMR (Energy Efficient Multi-hop Routing) is designed for the intelligent transportation wireless sensor network. In the EEMR algorithm, the base station runs the IAP clustering algorithm for network clustering after receiving the information of all surviving nodes. A method called ACRR (Adaptive Cluster-head Round Robin) is proposed for local dynamic election of cluster heads. In addition, the deep learning-based stochastic gradient descent algorithm and its evolution algorithm are sorted out, and its application in practical scenarios is analyzed. Due to the disadvantages of the adaptive algorithm in the current data processing process, the gradient optimization algorithm based on deep learning is adopted and the concept of adaptive friction coefficient is applied to the Adam algorithm to obtain a new adaptive algorithm (TAdam). The simulation experiment reveals that the proportion of network surviving nodes of the EEMR algorithm is still as high as 90% in 1,500 rounds of data collection. This shows that the EEMR algorithm achieves the energy balance of the network nodes as much as possible while minimizing the system energy consumption. In application scenario 1, when the network surviving nodes of the four data collection algorithms compares dropped below 40%, the number of surviving nodes in the EEMR algorithm is still as high as 97.6%. On the PTB (Penn Tree Bank) test data set, the TAdam algorithm shows the fastest convergence speed and the best generalization performance. The TAdam algorithm based on deep learning discussed in this work was of great significance for improving the transmission and sharing efficiency of intelligent transportation data.

Untethered miniature robots that can access narrow and harsh environments in the body show great potential for future biomedical applications. Despite the many types of millirobot that have been developed, swimming against the fast blood flow remains a big challenge due to lack of the ability to stay still and the large fluidic resistance from blood. This article proposes an on-wall-rotating strategy and a streamlined millirobot to achieve effective upstream motion in the lumen. First, the principle of on-wall-rotating strategy and the dynamic motion model of the millirobot is established. Then, a critical safety angle$\theta _{s}$is theoretically and experimentally analyzed for the safe and stable control of the robot. After that, a series of experiments are conducted to verify the proposed driving strategy. The results suggest that the robot is able to move at a speed of 5 mm/s against flow velocity of 138 mm/s, which is comparable to the blood flow of 2700 mm$^{3}$/s and several times faster than other reported driving strategies. This work offers a new strategy for the untethered magnetic robot construction and control for blood vessels, which would promote the application of millirobot for biomedical engineering.

Aiming at the problem that it is difficult to calculate the force of permanent magnets in the magnetic field, this paper proposes a nonlinear mechanical model of linear array magnetic field based on radial basis function neural network (RBFNN). Combined with the linear Halbach array adsorption module of the wall-climbing robot, the three-dimensional geometric magnetic fields of four typical linear array permanent magnets were constructed, and the theoretical models of the interaction between the magnetic fields were given respectively. Further, the finite element simulation calculation of the magnetic force was carried out using COMSOL Multiphysics. According to the parametric scanning results of the orthogonal test, a nonlinear intelligent prediction model of the force between magnetic fields with local loss sensitivity is established by using the RBFNN numerical fitting method. The average deviation of the network test set is 1.19, and the standard deviation is 0.80. The intelligent prediction model has strong generalization performance, faster convergence speed and stronger flexibility, which provides a theoretical basis for the interaction and control of array magnetic fields.

Touch modality identification has attracted increasing attention due to its importance in human–robot interactions. There are three issues involved in the tactile perception for the touch modality identification, including the high dimensionality of tactile signals, complex tensor morphology of tactile sensing units, and the misalignment among different tactile time-series samples. In this article, we propose a novel kernel-based approach to deal with these three issues in a unified framework. Specifically, the techniques, including sparse principal component analysis and subsampling, are employed to reduce the feature dimension. Then, a singular value decomposition (SVD)-based kernel is proposed to preserve the spatial information of the tactile sensing elements. The sample misalignment issue is addressed via the employment of a global alignment kernel. Moreover, the merits of these two kernels are fused through an ideal regularized composite kernel, which simultaneously takes the label information of the training set into consideration. The effectiveness of the proposed kernel-based approach is verified on a public touch modality data set with a comprehensive comparison with the competing methods.Note to Practitioners—In a wealth of tactile recognition tasks, we are in the face of various challenges. For instance, tactile measurements are commonly tensorial and high-dimensional. The misalignments among tactile measurements prevail, such as different durations of tactile measurements and the misaligned starting time point of tactile measurements. This article presents a kernel-based method using an ideal regularized composite kernel to deal with all challenges in a unified framework. The kernel-based method consists of two key components including the SVD-based kernel and the global alignment kernel. The proposed method may shed new insights on new advances in tactile signal processing particularly in human–robot interactions.

Sensor drift is one of the severe issues that gas sensors suffer from. To alleviate the sensor drift problem, a gas sensor drift compensation approach is proposed based on local discriminant subspace projection (LDSP). The proposed approach aims to find a subspace to reduce the distribution difference between two domains, i.e., the source and target domain. Similar to domain regularized component analysis (DRCA) which is a recently proposed sensor drift correction method, the mean distribution discrepancy is minimized in the common subspace in our approach. LDSP extends DRCA in two aspects, i.e., it not only takes the label information of the source data into consideration to reduce the possibility of the case that samples in the subspace with different class labels stay close to each other, but also borrows the idea of locality-preserving projection to deal with multimodal data. Specifically, inspired by local Fisher discriminant analysis (LFDA), the label information is utilized to maximize the local between-class variance of source data in the latent common subspace and simultaneously minimize the local within-class variance. The formulation of LDSP is a generalized eigenvalue problem that can be readily solved. The experimental results have shown the proposed method outperforms other gas sensor drift compensation methods in terms of classification accuracy on two public gas sensor drift datasets.

This article presents a novel discriminative subspace-learning-based unsupervised domain adaptation (DA) method for the gas sensor drift problem. Many existing subspace learning approaches assume that the gas sensor data follow a certain distribution such as Gaussian, which often does not exist in real-world applications. In this article, we address this issue by proposing a novel discriminative subspace learning method for DA with neighborhood preserving (DANP). We introduce two novel terms, including the intraclass graph term and the interclass graph term, to embed the graphs into DA. Besides, most existing methods ignore the influence of the subspace learning on the classifier design. To tackle this issue, we present a novel classifier design method (DANP+) that incorporates the DA ability of the subspace into the learning of the classifier. The weighting function is introduced to assign different weights to different dimensions of the subspace. We have verified the effectiveness of the proposed methods by conducting experiments on two public gas sensor datasets in comparison with the state-of-the-art DA methods.

In the context of robotic high-precision soldering, we propose an image-based pin alignment control method based on active plastic deformation. The plastic deformation is a well-known failure mechanism in most situations, which includes a phenomenon that the objects do not return original state. Here, in contrast to this convention, we utilize the plastic deformation of the metal pin to do pin alignment for improving the quality of the solder joint. To address this, we embed the springback compensation into the image-based pin alignment controller. Lastly, the proposed strategy is successfully demonstrated and evaluated in a practical modified robotic manipulation system. The result shows that the alignment error is less than 20µm, which is far less than pin alignment without considering plastic deformation and elastic recovery. This work considers active plastic deformation and spontaneous elastic recovery of soft object, which would greatly promote the use of robotics in micromanufacturing and microfabrication in lad and industry, especially for soft objects.

Steering microswimmers toward 3-D autonomous manipulation tasks has received extensive attention. Our previous works have accomplished autonomously manipulating microswimmers in the 2-D space. This article aims to extend the 2-D autonomous manipulation to 3-D autonomous manipulation. Specifically, this article addresses the problem of an autonomous system that consists of 3-D path planning and 3-D path following for magnetically driven helical microswimmers. The path-planning algorithm called optimal Bidirectional RRT* is formulated to explore the shortest route in the confined 3-D space. A proxy-based sliding mode control (PSMC) approach is developed to design stable controllers based on the error model in the Serret–Frenet frame. We transport the swimming model trained by a kind of neural network to another new helical microswimmer according to an online updating scheme. The updating scheme can identify and refine compensating angles between the swimming direction of the microswimmer and the magnetic direction in the 3-D space facing the weight disturbances of the swimmer and lateral disturbances. The experiments are conducted to quantitatively validate the 3-D autonomous manipulation system. Experimental results show the effectiveness of path planning and path following with submillimeter accuracy in a 3-D space. Future works will focus on autonomous manipulations in dynamic environments.Note to Practitioners—This article is motivated by the issue of 3-D autonomous manipulation tasks for magnetically driven helical microswimmers. The formulated path planning is responsible for finding the shortest route in the 3-D confined space. The closed-loop controller is charge of steering the helical microswimmers on a reference path based on an online updating model trained by neural networks. It is demonstrated that the helical microswimmer can find the shortest path and follow it in a 3-D space with submillimeter accuracy.

Tactile sensing with spiking neural networks (SNNs) has attracted increasing attention in the past decades. In this article, a novel SNN framework is proposed for the tactile surface roughness categorization task. In contrast to supervised SNN methods such as ReSuMe and Tempotron that require prespecifying target spike trains, the presented method performs the classification through directly comparing the distance between multineuron spike trains. Unlike simple spike train fusion methods using average pairwise spike train distance or pooled spike train distance, the proposed method merges spike trains from different neurons with the multineuron spike train distance, which can capture the complex correlation of multiple spike trains. Specifically, the spike trains are generated via the Izhikevich neurons from tactile signals. The similarity of the multineuron spike trains is computed using the multineuron Victor–Purpura spike train distance, which can be efficiently implemented in an inductive manner. The classification can be performed by incorporating$k$-nearest neighbors and the multineuron spike train distance as a similarity metric. The proposed framework is quite general, i.e., other multineuron spike train distances and spike train kernel-based methods can be readily incorporated. The effectiveness of the proposed method has been demonstrated on a tactile data set by comparing it with various feature- and spike-based methods.Note to Practitioners—In the soft neuromorphic implementation of biomimetic tactile sensing and the development of the tactile sensing capability in neurobotic systems, the processing and analysis of spike-like tactile signals are quite common. Inspired by human tactile perception, this article proposes a novel supervised spiking neural network method for tactile sensing tasks. The traditional methods have to prespecify target spike trains, which is still an open question. In addition, the current ways to fuse spike trains from multiple neurons are far from mature. This article tackles these two problems using spike train similarity comparison with multineuron spike train distance. The direct spike train similarity comparison avoids the need to prespecify target spike trains. The multineuron spike train distance can inherently fuse spike trains from different neurons. It is demonstrated that the proposed method is able to effectively perform classification in a tactile roughness discrimination task.

High flexible and high precise manipulation is one of the most critical technique for complex microsystem’s measurement, manufacture, and assembly. Although recent advances in microrobotics have successfully realized the automatic manipulation and positioning of tiny objects, their flexible manipulation in 3-D free space remains a challenge, such as the wide-angle rotation manipulation of microsize sphere, due to the complicate surface forces. Herein, this article proposed a feedforward model and realized the precise rotation for microsized sphere by two cooperative flexible micropipettes. Firstly, a microrobotic manipulation system with six degrees-of-freedom (DOFs) was developed and integrated with the microscope. Then, a feedforward compensation control strategy involving dual rotation was proposed for the precise manipulation of microsized sphere (${\sim }90~\mu \text {m}$) based on the analysis of contact forces. As a result, the rotation of the microsized sphere in two different planes was realized and the microsized sphere release procedure was also accomplished after rotation. Compared with existing techniques only allowing limited amplitudes rotation, this article realizes wide-angle rotation manipulation of microsized sphere in 3-D free space. This research opens new prospects for the microsized object accurate manipulation, which is expected to give a long-term impact for complex microsystem’s manufacture and assembly.Note to Practitioners—This article is motivated by the problem of the flexible manipulation of tiny object in 3-D space. The proposed nanorobotic manipulation system, two micropipettes and feedforward compensation model control strategy could realize the precise translational and rotational manipulation of microbeads in 3-D space, which offers obvious advantages of existing techniques. The proposed system and method could be a general solution for precise and flexible micromanipulation. Thus, it could find wide applications ranging from fundamental research to industrial applications, such as biological cell positioning, characterization of a particular micro/nanoregion, microassembly, and manufacturing.

High-precision positioning is an essential requirement for sample operation at a small scale. At the current stage, although nanometer-scale accuracy has been achieved for the linear positioning, the rotational positioning (attitude control) is still very challenging and rarely addressed. This paper presents a rotatable nanorobotic system with rotational degrees of freedom first. Then, the system error, i.e., nonaxisymmetrical eccentricity error of the mechanism, is investigated dynamically and its fault model is established. After that, a double-loop servo repetitive controller is accordingly designed based on the circle interpolation strategy. The theoretical analysis and experimental results verify that the rotational positioning accuracy can be controlled up to submicrometers stably, which improves at least one order of magnitude than the current static method. Finally, two application cases are given to highlight the significance of this approach, i.e., surface defect detection from 360° and in situ twisting characterization of 1-D micro/nanomaterial. This research paves a new avenue for the ultrahigh rotational positioning at microscopy environment, which is expected to generate a long-term impact on the micro/nanofields, such as microscopy imaging, material characterization, and so on.

Microscopy has been an indispensable tool for micro/nanosample imaging, manipulation, and characterization. However, viewing the micro/nanosample from multidirection is still a big challenge for current microscopy. To address the above issue, this paper proposes a novel nanorobotic manipulation system for the automatic alignment and multidirectional imaging under microscopes. First, a miniature rotation robot with three degrees of freedom is designed and integrated with a microscope. Then, a forward-backward alignment strategy containing three loops, i.e., position shift loop, angle loop, and magnification loop, is proposed to align the sample to the rotation axis of the robot automatically. After that, the sample is imaged from multidirection by rotating the robot with one revolution (360°). Finally, the alignment accuracy is evaluated and multi-directional images of various samples are implemented. This study provides a new way for the microscopic imaging, which is expected to exert a significant impact in multiple fields on a small scale, including microscopy imaging, microdefect detection, micromanipulation, in situ characterization, and so on.