For inspection robots to achieve generalizability, stability, and ease of use, it is crucial that they understand natural language commands and navigate accurately to specified target objects. We propose L-SNI, a semantic navigation system adapted for inspection tasks, offering generalizability, robust stability, and practical ease of use. In the perception phase, L-SNI constructs a precise geometric depth map of the environment using LiDAR, while RGB images are employed to extract object categories, which are then combined with depth data to generate a semantic map. To enable the large language model (LLM) to interpret the environment, L-SNI encodes the 3D semantic map into a plain text representation. During single-task execution, L-SNI decodes human commands into inspection primitives using an LLM constrained by system initial prompts. These inspection primitives guide the robot’s low-level planner for task execution. To address the challenge of traditional 3D LiDAR localization and navigation systems in accurately positioning the robot around target objects during inspection tasks, we propose a target cost gradient to assist in optimizing the robot’s target point selection and attitude control in maps with semantic information. Upon reaching the target, L-SNI uses a visual language model (VLM) to describe the scene, which is simplified by the LLM into a user-friendly response. Through testing on 18 indoor scenes from the Matterport 3D dataset, L-SNI achieves a 46.9% improvement in Success Rate (SR) and a 58.3% increase in Success weighted by Path Length (SPL) over existing state-of-the-art (SOTA) solutions, while also demonstrating superior target image understanding. Moreover, it can be easily deployed on real-world robots without complex initialization.

Robotic tactile sensors based on Electrical Impedance Tomography (EIT) have gained great attention in robotic sensing applications due to their features such as no internal wiring, "all-in-one" structure, and continuous sensing capabilities. However, the effectiveness of EIT-based tactile sensors is hampered by limited spatial resolution and artifacts in the reconstructed images. To address these challenges, various iterative optimization methods based on spatial regularizations and model-based methods have been proposed. In this study, a new EIT reconstruction method using null-space decomposition based on a diffusion model (NSDM) is proposed. Specifically, NSDM consists of a forward diffusion process that first gradually adds Gaussian noise to a clean conductivity image, followed by a backward process that learns to predict the noise that should be removed during each sampling step, utilizing a prior to ensure that the denoising process does not deviate from the correct direction. NSDM requires no training, no optimization, and only requires a pre-prepared diffusion model. Experimental results (both simulation and actual tests) demonstrate that the proposed method outperforms existing generation methods and provides higher quality reconstruction, providing a new solution for robotic tactile sensing in real scenarios.

Electrical Impedance Tomography (EIT) is a highly ill-posed inverse problem, with the challenge of reconstructing internal conductivities using only boundary voltage measurements. Although Physics-Informed Neural Networks (PINNs) have shown potential in solving inverse problems, existing approaches are limited in their applicability to EIT, as they often rely on impractical prior knowledge and assumptions that cannot be satisfied in real-world scenarios. To address these limitations, we propose a two-stage hybrid learning framework that combines Convolutional Neural Networks (CNNs) and PINNs. This framework integrates data-driven and model-driven paradigms, blending supervised and unsupervised learning to reconstruct conductivity distributions while ensuring adherence to the underlying physical laws, thereby overcoming the constraints of existing methods.

Electrical impedance tomography (EIT)-based tactile sensor has shown great potential in human–machine interaction due to its low manufacturing cost, large-area scalability. However, challenges, such as limited spatial resolution, and artifacts in reconstructed images, hinder their effectiveness. In response, this study proposes a model-driven deep learning reconstruction network for EIT-based tactile sensing, named PDCISTA-Net. The framework integrates a preprocessing filtering module and a dual-channel iterative shrinkage-thresholding algorithm (ISTA). Unlike traditional ISTA, PDCISTA-Net employs a dual-channel structural network tailored to capture and represent block correlations and sparsity within impedance change distributions. This approach enables end-to-end training, where parameters, such as step size, nonlinear transforms, and shrinkage thresholds, are learned from generated training data. In addition, a novel filtering module based on the sensitivity matrix is introduced to enhance reconstruction quality by mitigating measurement noise. Numerical metrics and visual results show that PDCISTA-Net outperforms traditional Newton's one-step error reconstructor, total variation, ISTA-Net, and FISTA-Net methods with higher structural similarity index measure and peak signal-to-noise ratio. Ablation experiments verified the effectiveness of the dual-channel structure in improving reconstruction quality. Finally, we developed an EIT-based tactile system to validate the practical application of our approach. The results from real-contact detection demonstrate enhanced image quality and greater noise robustness compared to traditional reconstruction methods.

Electrical Impedance Tomography (EIT)-based tactile sensors offer durability, scalability, and cost-effective manufacturing. However, simultaneously reconstructing force and shape from boundary measurements remains challenging due to EIT’s inherent location dependencies and image artifacts. This study presents a model-driven multimodal convolutional neural network (MM-CNN) for joint EIT-based force and shape sensing. The hybrid approach combines physics-inspired voltage preprocessing with an attention-based network to overcome EIT’s limitations. The preprocessing network applies a linearized one-step inverse solution with Tikhonov regularization to convert raw boundary voltage into a noise-reduced 2D image. The image reconstruction network uses an attention mechanism to focus on salient features, addressing location dependency issues. Quantitative metrics show that MM-CNN outperforms traditional EIT algorithms like NOSER and TV, reducing location dependency and improving shape discrimination. MM-CNN enables unified force and shape modalities, validated through real-contact experiments, enhancing EIT tactile systems for human-robot interaction by incorporating physical knowledge with deep learning.

This paper propose a novel transfer learning approach, Pseudo-Domain Adversarial Network (PDAN), to tackle the issue of electrode displacement in Electrical Impedance Tomography (EIT). Electrode displacement, caused by human movement or improper operation, significantly affects the accuracy of EIT by introducing data errors. Existing solutions either modify the electrode assembly at a high cost or employ recognition algorithms that require retraining from scratch. To overcome these limitations, our work leverages the power of transfer learning to enhance model performance in the target domain by utilizing knowledge from a related task in the source domain. PDAN extends the capabilities of deep adversarial learning by incorporating noisy images to simulate post-electrode rotation scenarios, aiding in the reduction of negative impacts caused by minor electrode displacements. Our method demonstrates superior performance in classifying leg posture data, achieving around 90% accuracy, and proving robust against sensor electrode offset. Experimental results across various datasets validate the effectiveness of PDAN, indicating its potential in addressing complex real-world situations with improved generalization capabilities.