Acupuncture, the core part of Traditional Chinese Medicine, relies on the accuracy of acupoint detection. But most of existing detection networks can’t achieve precise acupoint detection, whose challenges arise from posture-induced occlusion or edge effects, background clutter, and the lack of prominent surface cues for most acupoints. To address the issues, an Region-Aware Implicit Feature Extraction Network (RAIFE-Net) is proposed to enhance feature awareness and spatial reasoning for precise acupoint detection. RAIFE-Net has three fundamental components. Firstly, a Evolvable Gradient Convolution is designed to perceive gradient variations to refine edge features, which can motivate model to precisely recognize the acupoints appear near the edge of body. Secondly, the Saliency-Driven Feature Selector is able to highlight salient feature of acupoint regions by generating an attention mask which is available to suppress the interference of clutter background. At last, a Structured Position-Aware Relational Loss is developed to optimize spatial perception by constructing coordinate relationship for acupoint positions, enabling the network to accurately infer the relative positions among acupoints and improve detection precision. In the experiments, the proposed RAIFE-Net achieves state-of-the-art performance on the clinical acupoint dataset constructed for this study and public dataset.

Ultra-High Frequency Radio Frequency Identification (UHF RFID) is widely used for item-level traceability. Existing approaches often exploit analog features of backscattered signals to sense a single environmental factor, typically assuming static measurement setups. In this paper, we categorize antenna-sensitive factors into two types: time-independent and time-dependent, and propose a classification algorithm to decouple them from time-series analog features. First, we apply Dynamic Time Warping (DTW) Barycenter Averaging to cluster features under different power levels for each time-independent factor, constructing offline models. Second, we combine DTW similarity between online measurements and offline references with model-based state estimation to identify the time-independent factor. Then, the time-dependent factor is estimated using a sliding-window cumulative probability method. We validate our approach through an infusion monitoring system using a flexible passive UHF RFID tag attached externally to the infusion tube. Here, liquid level represents the time-dependent factor, while liquid material is the time-independent factor. Experimental results demonstrate that our method accurately identifies both liquid level and material under varying measurement setups.

Due to the low efficiency of manual inspection of pointer pressure gauges, numerous computer vision-based automatic reading recognition systems have been proposed in recent years. However, most existing work primarily focuses on using cameras or inspection robots for daily monitoring of gauges. Periodic calibration of gauges is equally important, as it ensures their optimal performance and imposes stricter precision requirements on inspections. To address these challenges, this study proposes a lightweight multitask interactive network (LMI-Net) and develops an intelligent robotic-assisted measurement system. LMI-Net, which generates a large number of feature maps through computationally inexpensive operation of a lightweight module, realizes multiple tasks via keypoint detection and instance segmentation branches. The keypoint detection branch identifies the position of the main scale and the center of the gauge, while the instance segmentation branch generates a segmentation mask of the pointer. Through our proposed multiple feature fusion (MFF) reading method, the value and order of the main scale are identified, the segmentation mask of the pointer is fitted to a straight line using a regional interpolation linear fitting method, and on the basis of geometric relationships, these pieces of feature information are fused to obtain a reading. Experimental results on a dataset containing multiple types of gauge demonstrate that with the proposed method, the reading error is controlled to within ±1/5 of the minimum resolution interval, which complies with the verification regulation for pointer pressure gauges.

Acupuncture plays an important role in the treatment of diseases, prevention, and health care. However, it is difficult to detect acupoints from the hand surface due to the complicated tasks: first, the acupoint regions are similar to the surrounding areas on the hand surface; second, the skin tone on the hand varies in depth. To address these challenges, an adaptive patch-correlation attention network (APCA-Net) is proposed for vision-based human hand acupoint detection in acupuncture, where the adaptive patch aware attention module is designed to distinguish acupoint from the surrounding skin tissue by capturing fine-grained features from small patches, while the adaptive correlation channel attention module is devised to detect acupoints of different skin tones by modeling channel correlation. Experiments on benchmarks show that the proposed APCA-Net can achieve the acupoint detection rate of 76.11% and 88.32% on the 11 K Hands Subset and Depth Hand datasets, respectively, outperforming other state-of-the-art approaches.

In minimally invasive surgery, surgical instruments are typically inserted through small incisions in the patient’s body, which serve as the remote center of motion (RCM). In robot-assisted minimally invasive surgery, developing control algorithms that comply with RCM constraints is highly challenging due to the nonlinear nature of robot motion models and the stringent precision requirements necessary for patient safety. This article introduces an enhanced zeroing neural network (EZNN) model for controlling redundant manipulators while maintaining RCM constraints. The proposed model eliminates the need for pseudoinverse matrix computations and features an explicit dynamic form. It guarantees finite-time convergence and demonstrates robustness through the application of nonlinear activation functions (AFs). These properties are rigorously validated using Lyapunov theory. Simulation and experimental results indicate that the EZNN model surpasses Jacobian-based algorithms and recurrent neural networks (RNNs) in terms of efficiency and stability, all while ensuring adherence to RCM constraints.

A promising ultrasound (US) image acquisition requires experienced sonographers holding the probe with proper force and pose to ensure an excellent acoustic coupling. To enable a robotic ultrasound system (RUSS) to acquire the sonographers’ skills from ultrasound image demonstrations, this paper proposes a cutting-edge framework that integrates an expert technique discrimination network and a robotic strategy generation network to learn expert scanning skills. In this framework, the expert technique discrimination network focuses on learning expert scanning techniques from the pre- and post-frame ultrasound images. Furthermore, to acquire expert scanning skills and obtain a standard image view, we design a knowledge-based algorithm grounded on inverse reinforcement learning (IRL) to generate a series of scanning policies concerning the expert technique discrimination network. Both simulations and experiments are conducted to validate the effectiveness of the proposed framework by comparing it with MI-GPSR and PTR. The scanning success rate and trajectory tracking error of the algorithm in the simulation environment are 68% and 12.0782, respectively, while in the phantom environment are 94% and 11.8367. The results demonstrate good performance in the task of imitating expert techniques for autonomous scanning. Note to Practitioners—The motivation for this work originates from the need for ultrasound scanning tasks, such as carotid plaque and thyroid scans, to follow specific procedures. In clinical practice, sonographers require extensive learning and training to acquire these scanning skills. Traditional autonomous robotic ultrasound research often focuses on achieving the final standard view, neglecting the logical flow of the scanning process. Moreover, current studies on imitation learning for robotic ultrasound typically require not only ultrasound images from expert demonstrations but also additional data like probe position, adding complexity and potential errors to the data collection process. The proposed method addresses this by designing a framework that learns and comprehends expert techniques solely from image demonstrations. This endows RUSS with the ability to perform a human-like ultrasound scanning task. This work can be applied in the field of autonomous ultrasound robotics to assist sonographers in achieving more precise scans.

Supernumerary robotic limbs (SRLs) can significantly enhance human manipulation capability. However, it is difficult to achieve a safe and friendly interaction for collaboration tasks in complicated and dynamic external environments, and it easily leads to safety issues between a human user and the SRL robot. To address the aforementioned issues, this paper designs a new type of SRL and proposes a teleoperated control method for human-robot safe interaction. Specifically, inspired by the principle that human arms can autonomously adjust their stiffness to safely interact with the external environment, this paper proposes an SRL interaction control method based on the variable stiffness of the human upper limb. Specifically, the proposed variable stiffness control parameters can be adaptively updated based on the characteristics of surface electromyography (sEMG) signals from the human upper limb. The effectiveness of the proposed framework is validated through experimental results.

Automatic segmentation of 3D dental models into individual teeth is an important step in orthodontic computer-aided design (CAD) systems. However, most existing methods rely on single-view dental models and ignore the intrinsic relationships between upper and lower dental models, hindering the handling of complex tooth structures. In this paper, a collaborative learning framework with coupling graph Transformers (CGT-CLF) is proposed for automatic tooth segmentation on 3D dental models. The framework collaboratively learns geometric features of both upper and lower dental models, capturing their interactivity and complementarity by facilitating interaction between graph-Transformer encoders to improve segmentation of complex and diverse teeth. Specifically, CGT-CLF consists of three key components as follows: First, a graph embedding-based boundary perception module (GEBPM) is developed to aggregate fine-grained geometric features within the neighborhood graph domain, enhancing the network's ability to perceive and distinguish intricate tooth boundaries. Then, coupling geometric Transformers are designed to capture the intrinsic relationships of pair-wise dental models by promoting the exchange of relevant information to gain a comprehensive understanding of the overall tooth structure, allowing for better identification of adjacent teeth with similar appearances. Finally, a collaborative cross-scale feature fusion (CCFF) strategy is utilized to obtain interactive and complementary information by modeling the inter-relationships between dual-stream features. Experimental results on a clinical dental model dataset demonstrate that the proposed CGT-CLF framework outperforms state-of-the-art methods, delivering superior segmentation performance.

In order to obtain a high-precision contact model that can properly describe the target soft tissue, this paper proposes a hybrid soft contact model based on a clustering algorithm ART-II, which selects the most suitable soft contact model according to the surgical environment. The least-square method is used to identify the parameters of the model online. In the experiments, different parts of animal tissues were used as the experimental objects. The hybrid model was used to identify and switch for the most appropriate soft contact model when dealing with a certain type of animal tissue. The performance of the hybrid model on force estimation was compared with several individual soft contact models. The results showed that the estimated/reconstructed force of the hybrid model was closer to the ground truth measured by the force sensor. In addition, a new reference soft contact model has been purposely added online to verify the expandability of the hybrid model.

Gesture recognition is an essential part in the field of human–computer interaction (HCI) and Internet of Things system. Compared with the existing technologies based on wearable sensors and dedicated devices, approaches using WiFi channel state information (CSI) signals are more desirable for passive and fine-grained gesture recognition. However, the existing CSI-based gesture recognition systems usually suffer from high model complexity and low accuracy caused by environmental dynamics. To address these issues, we propose a robust gesture recognition system (WiHGR) in this article. The WiHGR starts with a sparse recovery method to find the dominant paths from the multipath effect introduced by the orthogonal frequency division multiplexing (OFDM) technology, i.e., the main propagation paths disturbed by a human gesture. Then, the phase difference matrix is constructed according to the phase differences between two adjacent receiving antennas from the dominant paths. We propose a modified attention-based bi-directional gate recurrent unit (ABGRU) network to learn and extract discriminative features automatically from the phase difference matrix. The proposed attention mechanism assigns higher weights to the more important features, thus achieving a better recognition performance. The experimental results show that the WiHGR not only has a high accuracy for gesture recognition in the training environment, but also has a remarkable performance in new environment settings without retraining.

How to enable robotic compliant manipulation has become a critical problem in the robotics field. Inspired by a biomimetic adaptive control strategy, this article presents a novel representation model named human-like compliant movement primitives (Hl-CMPs) which could allow a robot to learn human-like compliant behaviors. The state-of-the-art approaches can hardly learn complete compliant profiles for a specific task. Comparatively, our model can encode task-specific parametric movement trajectories, correspondingly associated with dynamic trajectories including both impedance and feedforward force profiles. The compliant profiles are learned based on a biomimetic control strategy derived from the human motor learning in the muscle space, enabling the robot to simultaneously learn the impedance and the force while executing the movement trajectories obtained from human demonstration. Furthermore, both the kinematic and the dynamic profiles are learned in the parametric space, thus enabling the representation of a skill using corresponding parameters (i.e, task-specific parameters). Hl-CMps can allow the robot to automatically learn compliant behaviors in an online manner after kinematic demonstration. Our approach is validated by an insertion task and a cutting task based on a KUKA LBR iiwa robot.

In this paper, an improved motion planning scheme is proposed for surgical robot control with multiple active constraints, including joint constraints, joint velocity constraints and remote center of motion constraints. It introduces an improved recurrent neural network (RNN) to optimize the online motion planning respect to multiple constraints. The demonstrated surgical operation trajectory is derived using teaching by demonstration. An improved motion planning scheme using the novel recurrent neural network is then designed to achieve the accurate task tracking under the multiple constraints. The general quadratic performance index is adopted to represent the constraints. Finally, the effectiveness of the proposed algorithm is demonstrated using KUKA LWR4+ robot in a lab setup environment.

Accurately detecting tumors and estimating the depth of tumors is essential in the surgical removal of tumors. In robotic-assisted surgery, autonomous robotic palpation has the potential to provide more precise detection, tumors' depth estimation, and less intrusion when normal tissues surround tumors. In this article, by mimicking the human finger touch, we propose a tactile sensing-based deep recurrent neural network (DRNN) with long short-term memory (LSTM) architecture to improve the accuracy of the detection and depth estimation of tumors embedded in soft tissue. In the experimental setup, the hard inclusions simulate the tumors, while the phantom tissue is fabricated by silicon to simulate the soft tissue. During the experiment, the data from the force sensor and displacement of the robot palpation probe are for detection and depth estimation purposes. The collected sequential data set of the force and the displacement of the probe during one completed palpation process will go through the proposed DRNN network with deep LSTM architecture, in which the temporal dependencies of the sequential data will be captured in the cell states in the deep LSTM layers. Subsequently, the softmax classifier is adopted to determine if there is any hard inclusion exists and offer the depth estimation of the hard inclusions. Experiments based on 396 real data sets demonstrate that the detection accuracy for the testing data set is 99.2% and the depth estimation accuracy for the testing data set is 95.8%. The accuracy of the proposed method is best when comparing with other widely used methods. Note to Practitioners-The palpation of tumors motivated this article in the robot-assisted surgical systems through tactile feedback. In order to mimic the human touch on the soft tissue, this article presents a deep-learning-based approach to estimate the depth of the hard inclusions in the phantom tissue through force information. The displacement of the palpation probe and the touch force during one palpation are recorded as data sequences to train the deep model, which aims to capture dynamics and long-term dependence of the palpation process. In this article, we made the first successful attempt to accurately estimate the depth of the hard inclusions buried at different locations of the phantom tissue using only force information. The proposed approach can work in different robot-assisted scenarios, such as master-slave robotic surgery. In the clinic applications, the force sensor will be integrated at the end-effector of the robotic manipulator. According to the specific requirements, the force sensor and the robotic manipulator might be different from those used in this article. For some applications, such as the laparoscopic interventions, the complete vertical contact tends to be difficult to obtain due to the laparoscopic port effects. The projection of the recorded force data and displacement can obtain the information in the normal direction. The future work is going to be extended to tissue environments with arbitrary surface and tumors with various shapes/depths for more complex and prospective clinical applications.

Human activity recognition is a core problem in intelligent automation systems due to its far-reaching applications including ubiquitous computing, health-care services, and smart living. Due to the nonintrusive property of smartphones, smartphone sensors are widely used for the identification of human activities. However, unlike applications in vision or data mining domain, feature embedding from deep neural networks performs much worse in terms of recognition accuracy than properly designed handcrafted features. In this paper, we posit that feature embedding from deep neural networks may convey complementary information and propose a novel knowledge distilling strategy to improve its performance. More specifically, an efficient shallow network, i.e., single-layer feedforward neural network (SLFN), with handcrafted features is utilized to assist a deep long short-term memory (LSTM) network. On the one hand, the deep LSTM network is able to learn features from raw sensory data to encode temporal dependencies. On the other hand, the deep LSTM network can also learn from SLFN to mimic how it generalizes. Experimental results demonstrate the superiority of the proposed method in terms of recognition accuracy against several state-of-the-art methods in the literature.

Fingerprinting based indoor positioning system is gaining more research interest under the umbrella of location-based services. However, existing works have certain limitations in addressing issues such as noisy measurements, high computational complexity, and poor generalization ability. In this work, a random vector functional link network based approach is introduced to address these issues. In the proposed system, a subset of informative features from many randomized noisy features is selected to both reduce the computational complexity and boost the generalization ability. Moreover, the feature selector and predictor are jointly learned iteratively in a single framework based on an augmented Lagrangian method. The proposed system is appealing as it can be naturally fit into parallel or distributed computing environment. Extensive real-world indoor localization experiments are conducted on users with smartphone devices and results demonstrate the superiority of the proposed method over the existing approaches.

A moving-target-enclosing problem is investigated in the paper, where the velocity of the target is unknown and the neighbor topologies may change over time. Each robot only uses the relative position information of the target and its neighbors that may dynamically change over time. An adaptive scheme is proposed to estimate the velocity of the target. Then a distributed control law for each robot is presented, which consists of two parts: One amounts to ensuring the convergence of the distance between the robots and the target to the desired one and the other is used to achieve the uniform distribution when enclosing the target in motion. Lyapunov-based techniques and graph theory are brought together for rigorous analysis of the convergence and stability properties. Our control strategy is practically implementable with only onboard sensors. Simulations are provided to illustrate our results.