Dynamic stability refers to the ability of a magnetic actuation system to maintain equilibrium by damping oscillations. It is typically quantified by physical scalars that serve as critical indicators of both safety and effectiveness in real-world applications. Traditional real-time assessment methods often require measuring and calibrating task-specific parameters. While feasible in principle, these approaches are hindered in practice by procedural complexity, time consumption, measurement difficulty, and uncertainties inherent to the non-contact, non-rigid nature of magnetic actuation. In this work, we eliminate the need for parameter assumptions and direct measurements, enabling a more efficient and generalizable evaluation of dynamic stability. Our approach implicitly incorporates uncertain physical properties into a learning-based framework. Specifically, we propose a temporal difference policy that predicts dynamic stability by comparing multiple time-varying sequences, thereby reducing the influence of task-specific parameters. The robustness and effectiveness are validated through extensive experiments, including baseline comparisons and online implementations across diverse magnetically actuated objects. Our findings highlight its practical advantages and pave the way for innovative control strategies in precise magnetic actuation applications.

Endovascular intervention relies on accurate path planning and precise guidewire manipulation, which makes training difficult and time-consuming. We present a real-time vascular intervention simulator that couples Cosserat rod-based guidewire dynamics, mesh-contraction centerline extraction with shortest-path planning, and a Reinforcement Learning (RL) controller for autonomous navigation. A Proximal Policy Optimization (PPO) agent with a curriculum schedule and a visibility-aware error-correction strategy learns push-pull-rotate actions aligned with waypoint paths. The system runs at 30 Hz in Unreal Engine, achieves a success rate of over 95 % on integrated and real-model scenarios, and reduces the completion time compared with a distance-reward RL baseline and a rule-based policy. Results indicate that integrating physically-plausible simulation with path planning and curriculum-driven PPO agent yields robust recovery from deviations and efficient navigation.

With medical ultrasound becoming one of the most prevalent examination methods, robotic ultrasound systems offer the potential to simplify the scanning process and relieve professional sonographers from repetitive and tedious tasks. Despite recent advances, enabling robots to autonomously perform ultrasound examinations remains a challenge, mainly due to the difficulty in representing and generalizing professional ultrasound skills. In this paper, we present a comprehensive framework for learning autonomous ultrasound skills from freehand demonstrations in clinical settings. Our proposed framework consists of two key stages: offline learning and online adaptation. During the offline learning stage, ultrasound skills are encapsulated into a low-dimensional probabilistic model using a self-supervised architecture. The multimodal signals include ultrasound images, probe orientations, and contact forces. During the online adaptation stage, the model predicts the optimal actions either by direct regression or by using local exploration schemes. We perform clinical demonstrations with 24 volunteers and collect 120 experiences. Our benchmark includes 5 different tasks, including intra-patient, inter-patient, inter-sex, inter-age, and inter-obesity tasks. Both one-step and sequence-based predictions are achieved by using different variants of our framework. Customized and generic representation learning backbones are tested and analyzed. In conclusion, our autonomous ultrasound framework is flexible and robust, and potentially enriches the options for freehand/robotic ultrasound applications. Note to Practitioners—This paper is motivated by the problem of learning multimodal manipulation skills from human demonstrations, with a specific focus on freehand ultrasound skills. Our multimodal fusion framework is effective and compatible with some popular image representation backbones. Our adaptive methods and the variants have satisfactory prediction accuracy, with flexibility achieved by adjusting a few factors. We collect high-quality freehand demonstrations from ultrasound examinations in clinical settings. The data is openly available to facilitate the reproducibility of our work and to support the development of autonomous ultrasound strategies based on imitation learning.

Internet of Medical Things (IoMT) technology has significantly helped surgeons perform complex clinical procedures, including interventional and endoscopic surgeries. However, surgeons face challenges in quickly becoming proficient with some IoMT devices. This difficulty stems from the fact that IoMT devices are not commonly used or familiar in their daily work and lives. To address this issue, we propose a novel interactive framework for IoMT, named freehand interaction, which includes visual control and haptic feedback. Our idea is to allow surgeons to operate surgical instruments, such as needles, capsules, and catheters with their bare hands and regular experience. At the same time, identical instruments in the surgical environment mirror the surgeon’s actions through visual control. The interactive forces encountered in the surgical environment are quantitatively communicated to the surgeon through the hand-held instruments. Our IoMT framework achieves both visual control and haptic feedback using electromagnetic mechanisms, ensuring mid-air freehand manipulation and contactless remote actuation. We perform different tasks in suspended, liquid, and in-vitro environments. The hand-held and mirroring instruments show high similarity and correlation, even within different electromagnetic systems and confined workspaces. Tracking accuracy, response time, and haptic forces quantified by a mechanical gauge are satisfactory. Our algorithm processes raw video streams and maintains efficiency even in scenarios with partial hand occlusion and various types of image noise. In summary, this work contributes to improving visual intuition and haptic immersion in IoMT applications.

Abstract With the development of virtual reality technology, simulation surgery has become a low‐risk surgical training method and high‐precision positioning of surgical instruments is required in virtual simulation surgery. In this paper we design and validate a novel electromagnetic positioning method based on a uniform gradient magnetic field. We employ Maxwell coils to generate the uniform gradient magnetic field and propose two positioning algorithms based on magnetic field, namely the linear equation positioning algorithm and the magnetic field fingerprint positioning algorithm. After validating the feasibility of proposed positioning system through simulation, we construct a prototype system and conduct practical experiments. The experimental results demonstrate that the positioning system exhibits excellent accuracy and speed in both simulation and real‐world applications. The positioning accuracy remains consistent and high, showing no significant variation with changes in the positions of surgical instruments.

The potential diagnostic applications of magnet-actuated capsules have been greatly increased in recent years. For most of these potential applications, accurate position control of the capsule have been highly demanding. However, the friction between the robot and the environment as well as the drag force from the tether play a significant role during the motion control of the capsule. Moreover, these forces especially the friction force are typically hard to model beforehand. In this paper, we first designed a magnet-actuated tethered capsule robot, where the driving magnet is mounted on the end of a robotic arm. Then, we proposed a learning-based approach to model the friction force between the capsule and the environment, with the goal of increasing the control accuracy of the whole system. Finally, several real robot experiments are demonstrated to showcase the effectiveness of our proposed approach.