Open tracheostomy (OT) is considered the traditional way and golden standard for treating airway obstruction patients. However, OT has many unavoidable drawbacks, including strict performing scenarios, significant scarring, and the risk of surgeon infection. Percutaneous dilation tracheostomy (PDT) emerges, with advantages including a lower cost, smaller scarring, and better protection of surgeons from inflecting by aerosol. However, the outside-in puncture manner of PDT has a risk of piercing the post-tracheal wall and the esophagus with uncontrolled force. Additionally, locating tracheal rings and determining the puncture site externally can be challenging for certain patients, such as those who are obese or have undergone neck surgery, while this procedure typically relies on palpation and the surgeon's expertise. Hence, to improve the safety and simplicity of tracheostomy, a minimally-invasive endotracheal inside-out flexible needle-driving system towards microendoscope-guided robotic tracheostomy (MERT) has been proposed in this paper. Guided by an optical coherence tomography (OCT) probe and a microendoscope, the robot inserts into the trachea and performs an inside-out puncture using a flexible needle. The robot can work through a standard endotracheal tube (ETT), and the puncture direction of the flexible needle is variable. Kinematics and statics models of the flexible needle have been derived, and the minimum position errors generated in the kinematics and statics validation experiments are$0.57 \pm 0.21 \mathbf{~ m m}$and$0.27 \pm 0.21 \mathbf{~ m m}$. Finally, a porcine trachea puncture experiment is carried out, and the feasibility of the proposed system is verified.

In situ tissue biopsy with an endoluminal catheter is an efficient approach for disease diagnosis, featuring low invasiveness and few complications. However, the endoluminal catheter struggles to adjust the biopsy direction by distal endoscope bending or proximal twisting for tissue sampling within the tortuous luminal organs, due to friction-induced hysteresis and narrow spaces. Here, we propose a pneumatically-driven robotic catheter enabling the adjustment of the sampling direction without twisting the catheter for an accurate in situ omnidirectional biopsy. The distal end of the robotic catheter consists of a pneumatic bending actuator for the catheter’s deployment in torturous luminal organs and a pneumatic rotatable biopsy mechanism (PRBM). By hierarchical airflow control, the PRBM can adjust the biopsy direction under low airflow and deploy the biopsy needle with higher airflow, allowing for rapid omnidirectional sampling of tissue in situ. This paper describes the design, modeling, and characterization of the proposed robotic catheter, including repeated deployment assessments of the biopsy needle, puncture force measurement, and validation via phantom tests. The PRBM prototype has six sampling directions evenly distributed across 360 degrees when actuated by a positive pressure of 0.3 MPa. The pneumatically-driven robotic catheter provides a novel biopsy strategy, potentially facilitating in situ multidirectional biopsies in tortuous luminal organs with minimum invasiveness.

Deep learning can predict depth maps and capsule ego-motion from capsule endoscopy videos, aiding in 3D scene reconstruction and lesion localization. However, the collisions of the capsule endoscopies within the gastrointestinal tract cause vibration perturbations in the training data. Existing solutions focus solely on vision-based processing, neglecting other auxiliary signals like vibrations that could reduce noise and improve performance. Therefore, we propose V2-SfMLearner, a multimodal approach integrating vibration signals into vision-based depth and capsule motion estimation for monocular capsule endoscopy. We construct a multimodal capsule endoscopy dataset containing vibration and visual signals, and our artificial intelligence solution develops an unsupervised method using vision-vibration signals, effectively eliminating vibration perturbations through multimodal learning. Specifically, we carefully design a vibration network branch and a Fourier fusion module, to detect and mitigate vibration noises. The fusion framework is compatible with popular vision-only algorithms. Extensive validation on the multimodal dataset demonstrates superior performance and robustness against vision-only algorithms. Without the need for large external equipment, our V2-SfMLearner has the potential for integration into clinical capsule robots, providing real-time and dependable digestive examination tools. The findings show promise for practical implementation in clinical settings, enhancing the diagnostic capabilities of doctors. Note to Practitioners—This paper is motivated by the problem of estimating the depth and ego-motion information for the wireless capsule endoscopy in the human gastrointestinal tract to realize accurate, efficient, robust, and real-time inspection. Our estimation method does not engage any external localization equipment. Instead, inspired by the existing research on integrating capsule endoscopy and inertial measurement units, we introduce vibration signals into vision-based depth and ego-motion estimation approaches, improving the accuracy and robustness of the estimation results based on multimodal learning methods. Research on capsule robots or computer vision can readily be combined with our framework for various clinical and industrial applications.

Capsule endoscopes, predominantly serving diagnostic functions, provide lucid internal imagery but are devoid of surgical or therapeutic capabilities. Consequently, despite lesion detection, physicians frequently resort to traditional endoscopic or open surgical procedures for treatment, resulting in more complex, potentially risky interventions. To surmount these limitations, this study introduces a chained flexible capsule endoscope (FCE) design concept, specifically conceived to navigate the inherent volume constraints of capsule endoscopes whilst augmenting their therapeutic functionalities. The FCE’s distinctive flexibility originates from a conventional rotating joint design and the incision pattern in the flexible material. In vitro experiments validated the passive navigation ability of the FCE in rugged intestinal tracts. Further, the FCE demonstrates consistent reptile-like peristalsis under the influence of an external magnetic field, and possesses the capability for film expansion and disintegration under high-frequency electromagnetic stimulation. These findings illuminate a promising path toward amplifying the therapeutic capacities of capsule endoscopes without necessitating a size compromise.

Helicobacter pylori, a pervasive bacterial infection associated with gastrointestinal disorders such as gastritis, peptic ulcer disease, and gastric cancer, impacts approximately 50% of the global population. The efficacy of standard clinical eradication therapies is diminishing due to the rise of antibiotic-resistant strains, necessitating alternative treatment strategies. Photodynamic therapy (PDT) emerges as a promising prospect in this context. This study presents the development and implementation of a magnetically-guided origami robot, incorporating flexible printed circuit units for sustained and stable phototherapy of Helicobacter pylori. Each integrated unit is equipped with wireless charging capabilities, producing an optimal power output that can concurrently illuminate up to 15 LEDs at their maximum intensity. Crucially, these units can be remotely manipulated via a magnetic field, facilitating both translational and rotational movements. We propose an open-loop manual control sequence that allows the formation of a stable, compliant triangular structure through the interaction of internal magnets. This adaptable configuration is uniquely designed to withstand the dynamic squeezing environment prevalent in real-world gastric applications. The research herein represents a significant stride in leveraging technology for innovative medical solutions, particularly in the management of antibiotic-resistant Helicobacter pylori infections.

Miniature rotation actuators have been extensively developed and utilized in optical coherence tomography (OCT) endoscopy, enabling distortion-free OCT imaging in complex and tortuous environments. However, the use of electrical-driven rotation actuators raises safety concerns. Although magnetic-driven rotation actuators have been reported in OCT endoscopy, their use can potentially interfere with other medical devices in clinical settings. Here, we propose a pneumatic miniature rotation actuator that eliminates the electricity and magnetism concerns in circumferential imaging for OCT endoscopy. The rotor of the actuator is designed as a windmill, enabling it to convert air energy into rotation energy. In addition, to maintain the stable rotation, both a sliding bearing with two supporting points and a glass spindle with a half-ball end surface are developed. The rotation speed of our pneumatic actuator can be controlled from 66 to 97 revolutions per second by adjusting the airflow rate from 3.25 to 4.00 liters per minute. By OCT imaging of the human fingers, we demonstrate the feasibility of the pneumatic actuator in electricity-free distal scanning OCT endoscopy. Our pneumatic rotation actuator has wide-ranging potential in various fiber-imaging modalities, including not only OCT but also ultrasound imaging that requires similar rotation capabilities.

Wearable magnetic tracking systems (MTSs) offer a promising technology for the long-term tracking of wireless-capsule robots within the digestive tract. However, existing wearable MTSs are fixed in size and cannot accommodate patients with diverse abdominal circumferences. To address this limitation, we propose a wearable and reconfigurable MTS. First, we design a reconfigurable sensor array inspired by the structure of bamboo slips, allowing it to conform to the abdominal surface and accommodate individuals with different abdominal circumferences. Next, we formulate a magnetic tracking optimization problem based on the magnetic dipole model and our established kinematic model of the reconfigurable sensor array. Solving the magnetic tracking problem, we achieved outstanding localization accuracy of 1.44$\pm$0.50 mm and 1.07$\pm 0.16^\circ$. Experimental validation demonstrates our proposed system's portability, reconfigurability, and adaptability to varying abdominal circumferences, offering valuable technological means for diagnosing and treating gastrointestinal disorders.

Traditional magnetic tracking approaches based on mathematical models and optimization algorithms are computationally intensive, depend on initial guesses, and do not guarantee convergence to a global optimum. Although fully supervised data-driven deep learning can solve the above issues, the demand for a comprehensive dataset hampers its applicability in magnetic tracking. Thus, we propose an annular magnet pose estimation network (called AMagPoseNet) based on dual-domain few-shot learning from a prior mathematical model, which consists of two subnetworks: PoseNet and CaliNet. PoseNet learns to estimate the magnet pose from the prior mathematical model, and CaliNet is designed to narrow the gap between the mathematical model domain and the real-world domain. Experimental results reveal that the AMagPoseNet outperforms the optimization-based method regarding localization accuracy (1.87$\pm$1.14 mm, 1.89$\pm \text{0.81}^{\circ }$), robustness (nondependence on initial guesses), and computational latency (2.08$\pm$0.02 ms). In addition, the six-degree-of-freedom pose of the magnet could be estimated when discriminative magnetic field features are provided. With the assistance of the mathematical model, the AMagPoseNet requires only a few real-world samples and has excellent performance, showing great potential for practical biomedical and industrial applications.

High-precision pose adjustment for the self-charging of mobile robots remains a significant challenge. Permanent magnet (PM)-based magnetic tracking technique is a promising technical solution, with occlusion-free and simultaneous positioning and orientation tracking. However, the superposition of the geomagnetic vector and the magnetic field vector generated by the PM leads to the degrading of magnetic tracking performance. Thus, a magnetic tracking technique with real-time geomagnetic vector separation is investigated in this study. Firstly, the environmental magnetic field is accurately modeled, consisting of the PM field, uniform disturbance field, and non-uniform disturbance field. For the uniform disturbance field, we combine it with the PM pose as unknown parameters to be estimated. For the non-uniform disturbance field, a robust kernel function is employed to diminish its influence on positioning performance. Finally, the PM pose and geomagnetic vector are simultaneously estimated by optimization algorithms. A docking experiment for self-charging mobile robots was carried out based on the proposed tracking technique. The robot can successfully recharge its battery with only one alignment operation, where the repeat parking accuracy at the anchor point is 1.38 mm and ±1.27°, respectively.

Magnetically-actuated flexible endoscopes (MAFE) have been well used in minimally-invasive surgery because they can be steered by a magnetic field thus more flexible than traditional endoscopes. Model-free and uncalibrated visual-feedback control makes it possible to manipulate MAFE with a magnetic field without external tracking systems. Because no extra sensor is required to obtain position and posture information, the size of MAFE can be made smaller. However, the traditional control method focuses on 2DoF control, which lacks control over the posture of the end of MAFE. This may result in unnecessary contact between MAFE and tissue and cause injury during the advancement of the endoscope. In this letter, we propose algorithms to enhance the pose control of MAFE to 4DoF and 5DoF based on model-free and uncalibrated visual-feedback control. Experiments in structured environments verify that the control algorithms are able to realize 4DoF manual navigation and 5DoF automatic navigation.

Small-scale robots have great potential in minimally invasive surgery (MIS). In this paper, we propose an untethered magnetic gripper with small scale and build a double-magnet model for it. The gripper is 4.3mm long and its maximum width is 4mm. It contains a spindle and two magnets, which can achieve precise control of orientation, position and open angle with external magnetic driven field. As a result, it can perform operations such as transporting medicines in confined and constrained environments. Modeling and analysis of the magnetic gripper have been carried out. Relationship between the open angle and external magnetic field has been established. Kinematics model of the gripper has been built. A 3-axis Helmholtz-Maxwell coil system has been established to generate the magnetic field, in which orientation and open angle can be controlled with uniform magnetic field while position can be controlled with gradient field. The proposed gripper have been validated with phantom experiments. An opened angle control error of 0.63° and direction control error of 1.1° have been obtained.