This paper proposes a modular direct description method based on the relatively weak interdependence among hysteresis properties to establish a hysteresis model that includes the major curves fitter (MaCF) module, the turning points recorder (TPR) module, and the minor curves fitter (MiCF) module. The MaCF module utilizes the Fourier series fitting method to fit the major hysteresis curves, improving model scalability in conjunction with the modular approach. The TPR module is established to manage the turning points. A memory optimization strategy is further proposed to optimize the model in terms of memory consumption and execution time. The MiCF module then employs a two-stage curve fitting strategy to fit the minor hysteresis curves accurately. Finally, three modules are cascaded to construct the hysteresis model. Experiments have been performed to validate the excellent performance of the developed model in terms of model accuracy, memory usage, and execution time. The results indicate that the model only needs 0.28 KB of memory space to achieve accurate prediction and compensation of hysteresis phenomenon, the standardized normalized maximum error ($NME)$of hysteresis curve prediction is about 1%, and the$NME$value of compensation error is 1.37%. Meanwhile, the average execution time of the model is less than 1$\mu $s with a variance of 0.04.Note to Practitioners—The presented modular direct description (MDD) method can accurately describe and compensate hysteresis phenomenon. The MDD-based model offers the advantage of small memory consumption, making it suitable for application in memory-constrained scenarios, such as offline deployment of microcontrollers and embedded systems. Meanwhile, the execution time of this model is short and stable, which can be applied in applications requiring a high computational period, such as the precise position control of stick-slip drive motors, the precise speed control of ultrasonic motors, and other scenarios with the presence of high-frequency inputs. The MDD method has the potential to become a general modeling method for solving hysteresis challenges, and can be used by simply modifying the relevant modules according to the hysteresis rules in different domains, such as in the fields of artificial muscles and shape memory alloys.

Colon endoscopic robots represent a promising screening modality for the visualization of colon cancers with high sensitivity. However, current colonoscopy robots are often characterized by intricate and bulky mechanical structures, which pose practical challenges when moving through the complex and narrow environment of the colon. Moreover, these robots are typically equipped with a single camera, limiting their ability to accurately estimate the depth of haustral folds in the colon, which is of great importance for the active colonic navigation of the robots. To address these challenges, we develop a track-based stereoscopic endoscopic robot (TSER) which is equipped with four tracks positioned at the corners of its body. This innovative design maximizes the contact between the tracks and the colon wall, enhancing maneuverability. The tracks are constructed from de-molded polydimethylsiloxane (PDMS) and incorporate micro-patterns on their outer surfaces. We have proposed a straightforward strategy for detecting haustral folds using TSER’s stereo camera, which allows for precise identification of their position and depth. The TSER achieves an average motion speed of 9.8 mm/s in a bellows tube that contains silicone oil and a speed of 5.2 mm/s in an exvivo porcine intestinal segment. Impressively, the TSER boasts an 88.11% accuracy rate in haustral fold depth estimation, surpassing the performance of existing geometric shape fitting methods. These results demonstrate that the TSER holds great potential for effective and efficient movement and inspection within the colon, offering a promising solution for improved colon cancer screening.

Muscle force and joint kinematics estimation from surface electromyography (sEMG) are essential for real-time biomechanical analysis of the dynamic interplay among neural muscle stimulation, muscle dynamics, and kinetics. Recent advances in deep neural networks (DNNs) have shown the potential to improve biomechanical analysis in a fully automated and reproducible manner. However, the small sample nature and physical interpretability of biomechanical analysis limit the applications of DNNs. This paper presents a novel physics-informed low-shot adversarial learning method for sEMG-based estimation of muscle force and joint kinematics. This method seamlessly integrates Lagrange's equation of motion and inverse dynamic muscle model into the generative adversarial network (GAN) framework for structured feature decoding and extrapolated estimation from the small sample data. Specifically, Lagrange's equation of motion is introduced into the generative model to restrain the structured decoding of the high-level features following the laws of physics. A physics-informed policy gradient is designed to improve the adversarial learning efficiency by rewarding the consistent physical representation of the extrapolated estimations and the physical references. Experimental validations are conducted on two scenarios (i.e. the walking trials and wrist motion trials). Results indicate that the estimations of the muscle forces and joint kinematics are unbiased compared to the physics-based inverse dynamics, which outperforms the selected benchmark methods, including physics-informed convolution neural network (PI-CNN), vallina generative adversarial network (GAN), and multi-layer extreme learning machine (ML-ELM).

Image guidance during minimally invasive intravascular interventions is primarily achieved based on X-ray fluoroscopy, which has several limitations including limited 3-D imaging capability, significant doses of radiation to operators, and lack of contact force measurement between the cardiovascular tissue and interventional tools. Ultrasound imaging can be adopted to complement or possibly replace 2-D fluoroscopy for intravascular interventions due to its portability, safety to use, and the ability of providing depth information. However, it is challenging to precisely visualize catheters and guidewires in the ultrasound images. In this paper, we propose a novel method to figure out both the position and orientation of the catheter tip in 2-D ultrasound images in real time by detecting and tracking a passive marker attached to the catheter tip. Moreover, the contact force can be estimated simultaneously as well via measuring the length variation of the marker. A geometrical model-based method is introduced to detect the initial position of the marker, and a Kanade-Lucas-Tomasi-based algorithm is developed to track the position, orientation, and length of the marker. The ex vivo experiment results validate the effectiveness of the proposed approach in automatically locating the catheter tip in ultrasound images and its capability of sensing the contact force. Therefore, it can be concluded that the presented method can be utilized to better facilitate operators during intravascular interventions.

This paper reports the latest progress of an ongoing project utilizing a ducted-fan propulsion system to improve a humanoid robot's ability to step over a broad ditch with a height difference between the two sides. This work focuses on the methods of calculating the boundary and optimizing stepping posture to use less thrust and keep the robot balanced while stepping over the ditch. With the proposed methods and new two-dimensional gaits, the prototype robot, named Jet-HRl (Jet Humanoid Robot ver.l) was able to completely step over a broad ditch with 450mm in width (up to 97% of the robot's leg's length), and a height difference of 100mm between two sides.

Intravascular ultrasound (IVUS) imaging provides two-dimensional (2-D) real-time luminal and transmural cross-sectional images of intravascular vessels with detailed pathological information. It has offered significant advantages in terms of diagnosis and guidance and has been increasingly introduced from coronary interventions into more generalized endovascular surgery. However, IVUS itself does not provide spatial pose information for its generated images, making it difficult to construct a 3-D intravascular visualization. To address this limitation, IVUS imaging-driven 3-D intravascular reconstruction techniques have been developed. These techniques enable accurate diagnosis and quantitative measurements of intravascular diseases to facilitate optimal treatment determination. Such reconstruction extends the IVUS imaging modality from pure diagnostic assistance to intraoperative navigation and guidance and supports both therapeutic options and interventional operations. This paper presents a comprehensive survey of technological advances and recent progress on IVUS imaging-based 3-D intravascular reconstruction and its state-of-the-art applications. Limitations of existing technologies and prospects of new technologies are also discussed.

This paper reports the first robotic system for vitrification of mammalian embryos. Vitrification is a technique for preserving oocytes and embryos in clinical IVF (in vitro fertilization). The procedure involves multiple steps of stringently timed pick-and-place operation for processing an oocyte/embryo in vitrification media. In IVF clinics, vitrification is conducted manually by highly skilled embryologists. Processing one oocyte/embryo occupies the embryologist 15–20 minutes, depending on protocols chosen to implement. Due to poor reproducibility and inconsistency across operators, success rates and survival rates also vary significantly. Through collaboration with IVF clinics, we are in process to realize robotic vitrification and aim ultimately to standardize clinical vitrification from manual operation to fully automated robotic operation. Our robotic system is embedded with two contact detection methods to determine the relative Z positions of the vitrification micropipette, embryo, and vitrification straw. A 3D tracking algorithm is developed for visually servoed embryo transfer and real-time monitoring of embryo volume changes during vitrification. Excess medium is automatically removed from around the vitrified embryo on the vitrification straw to achieve a high cooling rate. Tests on mouse embryos demonstrate that the system is capable of performing vitrification with a throughput at least three times that of manual operation and achieved a high survival rate (88.9%) and development rate (93.8%).

This paper proposes a new vasculature reconstruction and catheter modeling scheme based on data fusion from intravascular ultrasound (IVUS) imaging, electromagnetic (EM) tracking and shape sensing for trans-femoral Transcatheter Aortic Valve Implantation (TAVI). The system is suitable for obtaining inner cross sectional images of the aorta with an IVUS probe, reconstructing its 3D virtual model using sensor fusion of the corresponding pose information of IVUS probe from an electromagnetic (EM) sensor, as well as reconstructing the catheter shape based on optical fibers with Fiber Bragg Grating (FBG) sensors. A hybrid probe consisting of an IVUS sensor, an EM sensor and an optical shape sensor has been created and tested on in-vitro silicone aortic phantoms. A practical image processing method based on the gradient vector flow (GVF) snake has been proposed, followed by fusion with pose information from an EM sensor for the anatomical model reconstruction. Demonstration of the proposed method was performed on two aortic phantoms. Preliminary results show how the catheter shape reconstruction is realized by the shape sensor. The proposed method could facilitate intra-operative surgical guidance for valve alignment, improve the precision for positioning, reduce the time of the TAVI procedure, minimize the use of contrast agent, and assess the status of the deployed valve after surgery.

This paper presents a new in-vitro simulator that can simulate the new technology of three-dimensional digital subtraction angiography (DSA) to perform a cerebral aneurysm surgery. Then, propose a new technique for the numerical evaluation of the surgical tools applied within this interventional surgery to aid in the design and selection of surgical instruments. Initially, the in-vitro intravascular simulator was constructed, followed by cerebral aneurysm models that were fabricated with highly sensitive photoelastic materials of epoxy resin. Then reconstruct virtual reality modeling of the blood vessel and catheter to record the position information and realize virtual guidance for training. Meanwhile, conduct image processing of photoelastic effects for numerical evaluations of surgical instruments such as catheter insertion and stent expansion. During the stent expansion experiment, the average stress in the selected region of interest caused by the expansion was calculated and compared with the corresponding value obtained before the expansion. The presented simulator and measurement technique can assist in improving a surgeons skills, quantifying the performance of medical mechanisms, and contribute to 3D analysis of stress distribution. This approach can also provide feedback control for robot assisted endovascular systems.

Quantification of human skills is a challenge for simulator based medical training, particularly for endovascular intervention. For that purpose it is needed to measure different biometric parameters from the user and additionally measure the interaction between endovascular instruments and vasculature models membrane. For that purpose in this research we present an optical sensor for encoding linear and rotational motion of the catheter through the insertion port. Additionally we present two studies for measuring tissue integrity, time to complete a task, reaction time, and wasting motion using two populations of expert and beginner. The optical encoder showed an average error of 0.39mm and 10.78deg. In both studies the presented cyber-physical system is useful to identify the users with qualities closer to the expert among users with low performance.

For the development of artificial intelligent diagnosis for cerebrovascular intervention, it is desirable to forecast the growth of cerebral aneurysms. In order to achieve such purpose, it is needed to evaluate wall shear stress, strain, pressure, deformation and flow velocity in the aneurysm region. In this research, we focus on in-vitro strain and deformation measurements of cerebral aneurysm models, and propose a cyber-physical system, in which a scaled-up membranous silicone model of cerebral aneurysm was built and integrated with a specialized pump for the pulsatile blood flow simulation, and a vision system was constructed to measure the strain on different regions on the model with pulsatile blood flow circulated inside. Experimental results show that both distance and area strain maxima were larger for the aneurysm neck (0.042 and 0.052), followed by the aneurysm dome (0.023 and 0.04) and then by the main blood vessel section (0.01 and 0.014), which were complemented with computer fluid dynamics simulation for the inclusion of wall shear stress, oscillatory shear index and aneurysm formation index. Medical imaging data of the cerebral aneurysm in 2008 and 2011 was obtained. Diagnosis results have concordance with the aneurysm growth in 2011. The presented measurement method offers an option for measuring strain and deformation to be complementary with computer fluid dynamics and photoelastic stress analysis for advanced diagnostic in the endovascular surgery.

This paper presents a sensor fusion between intravascular ultrasound (IVUS) and magnetic trackers for constructing the virtual reality three dimensional models of the blood vessels. We propose this approach for vasculature modeling as part of a guidance system relying on augmented reality for assistance during aortic stent graft deploy. This guidance will facilitate the alignment of the holes on the stent graft walls with the renal and mesenteric arteries ramifications. First we studied the disturbances on the magnetic tracker measurements induced by IVUS emitter after assembling the two sensors together. Then we performed a scan with the hybrid probe inside a blood silicone model submerged into a water tank, captured and fused data from both sensors. The dispersion of samples increased less than 1mm in the evaluated locations while the IVUS was activated. This enabled the construction of a three-dimensional model in virtual reality of the blood vessel model relying on the sensor fusion.