Accurate positioning of cells is a fundamental task for many procedures in assisted reproductive technologies (e.g. intracytoplasmic sperm injection or preimplantation genetic diagnosis) to extract or insert materials into and from the cell without causing damage to it. The current method of manual manipulation is based on a trial and error procedure performed by skilled embryologists, where they use two different micropipettes to rotate the cell and immobilize it in the desired orientation. This procedure is time consuming, inconsistent and has a low efficiency. Attempts to automate the process presented in the literature have not yet been implemented in IVF clinics because their high degree of automation requires extensive changes to the current systems used in the clinics. We designed a system that can easily be integrated into standard equipment of IVF clinics and allows automated as well as manual manipulation of cells. The system uses vibrations induced by a surface transducer at the pipette holder to rotate the cell around the pipette tip axis, resulting in 2D motion. To detect if the polar body is in the desired position after a vibration burst, we developed a polar body detection algorithm. We performed simulations and experiments to confirm that vibrations at the natural frequencies of the system cause rotation around the pipette tip axis. Experimental results show that the system is capable of positioning the polar body in plane in less than 5.41 seconds.

Navigating surgical tools in the dynamic and tortuous anatomy of the lung's airways requires accurate, real-time localization of the tools with respect to the preoperative scan of the anatomy. Such localization can inform human operators or enable closed-loop control by autonomous agents, which would require accuracy not yet reported in the literature. In this paper, we introduce a deep learning architecture, called OffsetNet, to accurately localize a bronchoscope in the lung in real-time. After training on only 30 minutes of recorded camera images in conserved regions of a lung phantom, OffsetNet tracks the bronchoscope's motion on a held-out recording through these same regions at an update rate of 47 Hz and an average position error of 1.4 mm. Because this model performs poorly in less conserved regions, we augment the training dataset with simulated images from these regions. To bridge the gap between camera and simulated domains, we implement domain randomization and a generative adversarial network (GAN). After training on simulated images, OffsetNet tracks the bronchoscope's motion in less conserved regions at an average position error of 2.4 mm, which meets conservative thresholds required for successful tracking.

Tendon-driven continuum manipulators are widely used in minimally invasive surgeries, yet autonomous control has proven difficult given the uncertainty in modeling the robot's interaction with the anatomy. In this paper, we test the hypothesis that the anatomy's effect on the robotic manipulator can be approximated as a rotation of the model-based Jacobian. Two methods are described that estimate the orientation using a magnetic position sensor at the robot's distal tip, enabling task-space control in unknown, constrained environments. One method, Rα, determines the orientation of the robot's model using the measured orientation of the distal tip. In the second approach, RΔx, an unscented Kalman filter transforms the tip's measured change in position into an orientation estimate. These methods are validated on the manipulator in free space at three pre-rotated positions (α = 0°, -45°, -90°) and compared to model-based control and model-less control in an autonomous trajectory-following task through an anatomically accurate silicone phantom of a human lung. The results show that by rotating the model-based Jacobian through either of the two methods presented, feedback control successfully navigates the continuum manipulator farther into the lung phantom than static model-based control and model-less control.

Continuum manipulators offer a means for robot manipulation in a constrained environment, where the manipulator body can safely interact with, comply with, and navigate around obstacles. However, obstacle interactions impose constraints that conform the robot body into arbitrary shapes regardless of actuator positions. Generally, these effects cannot be wholly sensed on a continuum manipulator and, therefore, render model-based controllers incorrect, leading to artificial singularities and unstable behavior. We present a task-space closed-loop controller for continuum manipulators that does not rely on a model and can be used in constrained environments. Using an optimal control strategy on a tendon-driven robot, we demonstrate this method, which we term model-less control, which allows the manipulator to interact with several constrained environments in a stable manner. To the best of our knowledge, this is the first work in controlling continuum manipulators without using a model.

Robotic control of flexible devices can enhance and simplify many medical procedures. We present a method for controlling a tendon-driven continuum manipulator by means of specifying the shape configuration. The basis for control is a linear beam configuration model that transforms beam configuration to tendon displacement by modeling internal loads of the compliant system. An essential aspect of this model is the inclusion of both the mechanical and geometrical coupling among serial articulating sections. Important capabilities of this model are the general forward kinematics and the decoupled inverse kinematics that allow for independent control of multiple sections. Tracking results are presented for a cardiac catheter with two articulating sections.

Rigid robotic manipulators employ traditional sensors such as encoders or potentiometers to measure joint angles and determine end-effector position. Manipulators that are flexible, however, introduce motions that are much more difficult to measure. This is especially true for continuum manipulators that articulate by means of material compliance. In this paper, we present a vision based system for quantifying the 3-D shape of a flexible manipulator in real-time. The sensor system is validated for accuracy with known point measurements and for precision by estimating a known 3-D shape. We present two applications of the validated system relating to the open-loop control of a tendon driven continuum manipulator. In the first application, we present a new continuum manipulator model and use the sensor to quantify 3-D performance. In the second application, we use the shape sensor system for model parameter estimation in the absence of tendon tension information.

Continuum robotic manipulators articulate due to their inherent compliance. Tendon actuation leads to compression of the manipulator, extension of the actuators, and is limited by the practical constraint that tendons cannot support compression. In light of these observations, we present a new linear model for transforming desired beam configuration to tendon displacements andviceversa. We begin from first principles in solid mechanics by analyzing the effects of geometrically nonlinear tendon loads. These loads act both distally at the termination point and proximally along the conduit contact interface. The resulting model simplifies to a linear system including only the bending and axial modes of the manipulator as well as the actuator compliance. The model is then manipulated to form a concise mapping from beam configuration-space parameters tonredundant tendon displacements via the internal loads and strains experienced by the system. We demonstrate the utility of this model by implementing an optimal feasible controller. The controller regulates axial strain to a constant value while guaranteeing positive tendon forces and minimizing their magnitudes over a range of articulations. The mechanics-based model from this study provides insight as well as performance gains for this increasingly ubiquitous class of manipulators.

Traditional image mosaicing usually relies on rigid image transformations. In many medical applications, however, tissue deformation during image acquisition or 3D parallax effects may require nonrigid transformations in the mosaicing process. This paper presents a new method that integrates deformable surface models into the image mosaicing algorithms. Our approach has two main contributions. First, we present a global alignment algorithm to efficiently deal with accumulated image registration errors. Second, we introduce a local alignment algorithm to accommodate local scene deformations. These two problems are integrated into a single optimization problem that simultaneously recovers the motion of the camera as well as the structure of the scene. Our approach is demonstrated on simulations, images from a hand-held digital camera, and microscopic images acquired with a micro-endoscope.