In this paper we describe work towards retinal vessel cannulation using an actively stabilized handheld robot, guided by monocular vision. We employ a previously developed monocular camera based surface reconstruction method using automated laser beam scanning over the retina. We use the reconstructed plane to find a coordinate transform between the 2D image plane coordinate system and the global 3D frame. Within a hemispherical region around the target, we use motion scaling for higher precision. The contribution of this work is the homography matrix estimation using monocular vision and application of the previously developed laser surface reconstruction to Micron guided vein cannulation. Experiments are conducted in a wet eye phantom to show the higher accuracy of the surface reconstruction as compared to standard stereo reconstruction. Further, experiments to show the increased surgical accuracy due to motion scaling are also carried out.

This paper presents robot-aided intraocular laser surgery using a handheld robot known as Micron. The micromanipulator incorporated in Micron enables visual servoing of a laser probe, while maintaining a constant distance of the tool tip from the retinal surface. The comparative study was conducted with various control methods for evaluation of robot-aided intraocular laser surgery.

Vitreoretinal microsurgery requires precise hand-eye coordination to manipulate delicate structures within the eye on the order of tens of microns. To achieve these tasks, surgeons use tools of diameter 0.9 mm or less to access the eye's interior structures. The level of force required during these manipulations is often below the human tactile threshold, requiring the surgeon to rely on subtle visual cues or to apply larger forces above the tactile threshold for feedback. However, both of these methods can lead to tissue damage. Excursions can be made into tissues which are not felt by the surgeon, while larger forces have a higher chance of damaging tissue within the eye. To prevent damage to the retina and other anatomy, we present the implementation of hybrid position/force control operating in the sub-tactile force range for a handheld robotic system. This approach resulted in a 42% reduction in the mean force and 52% reduction in maximum force during peeling tasks.

This paper presents a technique for automated intraocular laser surgery using a handheld micromanipulator known as Micron. The novel handheld manipulator enables the automated scanning of a laser probe within a cylinder of 4 mm long and 4 mm in diameter. For the automation, the surface of the retina is reconstructed using a stereomicroscope, and then preplanned targets are placed on the surface. The laser probe is precisely located on the target via visual servoing of the aiming beam, while maintaining a specific distance above the surface. In addition, the system is capable of tracking the surface of the eye in order to compensate for any eye movement introduced during the operation. We compared the performance of the automated scanning using various control thresholds, in order to find the most effective threshold in terms of accuracy and speed. Given the selected threshold, we conducted the handheld operation above a fixed target surface. The average error and execution time are reduced by 63.6% and 28.5%, respectively, compared to the unaided trials. Finally, the automated laser photocoagulation was demonstrated also in an eye phantom, including compensation for the eye movement.

This paper presents the characterization and comparison of physiological tremor for pointing tasks in multiple environments, as a baseline for performance evaluation of microsurgical robotics. Previous studies have examined the characteristics of physiological tremor under laboratory settings as well as different operating conditions. However, different test methods make the comparison of results across trials and conditions difficult. Two vitroretinal microsurgeons were evaluated while performing a pointing task with no entry-point constraint, constrained by an artificial eye model, and constrained by a rabbit eye in vivo. For the three respective conditions the 3D RMS positioning error was 144 μm, 258 μm, and 285 μm, and maximum 3D error was 349 μm, 647 μm, and 696 μm. A spectral analysis was also performed, confirming a distinct peak near in the 6-12 Hz frequency range, characteristic of hand tremor during tasks in all three environments.

Peeling delicate retinal membranes, which are often less than 5 µm thick, is one of the most challenging retinal surgeries. Preventing rips and tears caused by tremor and excessive force can decrease injury and reduce the need for follow up surgeries. We propose the use of a fully handheld microsurgical robot to suppress tremor while enforcing helpful constraints on the motion of the tool. Using stereo vision and tracking algorithms, the robot activates motion-scaled behavior as the tip reaches the surface, providing finer control during the critical step of engaging the membrane edge. A hard virtual fixture just below the surface limits the total downward force that can be applied. Furthermore, velocity limiting during the peeling helps the surgeon maintain a smooth, constant force while lifting and delaminating the membrane. On a phantom consisting of plastic wrap stretched across a rubber slide, we demonstrate our approach reduces maximum force by 40-70%.

This paper presents the design and analysis of a handheld manipulator for vitreoretinal microsurgery and other biomedical applications. The design involves a parallel micromanipulator utilizing six piezoelectric linear actuators, combining compactness with a large range of motion and relatively high stiffness. Given the available force of the actuators, the overall dimension of the micromanipulator was optimized considering realistic external loads on a remote center of motion representing the point of expected contact with the sclera of the eye during microsurgery. Based on optimization and workspace analysis, a benchtop version of the micromanipulator was built with a base diameter of 25 mm and a height of 50 mm. It provides a hemispherical workspace of 4.0 mm diameter at the tool tip. The manipulation performance of the constructed manipulator was measured under a lateral load applied at the remote center of motion. The micromanipulator tolerated side loads up to 200 mN.

We describe the design and performance of a hand-held actively stabilized tool to increase accuracy in micro-surgery or other precision manipulation. It removes involuntary motion such as tremor by actuating the tip to counteract the effect of the undesired handle motion. The key components are a three-degree-of-freedom piezoelectric manipulator that has 400 μm range of motion, 1 N force capability, and bandwidth over 100 Hz, and an optical position measurement subsystem that acquires the tool pose with 4 μm resolution at 2000 samples/s. A control system using these components attenuates hand motion by at least 15 dB (a fivefold reduction). By considering the effect of the frequency response of Micron on the human visual feedback loop, we have developed a filter that reduces unintentional motion, yet preserves intuitive eye-hand coordination. We evaluated the effectiveness of Micron by measuring the accuracy of the human/machine system in three simple manipulation tasks. Handheld testing by three eye surgeons and three non-surgeons showed a reduction in position error of between 32% and 52%, depending on the error metric.

Precise movement during micromanipulation becomes difficult in submillimeter workspaces, largely due to the destabilizing influence of tremor. Robotic aid combined with filtering techniques that suppress tremor frequency bands increases performance; however, if knowledge of the operator's goals is available, virtual fixtures have been shown to greatly improve micromanipulator precision. In this paper, we derive a control law for position-based virtual fixtures within the framework of an active handheld micromanipulator, where the fixtures are generated in real-time from microscope video. Additionally, we develop motion scaling behavior centered on virtual fixtures as a simple and direct extension to our formulation. We demonstrate that hard and soft (motion-scaled) virtual fixtures outperform state-of-the-art tremor cancellation performance on a set of artificial but medically relevant tasks: holding, move-and-hold, curve tracing, and volume restriction.

Active compensation of physiological tremor for handheld micromanipulators depends on fast control and actuation responses. Because of real-world latencies, real-time compensation is usually not completely effective at eliminating unwanted hand motion. By modeling tremor, more effective cancellation is possible by anticipating future hand motion. We propose a feedforward control strategy that utilizes tremor velocity from a state-estimating Kalman filter. We demonstrate that estimating hand motion in a feedforward controller overcomes real-world latencies in micromanipulator actuation. In hold-still tasks with a fully handheld micromanipulator, the proposed feedforward approach improves tremor rejection by over 50%.

In microsurgery, a surgeon often deals with anatomical structures of sizes that are close to the limit of the human hand accuracy. Robotic assistants can help to push beyond the current state of practice by integrating imaging and robot-assisted tools. This paper demonstrates control of a handheld tremor reduction micromanipulator with visual servo techniques, aiding the operator by providing three behaviors: snap-to, motion-scaling, and standoff-regulation. A stereo camera setup viewing the workspace under high magnification tracks the tip of the micromanipulator and the desired target object being manipulated. Individual behaviors activate in task-specific situations when the micromanipulator tip is in the vicinity of the target. We show that the snap-to behavior can reach and maintain a position at a target with an accuracy of 17.5 ± 0.4μm Root Mean Squared Error (RMSE) distance between the tip and target. Scaling the operator's motions and preventing unwanted contact with non-target objects also provides a larger margin of safety.