We contribute to new design directions towards robots that can socially engage with pediatric patients while imaging their eyes reliably. Eye imaging is essential to diagnose and manage ocular diseases, but practically impossible to conduct due to children's fear and aversion during the exam. Pediatric patients frequently require an exam under anesthesia, adding significant medical risk, stress, delay, and cost of care. We explored the design space of character design for an eye-imaging robot system to make eye exams more fun for children. Using contextual inquiry, we collected needs from stakeholders around eye exams, leading to the understanding of pain points. We then conducted design explorations of robot characters that could mitigate negative effects while amplifying moments of fun for pediatric patients. We built two low-fidelity robot characters and showed them to stakeholders. Our research highlights the need for these approachable characters to realize eye exams in pediatric patients with engagement.

Optical coherence tomography (OCT) has been widely used for high-fidelity biological tissue scanning but is traditionally limited to small lateral fields of view that preclude large-area scanning. To overcome this problem, we propose an integration of an OCT sensor to a 6-DOF robot arm end-effector combined with a geometry-aware stitching model for surface and volumetric data stitching. We firstly develop a simple but efficient Robot-OCT calibration method by using a three-marker calibration pattern and implement an optimization solver. Given a pre-defined trajectory, a local planner is developed to update the sensor pose by using the OCT point cloud information in order to maintain the effective imaging depth based on the distance and orientation constraints. The system calibration method is verified through repeated experiments with the three-marker targets and the result shows an average testing error of$0.132 \pm 0.071 ~\text{mm}$. The geometry-aware OCT stitching framework is demonstrated based on the experiments of different scanning trajectories and 3D-printed phantoms for large-area scanning. The OCT stitched point cloud is compared with the ground truth from the phantom CAD model and the result show an average surface alignment error of$0.441 \pm 0.241 ~\text{mm}$for the path following tasks.

In microsurgery, surgeons frequently encounter challenges due to the need for exceptional precision and dexterity, the lack of depth perception for micro-scale surgical maneuvers, and the inevitable effects of fatigue and hand tremor. In surgical robotics, conventional intraoperative perception systems normally provide real-time image feedback, but depth and volumetric information is typically lacking. To overcome these challenges, we propose a teleoperated robotic system with two arms to provide high-fidelity intraoperative volumetric imaging during micro-scale tissue manipulation. This system incorporates an optical coherence tomography sensor for real-time 3D visualization and a dual-arm teleoperated robot system controlled by haptic input devices for accurate and precise manipulation. We characterize the system's performance through a precision positioning task and a vessel following task in a retinal model, which shows average positioning errors of approximately 232μm and 83 μm, respectively. We demonstrate the fully integrated system through the completion of an eggshell membrane peeling task that simulates retinal membrane peeling.

Optical coherence tomography (OCT) has revolutionized diagnostics in ophthalmology. However, it requires pa-tient cooperation to fixate on multiple targets and stabilize their head utilizing both chin and forehead rests. Patient cooperation is particularly important for image montaging, where patients are asked to fixate on multiple targets to sequentially image different regions of interest on the retina. These individual volumes are then combined into a single large field of view volume. To automate the OCT image acquisition process, we previously developed a robot-mounted OCT scanner that auto-aligned with the retinal region of interest while compensating for subject motion. We utilized this system to self-align at multiple regions-of-interest and acquire stabilized volumes. We then montaged volume projections into a larger field of view image. The system tracked the 3D location of the subject's eye as well as their gaze orientation using a combination of face and pupil tracking cameras. We demonstrated automated OCT acquisition for live image montaging on free-standing subjects and evaluated the consistency of our live volumetric mapping on human subject data. Our results demonstrate that the system not only stabilized images, but also provided sufficient control of region-of-interest specific alignment to automatically acquire and montage OCT images, synthetically increasing the system's field of view by$20^{\circ}$.

Robotic automation has the potential to assist human surgeons in performing suturing tasks in microsurgery, and in order to do so a robot must be able to guide a needle with sub-millimeter precision through soft tissue. This paper presents a robotic suturing system that uses 3D optical coherence tomography (OCT) system for imaging feedback. Calibration of the robot-OCT and robot-needle transforms, wound detection, keypoint identification, and path planning are all performed automatically. The calibration method handles pose uncertainty when the needle is grasped using a variant of iterative closest points. The path planner uses the identified wound shape to calculate needle entry and exit points to yield an evenly-matched wound shape after closure. Experiments on tissue phantoms and animal tissue demonstrate that the system can pass a suture needle through wounds with 0.200 mm overall accuracy in achieving the planned entry and exit points, and over 20× more precise than prior autonomous suturing robots.

Ophthalmic microsurgery is technically difficult because the scale of required surgical tool manipulations challenge the limits of the surgeon's visual acuity, sensory perception, and physical dexterity. Intraoperative optical coherence tomography (OCT) imaging with micrometer-scale resolution is increasingly being used to monitor and provide enhanced real-time visualization of ophthalmic surgical maneuvers, but surgeons still face physical limitations when manipulating instruments inside the eye. Autonomously controlled robots are one avenue for overcoming these physical limitations. We demonstrate the feasibility of using learning from demonstration and reinforcement learning with an industrial robot to perform OCT-guided corneal needle insertions in an ex vivo model of deep anterior lamellar keratoplasty (DALK) surgery. Our reinforcement learning agent trained on ex vivo human corneas, then outperformed surgical fellows in reaching a target needle insertion depth in mock corneal surgery trials. This work shows the combination of learning from demonstration and reinforcement learning is a viable option for performing OCT guided robotic ophthalmic surgery.

Optical coherence tomography (OCT) has found great success in ophthalmology where it plays a key role in screening and diagnostics. Clinical ophthalmic OCT systems are typically deployed as tabletop instruments that require chinrest stabilization and trained ophthalmic photographers to operate. These requirements preclude OCT diagnostics in bedbound or unconscious patients who cannot use a chinrest, and restrict OCT screening to ophthalmology offices. We present a robotically-aligned OCT scanner capable of automatic eye imaging without chinrests. The scanner features eye tracking from fixed-base RGB-D cameras for coarse and stereo pupil cameras for fine alignment, as well as galvanometer aiming for fast lateral tracking, reference arm adjustment for fast axial tracking, and a commercial robot arm for slow lateral and axial tracking. We demonstrate the system's performance autonomously aligning with stationary eyes, pursuing moving eyes, and tracking eyes undergoing physiologic motion. The system demonstrates sub-millimeter eye tracking accuracy, 12 μm lateral pupil tracking accuracy, 83.2 ms stabilization time following step disturbance, and 9.7 Hz tracking bandwidth.

Deep anterior lamellar keratoplasty (DALK) is a promising technique for corneal transplantation that avoids the chronic immunosuppression comorbidities and graft rejection risk associated with penetrating keratoplasty (PKP), the standard procedure. In DALK, surgeons must insert a needle 90% through the 500 μm cornea without penetrating its underlying membrane. This pushes surgeons to their manipulation and visualization limits such that 59% of DALK attempts fail due to corneal perforation or inadequate needle depth. We propose a robot-assisted solution to jointly solve the manipulation and visualization challenges using a cooperatively-controlled, precise robot arm and live optical coherence tomography (OCT) imaging, respectively. Our system features an interface handle, with which the surgeon and robot cooperatively hold the tool, and a posterior corneal boundary virtual fixture driven by real-time OCT segmentation. A study in which three operators performed DALK needle insertions manually and cooperatively in ex vivo human corneas demonstrated an 84% improvement in perforation-free needle depth without an increased perforation rate.

Intraoperative 3D imaging has great potential for enhancing surgical visualization. This is especially so in ophthalmic surgery where live volumetric imaging from optical coherence tomography systems recently incorporated into surgical microscopes has freed surgeons from the otherwise universal top-down viewpoint. New viewpoints, however, disorient surgeons when directions of their hand motions and viewed tool motions do not align. We propose introducing a robotic surgery system to decouple surgeons' hands from their tools and ensure that viewed tool motions align in arbitrary viewpoints. We present a framework entitled Arbitrary Viewpoint Robotic Manipulation (AVRM) which governs how hand and tool motions should interact to minimize disorientation and thereby enable operations from desirable but previously untenable viewpoints. A crossover study in which 20 subjects completed mock surgical scenarios with an AVRM testbed system demonstrated that arbitrary viewpoints do not improve task performance unless automatic hand-tool misalignment correction is provided. When provided together with arbitrary viewpoints, automatic hand-tool misalignment correction reduces task completion time by 50% on average compared to a fixed top-down viewpoint.

Intel's newly-announced low-cost RealSense 3D camera claims significantly better precision than other currently available low-cost platforms and is expected to become ubiquitous in laptops and mobile devices starting this year. In this paper, we demonstrate for the first time that the RealSense camera can be easily converted into a real low-cost gaze tracker. Gaze has become increasingly relevant as an input for human-computer interaction due to its association with attention. It is also critical in clinical mental health diagnosis. We present a novel 3D gaze and fixation tracker based on the eye surface geometry captured with the RealSense 3D camera. First, eye surface 3D point clouds are segmented to extract the pupil center and iris using registered infrared images. With non-ellipsoid eye surface and single fixation point assumptions, pupil centers and iris normal vectors are used to first estimate gaze (for each eye), and then a single fixation point for both eyes simultaneously using a RANSAC-based approach. With a simple learned bias field correction model, the fixation tracker demonstrates mean error of approximately 1 cm at 20-30 cm, which is sufficiently adequate for gaze and fixation tracking in human-computer interaction and mental health diagnosis applications.

This paper presents a low-complexity, novel approach to wireless sensor network (WSN) assisted autonomous mobile robot (AMR) navigation. The goal is to have an AMR navigate to a target location using only the information inherent to WSNs, i.e., topology of the WSN and received signal strength (RSS) information, while executing an efficient navigation path. Here, the AMR has neither the location information for the WSN, nor any sophisticated ranging equipment for prior mapping. Two schemes are proposed utilizing particle filtering based bearing estimation with RSS values obtained from directional antennas. Real-world experiments demonstrate the effectiveness of the proposed schemes. In the basic node-to-node navigation scheme, the bearing-only particle filtering reduces trajectory length by 11.7% (indoors) and 15% (outdoors), when compared to using raw bearing measurements. The advanced scheme further reduces the trajectory length by 22.8% (indoors) and 19.8% (outdoors), as compared to the basic scheme. The mechanisms exploit the low-cost, low-complexity advantages of the WSNs to provide an effective method for map-less and ranging-less navigation.