This paper establishes the feasibility of robotically 3D printing biomaterials such as alginate hydrogels onto moving human anatomy and a stationary plane. The alginate hydrogels used are in-vivo compatible and a proven biomaterial for tissue scaffolds. We developed a control scheme for precision material deposition via piezo microjetting while tracking in real-time to continuously sense anatomy location and deposits material in a predefined trajectory derived from two pre-selected target geometries. We show that multilayer 3D structures can be created on a moving human hand with 1.6 mm average error and 87.8 % overall accuracy.

Safe, intuitive human-robot interaction requires that robots intelligently interface with their environments, ideally sensing and localizing physical contact across their link surfaces. We introduce a stretchable smart skin sensor that provides this function. Stretchability allows it to conform to arbitrary robotic link surfaces. It senses contact over nearly the entire surface, localizes contact position of a typical finger touch continuously over its entire surface (RMSE = 7.02mm for a 14.7cm×14.7cm area), and provides an estimate of the contact force. Our approach exclusively employs stretchable, flexible materials resulting in skin strains of up to 150%. We exploit novel carbon nanotube elastomers to create a two-dimensional potentiometer surface. Finite element simulations validate a simplified polynomial surface model to enable real-time processing on a basic microcontroller with no supporting electronics. Using only five electrodes, the skin can be scaled up to arbitrary sizes without needing additional electrodes. We designed, implemented, calibrated, and tested a prototype smart skin as a tactile sensor on a custom medical robot for sensing unexpected physical interactions. We experimentally demonstrate its utility in collaborative robotic applications by showing its potential to enable safer, more intuitive human-robot interaction.

Given the rise in surgeries performed with surgical robots and associated robotics research efforts, tool tracking methods have the potential to provide quantitative feedback concerning surgical performance and establish absolute tool tracking to help advance surgical robotics research. We have created a platform-agnostic method for low-cost tracking of surgical tool shafts in Cartesian space in near real time. We employ a joint Hough Transform - Geometric Constraint approach to locate the tool tips in the stereo camera channels independently. Cartesian coordinates are registered using a custom polynomial depth - disparity model. The algorithm was developed using a low-cost experimental webcam setup and evaluated using a da Vinci surgical endoscope. The algorithm was benchmarked for 3D tracking accuracy and computational speed. The system can locate the tool tip in 3D space with an average accuracy of 3.05 mm at 25.86 frames per second using the webcam setup. For the endoscope setup this algorithm has an average tracking accuracy of 8.68 mm in 3D and 1.88 mm in 2D with an average frame rate of 26.9 FPS. The algorithm also demonstrated successful tracking of tools using captured video from a real surgical procedure.