Steerable needles offer a minimally invasive method to deliver treatment to hard-to-reach tissue regions. We introduce a new class of tape-spring steerable needles capable of sharp turns ranging from 15 to 150 degrees with a turn radius as low as 3 mm, which minimizes surrounding tissue damage. In this work, we derive and experimentally validate a geometric model for our steerable needle design. We evaluate both manual and robotic steering of the needle along a Dubins path in 7 kPa and 13 kPa tissue phantoms, simulating our target clinical application in healthy and unhealthy liver tissue. We conduct experiments to measure needle robustness to stiffness transitions between non-homogeneous tissues. We demonstrate progress towards clinical use with needle tip tracking via ultrasound imaging, navigation around anatomical obstacles, and integration with a robotic autonomous steering system.

Recent progress in reinforcement learning (RL) and tactile sensing has significantly advanced dexterous manipulation. However, these methods often utilize simplified tactile signals due to the gap between tactile simulation and the real world. We introduce a sensor model for tactile skin that enables zero-shot sim-to-real transfer of ternary shear and binary normal forces. Using this model, we develop an RL policy that leverages sliding contact for dexterous inhand translation. We conduct extensive real-world experiments to assess how tactile sensing facilitates policy adaptation to various unseen object properties and robot hand orientations. We demonstrate that our 3-axis tactile policies consistently outperform baselines that use only shear forces, only normal forces, or only proprioception. Videos and details available on the project website.

Visuotactile sensors are a popular tactile sensing strategy due to high-fidelity estimates of local object geometry. However, existing algorithms for processing raw sensor inputs to useful intermediate signals such as contact patches struggle in high-deformation regimes. This is due to physical constraints imposed by sensor hardware and small-deformation assumptions used by mechanics-based models. In this work, we propose a fusion algorithm for proximity and visuotactile point clouds for contact patch segmentation, entirely independent from membrane mechanics. This algorithm exploits the synchronous, high spatial resolution proximity and visuotactile modalities enabled by an extremely deformable, selectively transmissive soft membrane, which uses visible light for visuotactile sensing and infrared light for proximity depth. We evaluate our contact patch algorithm in low ($\mathbf{1 0 \%}$), medium ($\mathbf{6 0 \%}$), and high$(100 \%+)$strain states. We compare our method against three baselines: proximity-only, tactile-only, and a first principles mechanics model. Our approach outperforms all baselines with an average RMSE under 2.8 mm of the contact patch geometry across all strain ranges. We demonstrate our contact patch algorithm in four applications: varied stiffness membranes, torque and shear-induced wrinkling, closed loop control, and pose estimation.

Soft robotic actuators are safe and adaptable devices with inherent compliance, which makes them attractive for manipulating delicate and complex objects. Researchers have integrated stiff materials into soft actuators to increase their force capacity and direct their deformation. However, these embedded materials have largely been pre-prescribed and static, which constrains the actuators to a predetermined range of motion. In this work, electroadhesive (EA) clutches integrated on a single-chamber soft pneumatic actuator (SPA) provide local programmable stiffness modulation to control the actuator deformation. We show that activating different clutch patterns inflates a silicone membrane into pyramidal, round, and plateau shapes. Curvatures from these shapes are combined during actuation to apply forces on both a 3.7 g and 820 g object along five different degrees of freedom (DoF). The actuator workspace is up to 12 mm for light objects. Clutch deactivation, which results in local elastomeric expansion, rapidly applies forces up to 3.2 N to an object resting on the surface and launches a 3.7 g object in controlled directions. The actuator also rotates a heavier, 820 g, object by 5 degrees and rapidly restores it to horizontal alignment after clutch deactivation. This actuator is fully powered by a 5 V battery, AA battery, DC-DC transformer, and 4.5 V (63 g) DC air pump. These results demonstrate a first step towards realizing a soft actuator with high DoF shape change that preserves the inherent benefits of pneumatic actuation while gaining the electrical controllability and strength of EA clutches. We envision such a system supplying human contact forces in the form of a low-profile sit-to-stand assistance device, bed-ridden patient manipulator, or other ergonomic mechanism.

Progress in soft robotics depends on the integration of electronics for sensing, power regulation, and signal processing. Commercially available microelectronics satisfy these functions and are small enough to preserve the natural mechanics of the host system. Here, we present a method for incorporating microelectronic sensors and integrated circuits (ICs) into the elastomeric skin of a soft robot. The thin stretchable skin contains various solid-state electronics for orientation, pressure, proximity, and temperature sensing, and a microprocessor. The components are connected by thin-film copper traces wetted with eutectic gallium indium (EGaIn), a room temperature liquid metal alloy that allows the circuit to maintain conductivity as it deforms under mechanical loading. In this paper, we characterize the function of the individual sensors in air and water, discuss the integration of the microelectronic skin with a shape-memory actuated soft gripper, and demonstrate the sensorized soft gripper in conjunction with a 4 degree-of-freedom (DOF) robot arm.

Recent technical advances have enabled new wearable augmented reality (AR) solutions that can aid people with visual impairments (VI) in their everyday lives. Here, we investigate an AR-based magnification solution that combines a small finger-worn camera with a transparent augmented reality display (the Microsoft Hololens). The image from the camera is processed and projected on the Hololens to magnify visible content below the user's finger such as text and images. Our approach offers: (i) a close-up camera view (similar to a CCTV system) with the portability and processing power of a smartphone magnifier app, (ii) access to content through direct touch, and (iii) flexible placement of the magnified image within the wearer's field of view. We present three proof-of-concept interfaces and plans for a user evaluation.