Virtual and augmented reality systems that immerse users in a 3D environment could benefit from haptic (force and/or tactile) feedback to increase realism and task performance. A canonical task that cannot be achieved compellingly without haptic feedback is the grasp and lift of an object with mass. Traditional kinesthetic (force) feedback devices provide realistic physical interactions with virtual objects, but typically require large actuators, making them bulky and encumbering. In contrast, tactile feedback devices can use smaller actuators and may enable freer movement of a user in a virtual world. One form of tactile feedback, skin deformation feedback, has been previously shown to provide effective haptic feedback of force, stiffness, and friction. Here we compare human perception of mass via kinesthetic feedback and skin deformation feedback in a virtual environment. Participants grasp and lift two virtual blocks and attempt to equalize their masses by the method of adjustments. From the accuracy of this equalization, we determined that the Weber Fraction (the just noticeable difference in proportion to the original stimulus value) of virtual mass during a grasp and lift task was 0.11 for kinesthetic force feedback and 0.35 for skin deformation feedback. In addition, participants exhibited differences in exploratory procedures between the two types of feedback.

During multi-lateral collaborative teleoperation, where multiple human or autonomous agents share control of a teleoperation system, it is important to be able to convey individual user intent. One option for conveying the actions and intent of users or autonomous agents is to provide force guidance from one user to another. Under this paradigm, forces would be transmitted from one user to another in order to guide motions and actions. However, the use of force guidance to convey intent can mask environmental force feedback. In this paper we explore the possibility of using tactile feedback, in particular skin deformation feedback, skin deformation feedback to convey collaborative intent while preserving environmental force perception. An experiment was performed to test the ability of participants to use force guidance and skin deformation guidance to follow a path while interacting with a virtual environment. In addition, we tested the ability of participants to discriminate virtual environment stiffness when receiving either force guidance or skin deformation guidance. We found that skin deformation guidance resulted in a reduction of path-following accuracy, but increased the ability to discriminate environment stiffness when compared with force feedback guidance.

During robotic teleoperation, the dynamics of the master manipulators and the control of remote-side instruments impose challenges on the motor system of the human operator, and may impact performance and learning. In teleoperated robot-assisted minimally invasive surgery, there is a clear correlation between patient outcomes and the surgeon's case experience. However, the effect of the teleoperator on human motor skills and the relationship between these motor skills and patient outcomes are unknown. We used the da Vinci Research Kit, a custom research version of the da Vinci Surgical System, to compare teleoperated and open needle-driving movements of experienced da Vinci surgeons and novices. The experimental protocol consisted of structured but unconstrained needle driving trials repeated 80 times to allow for computational modeling of movement coordination and learning. Kinematic analysis showed that teleoperation increases trial time but reduces path length, that the trial times and path lengths of experienced surgeons are smaller than those of novices. In addition, there are significant differences in learning between experienced surgeons and novice users. Modeling of the movements and learning processes of experienced and novice surgeons may be used in the design of novel controllers that will expand robotic surgery capabilities and improve robot-assisted surgical skill acquisition.

When a person interacts with an environment using a tool, he/she receives tactile information in the form of fingerpad skin deformation. Different interaction forces and torques on the tool cause different skin deformation patterns on the fingerpads. We designed a 6-degree-of-freedom tactile device that creates similar skin deformation patterns on the fingerpads. The device communicates force and torque information by translating and rotating skin deformation tactors relative to the fingerpads. An experiment was conducted to determine participants' ability to use skin deformation tactile cues to perform a peg-in-hole insertion task. Results show that participants can use the tactile cues to reduce interaction force and torque, and they use the tactile force cues to reduce interaction force more than they use the tactile torque cues to reduce interaction torque. Rendering force and torque cues simultaneously causes device saturation and degrades user performance. These results suggest that additional training may help participants use the skin deformation torque cues, and motivate a tactile device design that decouples force and torque skin deformation rendering to minimize device saturation. Fingerpad skin deformation is a promising form of tactile feedback to convey force and torque information in teleoperation systems such as robot-assisted surgery, where force feedback may be undesirable due to stability and safety concerns.

When we use a tool to explore or manipulate an object, friction between the surface of the tool and the fingerpads generates skin stretch cues that are related to the interaction forces between the tool and the object. In this study, we emulate these naturally occurring skin stretch cues in order to convey force direction and magnitude information to users during teleoperation. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. In this study, ten participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using five feedback conditions: skin stretch, force, reduced gain force, graphic, and vibration. When participants received skin stretch feedback, they localized the stiff region as well as with force feedback, with no increase in task completion time. Additionally, participants receiving skin-stretch feedback localized the stiff region statistically significantly more accurately than those using vibration feedback. Although participants using skin stretch exhibited higher interaction forces than when using force, vibration, and graphical feedback, skin stretch statistically significantly decreased interaction forces compared with reduced gain force feedback. Thus, skin-stretch feedback is a compelling substitute for force feedback and may be useful in scenarios where force feedback is reduced or infeasible.

An active needle is proposed for the development of magnetic resonance imaging (MRI)-guided percutaneous procedures. The needle uses a low-transition-temperature shape memory alloy (LT SMA) wire actuator to produce bending in the distal section of the needle. Actuation is achieved with internal optical heating using laser light transported via optical fibers and side coupled to the LT SMA. A prototype, with a size equivalent to a standard 16-gauge biopsy needle, exhibits significant bending, with a tip deflection of more than 14° in air and 5° in hard tissue. A single-ended optical sensor with a gold-coated tip is developed to measure the curvature independently of temperature. The experimental results in tissue phantoms show that human tissue causes fast heat dissipation from the wire actuator; however, the active needle can compensate for typical targeting errors during prostate biopsy.

During tool-mediated interactions with objects, we experience force and fingerpad skin stretch resulting from shear forces caused by friction between the fingerpad skin and the stylus. When probing an object, for the same penetration distance, a stiffer object causes a larger load force and, thus, greater fingerpad skin stretch. We hypothesized that rendering additional artificial skin stretch together with force will increase perceived stiffness. We created a Skin Stretch Stylus that renders skin stretch through tactor displacement, attached it to a force-feedback device, and performed a study to characterize the effect of tactor displacement-induced skin stretch on stiffness perception. Results showed that adding artificial skin stretch causes additive augmentation of perceived stiffness across a range of surface stiffness, and the addition is a linear function of tactor displacement gain. However, intersubject variability in the estimated slope coefficient was large. We propose a model that explains the additive effect and suggests potential sources for the intersubject variability. We conclude that augmenting force feedback with skin stretch can increase users' perception of stiffness, but the effect is user-specific. Such augmentation may be useful in virtual environment and teleoperation scenarios when force feedback gains must be kept low to prevent feedback-induced instabilities, or when force feedback is limited due to actuator force limits.

When interacting with everyday objects, we experience kinesthetic force feedback as well as various forms of cutaneous tactile feed-back. Skin stretch is part of the cutaneous tactile experience that is caused by friction between the skin and the grasped object. Interacting with stiffer objects causes larger force, and results in a larger amount of skin stretch. Therefore, we hypothesize that adding artificial fingerpad skin stretch to kinesthetic force feedback will increase users' perception of stiffness. A tactile display called the Skin Stretch Stylus was designed to augment kinesthetic force feedback with skin stretch feedback. The change in users' stiffness perception due to the addition of skin stretch feedback is quantified through a two-alternative forced-choice paradigm, method of constant stimuli experiment. In this experiment, subjects compared the stiffness of virtual springs with kinesthetic force feedback augmented with skin stretch feedback versus virtual springs with only kinesthetic force feedback. Results show that the addition of skin stretch causes a significant increase in the perception of stiffness, and this effect increases with higher amount of applied skin stretch. These results indicate that skin stretch feedback could be used to augment perceived stiffness in situations where it is not possible to increase force feedback gains. Such scenarios include teleoperation systems where force feedback gains must remain low to ensure stability, and haptic devices with limited actuator force.

Skin stretch is a novel haptic feedback method that can provide a human operator with information about the magnitude and direction of an applied force. To evaluate the potential for skin stretch feedback to be used as a sensory substitute for kinesthetic (force) feedback in robotic teleoperation systems, a study was conducted to measure the ability of users to discriminate environment stiffness using varying levels of fingerpad skin stretch instead of force feedback. A new, high-fidelity skin stretch feedback device was developed that imposes tangential fingerpad skin stretch in proportion to the intended level of force feedback. In psychophysical experiments, users received skin stretch feedback with magnitude proportional to the users' penetration depth into a virtual wall. Users' stiffness discrimination capability using skin stretch was comparable to that of using force feedback. Furthermore, larger skin stretch cues were perceived by users as portraying greater stiffness without any advance training, which indicates that skin stretch feedback would be an intuitive sensory substitute for force feedback. Thus, skin stretch feedback is a promising method for conveying kinesthetic force information in applications such as robot-assisted surgery, where high levels of force feedback may not be desirable due to stability or safety concerns.

A side optical actuation method is presented for a slender MR-compatible active needle. The needle includes an active region with a shape memory alloy (SMA) wire actuator, where the wire generates a contraction force when optically heated by a laser delivered though optical fibers, producing needle tip bending. A prototype, with multiple side heating spots, demonstrates twice as fast an initial response compared to fiber tip heating when 0.8 W of optical power is applied. A single-ended optical sensor with a gold reflector is also presented to measure the curvature as a function of optical transmission loss. Preliminary tests with the sensor prototype demonstrate approximately linear response and a repeatable signal, independent of the bending history.

A new generation of robots is being designed for human occupied workspaces where safety is of great concern. This research demonstrates the use of a capacitive skin sensor for collision detection. Tests demonstrate that the sensor reduces impact forces and can detect and characterize collision events, providing information that may be used in the future for force reduction behaviors. Various parameters that affect collision severity, including interface friction, interface stiffness, end tip velocity and joint stiffness irrespective of controller bandwidth are also explored using the sensor to provide information about the contact force at the site of impact. Joint stiffness is made independent of controller bandwidth limitations using passive torsional springs of various stiffnesses. Results indicate a positive correlation between peak impact force and joint stiffness, skin friction and interface stiffness, with implications for future skin and robot link designs and post-collision behaviors.

Joint stiffness plays an important role in both safety and control performance, particularly in human-friendly robots using artificial pneumatic muscles. Due to the limited control bandwidth of pneumatic muscles, stiffness characteristics and their effects on safety in the frequency domain should be taken into account. This paper introduces the concept of instantaneous stiffness and validates its model with the Stanford Safety Robot (S2ρ. The potential effects of instantaneous stiffness on safety is explored through experimental comparison of peak impact accelerations under various impact conditions. Instantaneous stiffness demonstrates different effects on the impact acceleration depending on impact velocity and controller gain. Finally, the paper discusses the stiffness characteristics as a guideline for design and control to improve the robot safety while maintaining the control performance.