One of the main barriers to immersivity during object manipulation in virtual reality is the lack of realistic haptic feedback. Our goal is to convey compelling interactions with virtual objects, such as grasping, squeezing, pressing, lifting, and stroking, without requiring a bulky, world-grounded kinesthetic feedback device (traditional haptics) or the use of predetermined passive objects (haptic retargeting). To achieve this, we use a pair of finger-mounted haptic feedback devices that deform the skin on the fingertips to convey cutaneous force information from object manipulation. We show that users can perceive differences in virtual object weight and that they apply increasing grasp forces when lifting virtual objects as rendered mass is increased. Moreover, we show how naive users perceive changes of a virtual object's physical properties when we use skin deformation to render objects with varying mass, friction, and stiffness. These studies demonstrate that fingertip skin deformation devices can provide a compelling haptic experience appropriate for virtual reality scenarios involving object manipulation.

During robot-assisted minimally invasive surgery, surgeons perform challenging dexterous tasks, including the manipulation of soft tissue and suture tying. In the absence of environment force sensing of tool-tissue interaction forces to provide force feedback, surgeons must rely on visual feedback to modulate the grip force they apply on the environment. Clinical systems, like the da Vinci Surgical System (Intuitive Surgical, Inc.), use physical springs to provide closing resistance on the gripper degree-of-freedom (DOF) of the master manipulator. This feedback provides increasing force resistance as the gripper is closed. To determine the effect of master manipulator gripper stiffness on performance in a teleoperated manipulation task, we designed a new and open source gripper, the OmniGrip. The OmniGrip attaches to a SensAble Phantom Omni (now available as Geomagic Touch), replacing the stySensAble Phantom Omnilus end effector, and providing the ability for user programmable force characteristics. We conducted a study in which participants used an OmniGrip to teleoperate a Raven II surgical robRaven II surgical robotic systemotic system in a pick-and-place task. Increasing the stiffness of the OmniGrip resulted in reduced interaction forces at the slave-side environment. Additionally, these interaction forces were significantly lower when the OmniGrip as compared to when using the Phantom Omni stylus.

During multi-lateral collaborative teleoperation, where multiple human or autonomous agents share control of a teleoperation system, it is important to be able to convey individual user intent. One option for conveying the actions and intent of users or autonomous agents is to provide force guidance from one user to another. Under this paradigm, forces would be transmitted from one user to another in order to guide motions and actions. However, the use of force guidance to convey intent can mask environmental force feedback. In this paper we explore the possibility of using tactile feedback, in particular skin deformation feedback, skin deformation feedback to convey collaborative intent while preserving environmental force perception. An experiment was performed to test the ability of participants to use force guidance and skin deformation guidance to follow a path while interacting with a virtual environment. In addition, we tested the ability of participants to discriminate virtual environment stiffness when receiving either force guidance or skin deformation guidance. We found that skin deformation guidance resulted in a reduction of path-following accuracy, but increased the ability to discriminate environment stiffness when compared with force feedback guidance.

When a person interacts with an environment using a tool, he/she receives tactile information in the form of fingerpad skin deformation. Different interaction forces and torques on the tool cause different skin deformation patterns on the fingerpads. We designed a 6-degree-of-freedom tactile device that creates similar skin deformation patterns on the fingerpads. The device communicates force and torque information by translating and rotating skin deformation tactors relative to the fingerpads. An experiment was conducted to determine participants' ability to use skin deformation tactile cues to perform a peg-in-hole insertion task. Results show that participants can use the tactile cues to reduce interaction force and torque, and they use the tactile force cues to reduce interaction force more than they use the tactile torque cues to reduce interaction torque. Rendering force and torque cues simultaneously causes device saturation and degrades user performance. These results suggest that additional training may help participants use the skin deformation torque cues, and motivate a tactile device design that decouples force and torque skin deformation rendering to minimize device saturation. Fingerpad skin deformation is a promising form of tactile feedback to convey force and torque information in teleoperation systems such as robot-assisted surgery, where force feedback may be undesirable due to stability and safety concerns.

When we use a tool to explore or manipulate an object, friction between the surface of the tool and the fingerpads generates skin stretch cues that are related to the interaction forces between the tool and the object. In this study, we emulate these naturally occurring skin stretch cues in order to convey force direction and magnitude information to users during teleoperation. We hypothesize that skin stretch feedback is a useful substitute for kinesthetic force feedback in force-sensitive teleoperated tasks. In this study, ten participants performed teleoperated palpation to determine the orientation of a stiff region in a surrounding artificial tissue using five feedback conditions: skin stretch, force, reduced gain force, graphic, and vibration. When participants received skin stretch feedback, they localized the stiff region as well as with force feedback, with no increase in task completion time. Additionally, participants receiving skin-stretch feedback localized the stiff region statistically significantly more accurately than those using vibration feedback. Although participants using skin stretch exhibited higher interaction forces than when using force, vibration, and graphical feedback, skin stretch statistically significantly decreased interaction forces compared with reduced gain force feedback. Thus, skin-stretch feedback is a compelling substitute for force feedback and may be useful in scenarios where force feedback is reduced or infeasible.

During tool-mediated interactions with objects, we experience force and fingerpad skin stretch resulting from shear forces caused by friction between the fingerpad skin and the stylus. When probing an object, for the same penetration distance, a stiffer object causes a larger load force and, thus, greater fingerpad skin stretch. We hypothesized that rendering additional artificial skin stretch together with force will increase perceived stiffness. We created a Skin Stretch Stylus that renders skin stretch through tactor displacement, attached it to a force-feedback device, and performed a study to characterize the effect of tactor displacement-induced skin stretch on stiffness perception. Results showed that adding artificial skin stretch causes additive augmentation of perceived stiffness across a range of surface stiffness, and the addition is a linear function of tactor displacement gain. However, intersubject variability in the estimated slope coefficient was large. We propose a model that explains the additive effect and suggests potential sources for the intersubject variability. We conclude that augmenting force feedback with skin stretch can increase users' perception of stiffness, but the effect is user-specific. Such augmentation may be useful in virtual environment and teleoperation scenarios when force feedback gains must be kept low to prevent feedback-induced instabilities, or when force feedback is limited due to actuator force limits.

When interacting with everyday objects, we experience kinesthetic force feedback as well as various forms of cutaneous tactile feed-back. Skin stretch is part of the cutaneous tactile experience that is caused by friction between the skin and the grasped object. Interacting with stiffer objects causes larger force, and results in a larger amount of skin stretch. Therefore, we hypothesize that adding artificial fingerpad skin stretch to kinesthetic force feedback will increase users' perception of stiffness. A tactile display called the Skin Stretch Stylus was designed to augment kinesthetic force feedback with skin stretch feedback. The change in users' stiffness perception due to the addition of skin stretch feedback is quantified through a two-alternative forced-choice paradigm, method of constant stimuli experiment. In this experiment, subjects compared the stiffness of virtual springs with kinesthetic force feedback augmented with skin stretch feedback versus virtual springs with only kinesthetic force feedback. Results show that the addition of skin stretch causes a significant increase in the perception of stiffness, and this effect increases with higher amount of applied skin stretch. These results indicate that skin stretch feedback could be used to augment perceived stiffness in situations where it is not possible to increase force feedback gains. Such scenarios include teleoperation systems where force feedback gains must remain low to ensure stability, and haptic devices with limited actuator force.

Skin stretch is a novel haptic feedback method that can provide a human operator with information about the magnitude and direction of an applied force. To evaluate the potential for skin stretch feedback to be used as a sensory substitute for kinesthetic (force) feedback in robotic teleoperation systems, a study was conducted to measure the ability of users to discriminate environment stiffness using varying levels of fingerpad skin stretch instead of force feedback. A new, high-fidelity skin stretch feedback device was developed that imposes tangential fingerpad skin stretch in proportion to the intended level of force feedback. In psychophysical experiments, users received skin stretch feedback with magnitude proportional to the users' penetration depth into a virtual wall. Users' stiffness discrimination capability using skin stretch was comparable to that of using force feedback. Furthermore, larger skin stretch cues were perceived by users as portraying greater stiffness without any advance training, which indicates that skin stretch feedback would be an intuitive sensory substitute for force feedback. Thus, skin stretch feedback is a promising method for conveying kinesthetic force information in applications such as robot-assisted surgery, where high levels of force feedback may not be desirable due to stability or safety concerns.