Nowadays, touch remains essential for emotional conveyance and interpersonal communication as more interactions are mediated remotely. While many studies have discussed the effectiveness of using haptics to communicate emotions, incorporating affect into haptic design still faces challenges due to individual user tactile acuity and preferences. We assessed the conveying of emotions using a two-channel haptic display, emphasizing individual differences. First, 24 participants generated 187 haptic messages reflecting their immediate sentiments after watching 8 emotionally charged film clips. Afterwards, 19 participants were asked to identify emotions from haptic messages designed by themselves and others, yielding 593 samples. Our findings suggest potential links between haptic message decoding ability and emotional traits, particularly Emotional Competence (EC) and Affect Intensity Measure (AIM). Additionally, qualitative analysis revealed three strategies participants used to create touch messages: perceptive, empathetic, and metaphorical expression.

Text-based Computer-Mediated Communication (CMC) diminishes the social presence of the conversational partner owing to the lack of nonverbal cues exchanged in face-to-face (FtF) communication. This study aims to enhance social presence in a dyadic text chat by employing a robot avatar that supplements nonverbal cues. In particular, we propose a chat system with a robot avatar that can express gestures in response to the user’s actions (typing, sending messages) and read the user’s messages aloud. In the evaluation experiment, nine pairs of participants (for a total of 18 participants) used two types of chat system: the proposed system and the baseline system in which a robot avatar only reads messages aloud. As a result, the proposed robot avatar significantly increased social presence compared to the baseline system. Furthermore, the proposed gesture expressions significantly improved the ease of chatting. Specifically, our results suggest that robot gestures in response to typing might have an effect similar to nonverbal cues in FtF communication.

This study aims to investigate the effects of sharing operations on the training of an avatar robot. In this study, we employed an operational platform for the avatar robot called OriHime-T, which integrates head and hand movements with wheel-based mobility. This platform allows both an expert and a learner to share the same control screen during training. Two tasks were conducted: the Time Attack and the Candy Delivered tasks in which candy is delivered to a customer. The experiments showed that compared to the conventional method in which experts observe learners’ operations and provide advice, our proposed method, which incorporates immediate intervention, led to a reduction in Time Attack completion times and an improvement in the success rate of the Candy Delivered task. These results suggest that sharing operations effectively facilitates the transmission of abstract judgment criteria and operational sensations.

The human arm has seven DoF. If the human arm were more redundant, it would improve workability. However, intuitive manipulation of redundant arms is difficult due to the complexity of the input system or the limitations of the human arm. In this study, we constructed a system in which multiple operators integrate their bodies to intuitively operate a redundant multi-DoF robotic arm. The robotic arm was assembled by connecting two arms, and the roles of each section –distal arm and proximal arm– were assigned to different operators. First, vibrotactile feedback methods to the users based on the operation movement were investigated to compensate for the recognition of the partner’s movement. Three types of feedback methods —the actions of both oneself and one’s partner, the actions of the partner, and none— were compared in order to improve the body awareness. It was shown that feedback of the movement from the partner who operates the proximal robotic arm significantly enhanced the sense of body ownership. Then, the proposed co-operation of the redundant robotic arm was evaluated in the workability. Two types of manipulation methods —a single-operator system that switched the target robotic arm and a multi-DoF body-integration system used by two people— were compared through an object-manipulation task involving obstacle avoidance. It was shown that the two-person co-operation through body integration system to be faster than the single-person operation.

Teleoperation makes it possible to work in in-accessible environment. To do so safely and effectively, some means of force feedback is necessary. However, force feedback methods that involve a reaction force often require bulky equipment. Instead, our study has utilized force feedback using vibrotactile stimulation. Although the modulation method of force information to vibrotactile stimulation has previously been demonstrated in magnitude (AM) and frequency (FM), we propose a method called AFM that uses amplitude and frequency modulation simultaneously to improve performance. In particular, we adjusted the amplitude using the sensation level and used logarithmized frequency for the AFM. First, the vibrotactile detection threshold of participants was measured to determine the amplitude to be equivalent in the same sensation level among participants. Then, the performance of discrimination was evaluated through stair-case psychophysical experiments. The results showed that our proposal AFM method significantly improved the discrimination sensitivity for the force information as compared with AM and FM methods.

Humans have spatiotemporal limitations. Robotic avatar technology enables the expansion of an individual's innate physical characteristics, physical capabilities, and perceptual and cognitive abilities. Our research group has focused on scalability and co-creativity through co-operation and collaboration between humans, and has developed a system in which two operators are physically integrated into a single robotic arm. In particular, to enable more human-like tasks, such as receiving and delivering, we extended the system to a dual-arm robotic avatar and built a system in which multiple persons can integrate their bodies, with the expectation of further task efficiency and human extension. Specifically, we investigated the workability and operator cognition of the dual-arm robotic avatar when it was operated by one, two, or three individuals. The results showed that stability improved with the three-person operation. Furthermore, it was found that co-operation among multiple persons can overcome the original individual abilities for operation. In the three-person condition, all participants obtained a higher feeling of control than the actual amount of manipulation, which has potential applications in skill transfer. The three-person condition tended to be better, validating the usefulness of multi-person body integration.

Tactile sensation is essential for recognizing mechanical interactions between the human body and its environment; it is generally uniquely experienced by individuals. Haptic sharing technologies enable to share of other person’s (or robot’s) tactile sensations, and it is expected to apply for skill learning, collaborative work, and robot operation. While wearable tactile sensors are useful for tactile measurements, they are affected by differences in physical characteristics between individuals. Therefore, to accurately convey the tactile information felt by others and utilize it for work, it is necessary to remove individual differences due to differences in individual physical characteristics. In this study, based on the frequency based intensity of skin vibration of an individual, the vibration detected by the acceleration sensor affixed to the fingertip was modulated to remove individual differences. To confirm the effectiveness of the proposed method, we investigated both physical and perceptual similarities for skin vibrations collected from three persons with three different fine textures. The results showed that vibrations individually modulated with the proposed method was similar in the intensity and power spectrum density than those using the uniform modulation function, and was perceived similar when presenting those with a vibrator.

Tactile sensation contributes to sensory-motor control and facilitates skillful manipulation of objects. If the tactile sensation can be shared between two partners, the state of each partner can be partially observed by the other, and the possibilities for cooperative work may be expanded. Recently, examples of utilizing tactile sensation in human-robot cooperative work have been proposed. For cooperative work between people, it is necessary to investigate the basic ability of humans to identify other person's motions and operations by tactile presentation. To avoid hindering work performed with the hands and fingertips, the sensation must be presented elsewhere. This study investigates the possibility of identifying other person's tool operations by presenting tactile information induced on his/her finger with a wearable vibrator on the arm. A wearable skin vibration sensor was employed to acquire tactile information during an experiment in which five different tool operations were tested. This sensor measures skin vibration while directly touching the target. We proposed a non-linear signal processing function to adjust the intensity of the skin vibration to within the range of human sensitivity for tactile presentation. We compared vibrotactile stimulation between the non-linear and linear corrections, and then conducted experiments on identifying operations. The results showed that the non-linear correction increased small signals and enhanced the variance of large signals, and that operations were significantly identified by tactile presentation to the arm.

Contracture is assessed by the palpation performed by a physician or physical therapist based on the range of motion and slidability of a joint. However, the diagnosis of contracture depends on the experience and skills of the therapist and is subjective. Quantification could improve the efficiency and accuracy of the diagnosis. Therefore, we aim to quantify contracture palpation using a wearable tactile sensor by focusing on the frictional vibrations that are generated by the disturbance in the slidability during palpation. In a previous study, the pulse density that was measured by detecting a pulse from the sensor output at a threshold exhibited a good relationship with the subjective rating of the frictional vibration. However, as this sensor measures the propagational vibrations of the skin, it is influenced by the characteristics of individual therapist's fingers. Thus, in this paper, three types of pulse detection threshold setting methods (constant, adjustment with a sine wave, and adjustment with white noise) are proposed. While conducting experiments, the same frictional vibration was applied on several participants by reproducing the frictional vibrations with a vibrator as a tactile display. The results demonstrated that the threshold adjusting method with white noise, which overlapped with the frequency band of the frictional vibrations, significantly reduced the variance in the finger characteristics.

In obstetrics and gynecology, internal examination, i.e., palpation of the cervix, is performed to diagnose the risk of miscarriage and preterm birth based on its hardness. However, the diagnosis is subjective and requires high skill. The introduction of technologies for quantifying cervical stiffness can increase the accuracy of the diagnosis. Various softness sensors have been proposed, but because the route to the cervix is curved and narrow, they are not suitable for the internal examination. Safety for pregnant women and fetuses must be also taken into consideration. In this paper, we propose a soft wearable tactile sensor made entirely of silicone rubber. The structure is simple and the sensing principle is based on acoustic reflection, which is safe because no electricity flows inside the body and its texture is soft. Furthermore, the sensor is worn at the clinician’s fingertip thus allowing him/her to make use of his/her skill. The sensor measures contact forces at three points of the fingertip considering human’s softness perception. Experimental results showed that the fabricated sensor is capable of determining two magnitudes of hardness (Young’s modulus 120 kPa and 230 kPa) within a fingertip orientation from 30° to 60°.

Robots have replaced people in many manufacturing production lines but the information they gather from sensors might not be sufficient to autonomously accomplish dexterous manipulation operations. Symbiotic human-robot cooperation appears to be a more realistic near future in industrial scenarios. In this paper we present a configuration of human-robot collaboration in which the robot is sensory-augmented by means of a set of tactile signals coming from the human operator. The incorporation of low-level robot “intelligence” permits the cooperative manipulation of an object while enabling the human operator to stay focused on task itself and carry it out in the most natural way. The effectiveness of this approach is demonstrated in a use case in which a robot helps a human operator to successfully accomplish a writing task. System performance has been evaluated, considering several positions of the tiny vibration sensor in charge of gathering the human perception, by testing it on both the human hand and the co-manipulated object. Results suggest that the sensor provides valuable information for recognizing operator actions when it is placed either on the human hand or on the co-manipulated object. However, the sensor on the finger directly represents the operator's perception, while the output of the sensor attached to the object changes according to the distance between the interaction point and the sensor itself. In addition, in wearing the sensor, neither the object nor the robot need to be instrumented: the operator is free to interact with a large set of objects and collaborate with any existing robot without requiring supplemental equipment.

When we touch an object, we can obtain tactile information through our skin. Thus it is important to reveal the relationship between the subjective sensation and objective quantities like the skin vibration elicited on our skin. In this paper, we focus on roughness perception, one of the main tactile sensations of materials. We measured skin vibration and contact force in natural active roughness perception for a large set of different materials, of which the physical roughness could be expressed in various industrial roughness measures like Ra. The results showed that there was no relationship between contact force and subjective roughness, so apparently humans do not control contact forces intentionally in a roughness estimation task. Power functions were fitted to each relationship between the other objective parameters and estimated roughness. The goodness-of-fit value of skin vibration was significantly better than any of the other parameters. Thus, the main conclusion is that these results suggest that skin vibration plays a determining role in roughness perception.

We developed HapticAid, which is a wearable haptic augmentation system. This system enhances the skin vibrations that occur during touch and provide them as haptic feedback on the wrist. We revised HapticAid to be wearable and mobile by using a mobile platform and installed a digital signal processing to control haptic feedback for haptic experiences. This paper describes the implementation of our system, design of digital signal processing the method of wearing our system for haptic experiences.

This paper aims to develop a wearable tactile sensor for measuring skin vibrations using a polyvinylidene fluoride (PVDF) film, which is a polymer piezo material. The sensor is worn on the finger pad where is remote from contact fingertip and detects skin-propagated vibrations when fingertip touches an object. The proposed sensor allows users to touch with bare fingers and to conduct active touch. A transfer function from vibrations applied on the fingertip to the sensor output is expressed by using a finger model, a sensor model, and an electric model of the PVDF film. On the basis of the transfer function, frequency response of the sensor is measured and estimation of vibrations is tested. Furthermore, the sensor output is investigated for three materials with different textures. Results show the validity and availability of the sensor.

We have developed a forceps-type tactile sensor for palpation of a tumor in laparoscopic surgery. The sensor responds to contact position and force at deformable cavity in the sensor probe by using acoustic reflection. The sensor has high applicability for laparoscopic surgery because of its simple structure and disuse of electrical elements within inserted part into patient's body. In this study, we try to measure softness of a contact tissue by using the sensor. Human can estimate softness from contact area and force. Indentation depth is not always required for softness perception. This perception mechanism suggests that our sensor has a possibility for softness measurement because the sensor can respond to contact position and force. Softness measurement contributes to tumor localization and improvement of diseases database. First, the sensor output characteristic is investigated for different soft samples. Then, a method of stiffness measurement using the sensor outputs is proposed. Experimental results show that the sensor has the capability of softness measurement without any additional sensors.

Haptic perception is bidirectionally related to the exploratory movements. This means that exploration influences perception, but also perception influences exploration. We can optimize/change the exploratory movements according to the perception and/or the task, consciously or unconsciously. In this paper, a tactile sensing system including bidirectionality is proposed and its potential and application to laparoscopic surgery are discussed. The proposed system is composed of a tactile sensor and a tactile display. The tactile sensor is mounted on a forceps, and the tactile feedback based on the sensor output is provided by using the tactile display. The user can feel sensor outputs and adjust the forceps manipulation according to the environment and the target. This leads to collect adequate sensor outputs. Implementing the bidirectionality gets the most out of the capability of the tactile sensor. Furthermore, the developed system might have an additional effect due to using the tactile display to the other hand, not to the hand with the forceps. It is an enhancement of haptic perception by tactile feedback to distant part. The developed system is introduced and establishment of the bidirectionality and its effect are discussed. Psychophysical experiments on the enhancing effect are conducted. Experimental results show a potential of the proposed tactile sensing system.

The development of modern technologies, touch and/or tactile sensors have been well-developed, exploring various possible methods of transduction and available in many commercial products. The demand for more friendly and safe products, such as household robots, is growing up day by day. But the issues of tactile sensors such as wiring and robustness still remain. By researching human mechanoreceptors, we got an idea of a novel type of tactile sensor based on tissue engineering, which is expected to provide some solutions for those issues. In this paper, we propose the basic concept of tactile sensor based on tissue engineering and the hypotheses of approaching methods.

Human tactile perception incorporates two important characteristics - self-reference and bidirectionality. Self-reference is an important factor when evaluating tactile sensations. Bidirectionality contributes to sensing capability enhancement. In this paper, a tactile sensor including self-reference and bidirectionality is presented. The sensor with two microphones is mounted on a human finger. Users can apply the sensor while retaining their normal tactile perceptions and simultaneously obtaining sound and skin deformation data based on the mechanical interaction between the finger and the object. Experimental results on roughness evaluations and convexity detections show the validity of the proposed sensing method and its potential for use in various applications.

The aim of this study is the development of a human-friendly tactile sensor for measuring softness, sliminess, smoothness, and so on. We have proposed active tactile sensing method using balloon expansion. A balloon is contacted with an object and expanded by using fluid. Tactile information is evaluated by measuring the expansion process. In this paper, we developed a sensor system using syringe. It is a compact system and can ensure enough safety for human body tissues in safe contact, using no electric power, and sterilization. Pressure changes and volume changes of the balloon can be measured together in the balloon expansion. First, overview of the developed sensor system is presented. Next, composition of the sensor system and the sensing process are presented. Then, experiments using the developed sensor are conducted on samples with different stiffness and surface conditions. Features extracted from sensor outputs indicate that the sensor can know the difference both the stiffness and surface condition. The results show the validity of the proposed sensor system.

This paper presents the development of a finger-mounted tactile sensor for surface irregularity inspection. The sensor is composed of a thin rubber sheet and a silicone rubber with the void in the center. On the center of the rubber sheet, two strain gauges are attached point-symmetrically. In addition, a silicone oil sheet is attached on the bottom of the sensor as a viscous fluid medium for keeping steady contact with the object surface. For mounting the sensor on the finger, the sensor is embedded in a cotton work glove. The user puts his/her finger on the silicone rubber of the sensor and slides the sensor over the object surface. By obtaining difference between the two strain gauges, the sensor responds to the surface irregularity, not to the uniform surface. Theoretical analysis of sensor output signals is presented and real-time signal processing based on the wavelet transform for detection and evaluation of surface irregularity is introduced. Then, experiments using the developed sensor are carried out on some protrusions. The results show the availability of the proposed tactile sensor.

This paper presents the development of an active tactile sensor using fluid. Measurement of stiffness and surface conditions, such as sliminess, smoothness, and others, of body tissue is discussed. This sensor is made of a silicone balloon catheter. The balloon is contacted with a soft object and expanded by using fluid. In the process of the expansion, the balloon pushes the object and the contact surface of the balloon slips or sticks. These phenomena depend on stiffness and surface conditions of the object. The expansion can be evaluated by measuring the flow of the fluid. The balloon that is a sensing element is very soft and flexible and does not need electricity. This sensor is available as a medical sensor since it provides safety for human. Experiments using the sensor are carried out on soft samples with various stiffness and surface conditions. It is confirmed that the sensor output are different from each sample. Furthermore, some experiments under different conditions on inner pressure and load are conducted. The results show that the sensor has a potential to measure stiffness and surface conditions.

This paper presents the development of a finger-mounted tactile sensor for surface irregularity detection. The sensor is composed of a thin rubber sheet and a silicone rubber with the void in the center. On the center of the rubber sheet, two strain gauges are attached point-symmetrically. The user puts his/her finger on the silicone rubber of the sensor and slides the sensor over the objective surface. For mounting the sensor on the finger, the sensor is embedded in a cotton work glove. By obtaining difference between the two strain gauges, the sensor responds to the surface irregularity. It does not respond to the uniform surface. Furthermore, real-time signal processing utilizing the wavelet transform is proposed for detecting surface irregularity from sensor output signals. Experiments are carried out on the plane surface and curved surface with a small protrusion. Moreover, the sensor is improved on the basis of the experimental results. Silicone oil sheet is attached on the bottom of the sensor as a viscous fluid medium for improving unstable contact between the sensor and the objective surface. Using the improved sensor, experiments are carried out. The results show that the proposed sensor is capable of detecting the surface irregularity.