Vibratory feedback, when applied during emotion-ally charged scenes in movies and music, has been shown to intensify viewer experiences. Previous methodologies primarily relied on pre-determined timings based on audiovisual content, rendering them less effective for interactive media. This study introduces a new approach that triggers vibratory feedback in real-time, guided by the viewer's physiological responses, aiming to amplify emotional experiences. We executed a user study involving ten participants who engaged in a treasure-hunting game. Vibratory feedback was administered to their upper bodies in instances deemed emotionally arousing, as indicated by their skin conductance responses (SCR) during gameplay. Compar-ative analysis of sessions with and without vibratory feedback revealed that vibrations significantly enhanced subjective feelings of anger and excitement. Moreover, vibrations led to a notable increase in the number of peaks and standard deviation of SCR signals, substantiating the efficacy of vibratory feedback in enriching emotional experiences. The findings affirm the potential of leveraging users' physiological signals to enhance emotional responses in interactive content. This advancement holds promise for a wide range of applications, particularly in personalizing emotional experiences in dynamic scenarios shaped by user interactions.

Body ownership, which is the feeling that one’s body part belongs to oneself, and agency, which is the sense of being able to control one’s own body part, can be felt towards fake body parts and those depicted by computer graphics. As part of an attempt to transfer self-body awareness to fake body segments, we investigated whether body ownership and agency are felt towards two visible left hands in an immersive virtual reality environment. One of the two hands shown through virtual reality goggles spatially matched the unseen actual hand. The other hand was fake and displayed at either the 10-cm lateral or medial side of the position of the actual hand. These two left hands moved synchronously with the actual left hand. Participants completed a behavioral test and questionnaire after adapting to the two left hands. In the behavioral test, participants accessed randomly emerging spheres using the seen hands as fast as possible. They used the fake hand to touch a sphere when it appeared near the fake hand 41% of the time when the fake hand was displayed at the medial position of the actual hand. The results of the questionnaire suggest that agency was experienced for the two visible left hands for this condition. By contrast, body ownership was felt mostly against the displayed hand that was spatially consistent with the actual hand. These findings indicate that although agency can be simultaneously felt for two seen left hands, body ownership is felt only for either of the two visible left hands.

Exposure to stimuli gives rise to sensory and affective experiences. Computing the relationships between these experiences is instrumental to designing affectively appealing products and understanding human experiences. Hierarchical structures of sensory and affective responses, which can be built through adjective rating tasks, are regarded as an effective means for expressing relationships between sensory and affective responses. Naturally, these hierarchical structures depend on the type of stimulus; however, so far, their dependencies on material categories have yet to be examined. We therefore investigated how hierarchical structures of affective and sensory responses depend on seven material categories: fabric, leather, wood, paper, foliage, stone, and glass. Each material category had 100 visual representations selected from the Flickr Material Database. Thirty-nine participants were asked to rate 368 pictures across a set of materials. The questionnaire adjectives included visually- and haptically-centered items although the stimuli were purely visual. We found that the structures differed substantially among the material categories, although there were some commonalities. Particularly, the positions of polysemic or multimodal adjectives such as “light” and “uncomfortable” in the hierarchy were highly dependent on the material category. For example, “light” has both physical (lightweight) and psychological (e.g., non-solemn, cheerful) meanings. For stone and glass (generally considered to be of heavy weight), the psychological meanings were primarily considered. Conversely, the physical meanings were predominant for fabric, leather, wood, paper, and foliage, for which weight is a factor in judging quality. The present study helps interpret affective descriptors whose meanings vary among types of stimuli.

There are few mathematical models that describe how human perceptual and affective responses evolve with time. We developed a state-space model of the interrelationships based on time-evolving perceptual and affective responses acquired by the temporal dominance (TD) method. We defined and computed the state variables that endow the system with memory using canonical variate analysis. Furthermore, we determined the model parameters on the basis of a cross-validation approach such that the observed and estimated changes in the affective responses are highly correlated. We applied this method to the TD curves of responses to eating strawberries and plum pickles, reported in our previous work. The model describes what happens during food intake in terms of a few state variables that summarize a large number of observable responses reductively, thus helping to make the perceptual and affective dynamics understandable.

Human perceptual and affective responses change dynamically when stimuli are experienced. In this study, we developed a method for modeling the causal structures of affective dynamics using time-series data. Using the temporal dominance of sensations method, perceptual and affective data were collected from individuals eating strawberries, and the resulting time-series data were mathematically represented using a vector auto-regression model. Multihierarchical and multidimensional causality structures that explain the temporal evolution of perceptual and affective responses were then established based on Granger causality and the information criterion. The established model suggests how affective and preferential responses are triggered following exposure to stimuli. We also assessed the quantitative and semantic validity of the model.

Temporal dominance (TD) methods can be used to record temporal changes in multiple sensory and affective responses. TD methods have found wide applications in the analysis of eating experiences of humans. However, extant analyses performed on TD data do not fully utilize the time-series properties of such data. The present study validates the prospect of principal motion analysis (PMA) of TD data. PMA is an extension of principal component analysis, and can be used to resolve multivariate motion data into base principal motions. In this study, panelists were asked to evaluate the tastes of ten types of pickled plums using the temporal dominance of sensations (TDS) and emotions (TDE) methods. Additionally, the panelists were asked to rate the plums using the semantic differential method. Results obtained using both methods were observed to demonstrate good agreement with each other in terms of the structures of reduced variable spaces. As realized in this article, implementation of the combined TD–PMA approach can potentially facilitate statistical discrimination of all food products, whereas conventional methods, such as principal component analysis of data provided via use of the semantic differential method, can at best discriminate only 67 percent of product pairs. PMA can, therefore, be considered as a suitable technique to reveal the characteristics of TD data.

There is increasing demand for tactile feedback functions for touch panels. We investigated whether virtual roughness texture quality can be improved through simultaneous use of vibrotactile and electrostatic-friction stimuli. This conjunctive use is expected to improve the perceptual quality of texture stimuli, because vibrotactile and electrostatic-friction stimuli have complementary characteristics. Our previous studies confirmed that these conjunct stimuli yield enhanced realism for simple grating roughness. In this study, we conducted experiments using simple and complex sinusoidal surface profiles consisting of one or two spatial wave components. Three different evaluation criteria were employed. The first criterion concerned the subjective realism, i.e., similarity with actual roughness textures, of virtual roughness textures. Participants compared the following three stimulus conditions: vibrotactile stimuli only, electrostatic-friction stimuli only, and their conjunct stimuli. The conjunct stimuli yielded the greatest realism. The second criterion concerned roughness texture identification under each of the three stimulus conditions for five different roughness textures. The highest identification accuracy rate was achieved under the conjunct stimulus condition; however, the performance difference was marginal. The third criterion concerned the discrimination threshold of the grating-scale spatial wavelength. There were no marked differences among the results for the three conditions. The findings of this study will improve virtual texture quality for touch-panel-type surface tactile displays.

As touchscreens have become a standard feature in mobile devices, technologies for presenting tactile texture feedback on the panel have been attracting attention. We tested a new method for presenting natural materials using an electrostatic tactile texture display. In this method, the frictional forces are decomposed into low- and high-frequency components. The low-frequency component was modeled based on Coulomb’s friction law, such that the friction force was reactive to the finger’s normal force. The high-frequency component was modeled using an auto-regressive model to retain its features of frequency spectra. Four natural material types, representing leather, cork, denim, and drawing paper, were presented to six assessors using this method. In a condition where only the low-frequency friction force components were rendered, the materials were correctly recognized at 70%. In contrast, when the high-frequency components were superposed, this rate increased to 80%, although the difference was not statistically significant. Our approach to combine a physical friction model and frequency spectrum for low- and high-frequency components, respectively, allows people to recognize virtual natural materials rendered on touch panels.

Developing a multilayered structure of adjective words that explains semantic relationships between human perceptual and affective responses to stimuli is instrumental in the design of affective aspects of products. However, the determination of multilayered structure is demanding and, thus far, it has been conducted by experienced developers in a trial-and-error manner. This study developed a method to systematically establish such structures through common tasks for sensory evaluation where products are rated along adjective labels. This method gradually expands the model from a simple two-layered to complex multilayered structures until it is accepted by structural equation modeling. The lower and higher layers of the initial two-layered model are composed of sensory and affective adjectives, respectively. The parts with weak fit indices of the higher layer are then remodeled, resulting in a multilayered affective structure. To validate the method, we built adjective structures based on responses to touching plastic plates. The method resulted in three- and four-layered structures that were quantitatively and semantically valid.

In this study, improved realism of virtual roughness textures is verified by using vibrotactile and electrostatic friction stimuli in conjunction. Because these two types of stimuli have complementary features, they are expected to enhance the perceived quality of stimuli. In our previous research, conjunct stimuli improved the realism of virtual surfaces with sinusoidal surface roughness profiles. In this research, we tested with complicated surface roughness, and experimental participants ranked vibrotactile, electrostatic friction, and conjunct conditions in terms of realism compared to actual roughness textures. The conjunct condition was judged to be more realistic for simple and complicated surface profiles consisting of one or two spatial wave components.

In recent years, the number of patients suffering from diabetes mellitus has continued to increase. When diabetes becomes severe, ulcers may form on the feet of the patient. In the past few years, several researchers have focused on the risk factors and avoidance of ulceration. One effective method to predict the occurrence of diabetic foot ulcers is developing a machine learning model. However, few studies combine both clinical indices and mechanical data as the attributes of the training datasets. In this study, we developed a composite model of a convolutional neural network and K-means clustering to extract features from diabetic patients with or without ulceration as well as healthy individuals. Using a deep clustering model, the center of pressure (CoP) trajectory images were divided into three clusters. Furthermore, we evaluated the performance of the clustering by extracting the features from the CoP trajectory images in each cluster and combining them with the clinical indices of the patients. The results showed that patients with ulcers when walking tend to contact the ground with a narrow area of the plantar and apply a small force. Furthermore, it was found that patients undergoing diabetic neuropathy or with a toe amputation have a high potential of suffering from ulcers.

Many manufacturing industries have a high demand for the construction of collaborative operation systems using industrial robots. Although there is a preexisting set of safety verification data in ISO/TS 15066 for collaborative operations, there is no established testing method for safety validation. To establish a testing method, it is effective to use a dummy that has mechanical properties similar to those of a human. However, there is no parametric study that exists for designing a dummy that represents the static and dynamic mechanical properties and the nonlinearity of the static mechanical properties. In this study, static and dynamic experiments were conducted to obtain the mechanical stiffness of human subjects and the contact force transitions during the dynamic contact between a robot system and a human. Subsequently, the same experiment was conducted using the proposed dummy. The biofidelity of the dummy was examined by comparing the parameters of a viscoelastic model. This study contributes to increasing the safety of collaborative operations by developing a dummy that can be used for risk assessments in collaborative operations.

Although a wearable assist robot helps to enhance the gait ability of the wearer, it can cause gait instability in an emergency. A collision with an environmental object is one source of such instability. Insufficient adaptation to change by the physical frame, and restriction of joint motion, increases the risk of collision and fall of the wearer owing to the robot. In this article, the reaction motion owing to side contact, which applies a spin moment to the body, is investigated for the wearers of a physical assistant robot. In particular, reaction motions in the horizontal plane, such as body rotation, foot direction, and center-of-mass position, are investigated. Factor analysis and cluster analysis are performed, and two reaction patterns-the rotation group and the straight group-are determined. The motion of these groups suggests an approach for navigation through an obstacle while using an assist robot. Furthermore, the gait phase at collision time is considered as a determinant of the reaction pattern and the reaction pattern is found to affect the fall mode. The results of the article suggest that the joints of the physical assistant robot should be equipped with sufficient degrees of freedom, at least in hip rotation, such that it is capable of rotating the body and feet in the horizontal plane to reduce fall risk.

Vibrotactile and friction texture displays are good options for artificially presenting the roughness and frictional properties of textures, respectively. These two types of displays are compatible with touch panels and exhibit complementary characteristics. We combine vibrotactile and electrostatic friction texture displays to improve the quality of virtual textures, considering that actual textured surfaces are composed of both properties. We investigate their composition ratios when displaying roughness textures. Grating roughness scales with one of the six surface wavelengths are generated under 11 display conditions, and in 9 of which, vibrotactile and friction stimuli are combined with different composition ratios. A forced-choice experiment regarding subjective realism indicates that a vibrotactile stimulus with a slight variable-friction stimulus is effective for presenting quality textures for surface wavelengths greater than or equal to 1.0mm.

Estimating the physical burdens of people with motor dysfunction is helpful for developing care equipment and facilities. We propose a patient simulation where a trained healthy subject simulates impaired movement with instruments to guide his/her body motion with less possible incurred pain. Electric cutaneous stimuli are used to intuitively indicate possible pain of an injured body part. We used this simulator to investigate a suitable handrail position for knee pain holders during sit-down and stand-up motions. Electric stimuli were presented in proportion to the moment at the supposedly impaired knee. The results suggest that a handrail on either the healthy or impaired side significantly reduces the experienced physical burden (i.e., internal knee flexion-extension moment). Specifically, the maximum knee moment was smaller when the handrail was on the impaired side than it was on the healthy side.

Humans recognize textures through the interaction induced by the normal and shear forces generated between the fingertip and the surface of objects when sliding their fingers on material surfaces. We investigated the stochastic relationships of these two axial forces during tactile exploration. The characteristic values of the elliptical distribution obtained by the variations of the two axial forces were analyzed. It was found that the sizes, shapes, and gradients of the elliptical distribution of the low and high frequency forces statistically varied among the 18 types of textures. These parameters may be the indicators to distinguish the unevenness of the surface profile and the behavior of friction between the surface of material and the fingertip. The fact that the stochastic relationships between the two axial forces depend on the surface of material help us understand human texture perception.

Forward fall is one of the most common causes of upper extremity fractures. Significant factors influencing impact force and injuries were widely studied; however, it is necessary to investigate the natural reactions of humans during a forward fall to obtain a realistic evaluation of injuries. The purpose of this study was to analyze the natural motion of the upper extremity during an induced trip. We carried out a tripping experiment using an obstacle colliding with one leg; while recovery step was prevented to produce a forward fall. Results showed that the elbow extension had a slight ascending trend during the forward fall and elbow angle at the moment of hand-ground contact was appropriate to reduce the peak force. Landing on the obstacle-side hand was more likely due to body rotation towards the obstacle-side. To prevent injuries, subjects were connected to a safety harness not to strike the ground with high impact velocity. Thus, the fall motion was simulated using a 12 DOF model to obtain a realistic evaluation of the impact velocity and the related impact force caused by the forward fall was estimated using a sagittal 3-segment model. Results of this study can be useful in human-robot collaboration, where a collision between human and robot may cause a forward fall.

Many wearable robots are not fully capable of fitting the motion of the wearer, owing to their limited degrees-of-freedom (DOF). This limitation disturbs motions such as corner curving, which are required in daily life. In this study, the effect of the DOF restriction on the corner curving motion was observed and analyzed. The gait motions, when curving a round corner with and without the restriction of out-of-sagittal plane motion, were compared using a wearable device which could restrict hip adduction/abduction and rotation. The result suggested that the restriction not only decreased the range of motion of hip adduction/abduction and rotation, but also changed the center of mass (COM) trajectory. Owing to the disturbance applied during the stepping motion of the outer leg, the COM trajectory moved further from the corner in the trials with restriction. Thus, the distance between COM and the base of support of the inner foot became larger during the swing phase of the outer leg, which possibly increased the risk of loss of balance. Furthermore, it was also suggested that an assistive torque, which does not consider curving motion, possibly increases such risk.

This study demonstrates that the simultaneous use of two types of representative principles for tactile texture displays, vibrotactile and electrostatic friction stimulation surpasses either type of stimulation. We used a set of sinusoidal grating scales with different surface spatial periods as simulated textured surfaces for testing the quality of texture displays. Experimental participants judged the best stimulus condition among the vibrotactile, electrostatic friction, and the conjunctive stimuli in terms of the reality perceived from the simulated texture stimuli. The use of vibrotactile and electrostatic stimuli in conjunction was judged to be the most realistic for the spatial periods of 1 and 2 mm, whereas noticeable differences among the three stimuli conditions were not reported for small spatial periods of 0.2 and 0.5 mm. The simultaneous use is effective for simulating the surfaces with middle-level asperity.

Equinovarus is a foot deformity characterized by the patient's foot being at rest in an abnormally supinated state. In a clinical setting, physical therapists manually stretch the foot to a pronated state to allow the patient to retain some degree of mobility. Currently, no automated ankle stretching machine is commercially available. To tackle this issue, we have developed an automated stretching machine that controls the eversion and flexion angles of a patient's deformed foot using two pneumatic actuators. We designed a proportional and integral (PI) controller to place the foot in the desired dorsiflexed position and performed a user test involving a healthy participant. Even when the initial foot position of the participant was in an equinovarus position, the foot was successfully everted and dorsiflexed to match the desired reference posture via the stretching machine. The average differences between the reference and measured foot angles at the final state were found to be within 2° for both the dorsiflexion and eversion angles. The machine replicated the reference angles with acceptable errors.

The elderly sometimes fall owing to tripping caused by decreased dorsal flexion during the swing phase. A wearable assist device that was able to increase dorsal flexion to reduce the risk of tripping was proposed in this study. This device assists in dorsal flexion during the swing phase by transferring the tension force from the upper body to the foot by connecting them using strings. To improve the efficiency of the assist, the string path was optimized to maximize the assist torque at the time of minimum foot clearance (MFC). In this study, two string-path patterns were tested. Based on the walking experiment results, we confirmed that the wearable assist device that we developed successfully increased both the dorsal flexion and minimum foot clearance during the swing phase.

Three-dimensional stretching is needed to treat equinovarus, which deforms the patient's foot to plantarflexion, adduction, and inversion postures. We have prototyped a three-dimensional stretching machine that the patient can use for the treatment of equinovarus by him- or herself. By adopting a cable-driven mechanism with two independently controllable pneumatic actuators, the stretching machine can apply a force to the foot along the dorsiflexion direction as well as the direction combining abduction and eversion. In this study, we verified the effectiveness of a prototype stretching machine for healthy subjects. For evaluation, the muscle stiffness and maximum voluntary contraction (MVC) of plantarflexion were compared immediately before and after stretching and 10 min later. As a result, the MVC decreased after stretching, which is a clinical index for effective stretching.

The estimation of the physical burdens from which people with motor impairment suffer helps us establish welfare techniques comprising personal care equipment and assessment of critical risks, such as fall risks. However, the involvement of actual patients in the evaluation and development of this equipment is costly and involves the exposure of patients to long and exhausting experiments. To solve this problem, we developed a robot wearable by a healthy person and the associated control algorithm to simulate typical motions of patients with knee osteoarthritis, which is a common symptom for the elderly and causes pain during movement. To estimate the physical burdens inflicted by knee malfunctions, we computed the knee flexion and extension moment of the simulated patient during the standing-up and sitting-down motions. The moments, estimated under certain conditions, are qualitatively consistent with those considered clinically, which corroborates the validity of our patient simulation techniques.

We developed a method to present an impulsive resistance force by using the passive damping brake of a DC motor. The rapid pulsive change in the resistance forces preceding the main brake increased the perceived resistance of the brake. In a ranking task involving five naive participants, a damping brake with two preceding pulsive resistances was reported to deliver a resistance larger than that due to a simple stepwise brake that achieves the maximum physical braking force. This finding indicates that perceptually effective impulsive resistances are different from the physically optimal resistance. Our technique enhances the abilities of passive force displays, which are inherently safe human-machine interfaces.

It is ethically problematic to engage people with impaired motor functions in experiments, wherein they may face severe pain and unacceptable risk of injuries. We developed a wearable robot for healthy adults and its control algorithm to simulate the behaviors of patients with knee osteoarthritis. We focused on invariant motions, which are typically presented for avoiding knee pain. In order to simulate patient movements by a healthy person wearing a robot, we formulated a model motion that represented invariant patient motions. We then determined the output torque of the exoskeletal knee robot such that the wearer's motion followed the model motion. The effectiveness of the method was testified for standing-up and sitting-down motions, and some characteristic impaired motions such as body inclination to the healthy side and imbalanced right and left knee angles were manifested by the simulator.

Physical assistant robots are expected to be able to assist in our working and daily lives. However, safety problems arise when robots are used in indoor environments with obstacles. The focus of this study was on collision of the side of a robot with an obstacle, and the recovery motion after the collision was measured. A healthy young man participated in this experiment. Two different types of recovery motions were observed. These two motions differed in the degree of deceleration of the body rotation in the horizontal plane. The part of the robot and the timing of the collision with the obstacle affected the recovery motion strategy. A collision at the hip side in the middle swing phase might be the most severe condition from which to recover, among those considered in this study. The results of this study will be helpful in improving the safety of the use of physical assistant robots.

Humans can perceive the hardness of an object from the damped natural vibration produced by tapping its surface. This damped natural vibration can be used in hap tic rendering to effectively present a sense of the hardness of an object. Although its frequency is known to influence the subjective hardness, the effects of the viscosity or decay rate have yet to be studied. We researched the contributions of the mass, viscosity, and stiffness to the hardness perception using a commercial force display and psychophysical experiments, where only the vibratory displacement was presented. As a result, the stiffness was found to have a positive effect on the subjective hardness, similar to the usual hap tic rendering of an object. The mass of the object was negatively correlated with the subjective hardness. These results indicated that the vibratory frequency was the dominant parameter in determining the perceived hardness, which agreed with the previous reports on the perceptual effect of damped natural vibration. In contrast, the perceptual effect of the viscosity depended on the individual. For approximately half the participants, the viscosity did not directly influence the subjective hardness of an object. Some participants felt that a damped natural vibration with greater viscosity was harder. However, the other participants felt that it was harder with a smaller viscosity, which suggested that the decay rate of the mechanical system was used as a criterion to judge the hardness of an object. Therefore, the results indicated that the vibratory frequency is a general parameter that can be used to specify the subjective hardness, whereas the perceptual influence of the viscosity or decay rate depends on the individual.

Falls are one of the main causes of decreased quality of life and activities of daily living among the elderly individuals. Because tripping is the most common cause of falls, some types of recovery motions have been reported after tripping induced by a flat plate or block. Common recovery strategies observed under these conditions included elevating and lowering strategies. Especially, in case that the swinging leg contacts an obstacle at its early phase, the elevating strategy has been considered as a typical recovery motion. In this study, we performed a tripping experiment using a crossbar in the early and middle of swing phase. As a result, a new recovery strategy, which differed from those previously reported, was observed. In the new strategy, referred to as the skip strategy, the subjects lifted and moved the standing leg forward after tripping. The subjects utilized this strategy to recover from an early phase trip. The participants had to exert enough knee flexion and ankle plantar flexion torques of recovery leg with the skip strategy and enough knee extension, hip extension, and ankle plantar flexion torques of recovery leg with the lowering strategy. Especially for the elderly who cannot perform large joint torques, gait assistant devices such as physical assistant robots are required to compensate such joint torques in order to perform these recovery motions.

Measuring the shear deformation of a finger pad during active hap tic exploration of materials is important for analysis of textural sensations and development of tactile texture displays. Thus far, there has been no general methods to measure such deformations for active touch. To create a sensor system, we have been developing a new method for estimating the deformation of a finger pad based on the skin deformation propagated to the radial side of a finger tip. In this study, in order to validate the method, we compared the deformation or its acceleration of finger pad estimated on the basis of the acceleration measured at the radial side and those estimated by the shear force applied to the finger pad while exploring a few types of materials. The estimation errors for the deformations at 40 -- 450 Hz were smaller than human discrimination thresholds, indicating that the accuracy of our method is satisfactory compared with human perceptual sensitivity.

We have developed a texture display system that modifies the perceived roughness of textured surfaces via a voice coil actuator worn on the finger. To increase the roughness sensations, the vibrotactile stimuli from the actuator simulate the skin deformations that are activated when a wavy surface is scanned. Conversely, to decrease the roughness sensations of the textured surface, a high-frequency vibrotactile stimulus offsets the activity levels of tactile mechanoreceptors. This offset suppresses the perceived roughness of the surfaces being touched, with increase in the offset correlating with increase in the feeling of smoothness of the surfaces. We conducted an experiment in which we tested the effects of these two types of vibrotactile stimulation on grating roughness specimens, with subjective responses acquired from eight participants via the magnitude estimation method. The results obtained indicate that our method selectively increases and decreases the roughness felt. The intention is for the technique to be used to develop an augmented reality device for textures.

Human grip forces are automatically adjusted upon occurrence of an external disturbance experienced by an object that is held by a thumb and index finger. We investigated some of the cues that may be used by the brain to perform rapid grip restabilization. To this end we ask subjects to grip and hold an instrumented and actuated parallelepiped-shaped handle between the index finger and the thumb. Under computer control, the handle could be jerked from the still grip and could independently provided vibration of 250 or 100 Hz to the gripping fingers. We found that the latency of the motor corrective action was 139 ms on average, but when a vibrotactile stimulation was applied 50 ms before the application of the pulling force, the latency was reduced on average to 117 ms. The average latency of the conscious response to the vibrotactile stimuli was 230 ms, suggesting that vibrotactile stimulation was capable of influencing the reflex action.

Augmented reality technology realizes variable sensory feedback while maintaining a high perceptual quality by superposing real and virtual stimuli. In this research, we experimentally verified whether the masking effects of high-frequency vibrotactile stimulation on fingers is applicable to augmented reality of the tactile texture of materials. A vibrotactile stimulator was worn on a finger (ring-vibrating condition) or mounted on materials (material-vibrating condition). We investigated the effect of vibratory stimuli on perception of sand-paper roughness under these two conditions. Under both conditions, participants' performances to discriminate the roughness diminished because of the masking effects of the high-frequency vibratory stimuli (experiment 1). Subsequently, we specified how the masking effects influenced the perceived roughness using Scheffe's paired comparison. For the ring-vibrating condition, as accelerations of the vibrotactile stimuli increased, all participants reported a decrease in the perceived roughness of the sand-paper. For the material-vibrating condition, the roughness percepts increased or decreased depending on the individual (experiment 2). Although both implementation methods affect roughness perception, the ring-vibrating condition is recommended for augmented reality of tactile textures because it can steadily control the decrease in perceived roughness of touched materials.

In this study, psychological experiments are conducted to investigate harm-avoidance action characteristics in humans in close contact with robotic devices. For the experiments, a situation is created in which the sharp end-effector tip of a robot suddenly approaches the eyes of a facing participant. The avoidance reaction time is defined as the time interval from the beginning of the end-effector motion to when the participant begins movement to avoid harm. The results suggest that the avoidance reaction time does not depend on the type of work being performed but on the initial distance between the human's eyes and the approaching object. Using this information, safe human-robot working distances can be determined.

Wearable robots have finally reached a point at which they can be put to more practical use. However, methods for assessing their safety have yet to be properly established. Furthermore, for ethical reasons, such assessments must be conducted without exposing the human user to unnecessary danger. Therefore, in this study, a lower-leg dummy for the safety assessment of a wearable robot was developed to evaluate the effects of a misaligned knee joint.

In this study, we developed vibrotactile display methods that can assist designers in product design. In order to achieve realistic sensations required for such designing purposes, we used real materials such as cloth, paper, wood, and leather and applied vibrotactile stimuli to modify the roughness sensations of these materials. This approach allowed us to present textures of various virtual materials with a strong sense of reality. We verified that our proposed methods could selectively modify the fine and macro-roughness sensations of real materials. The methods are expected to aid product designers in deciding tactile sensations suitable for their products.

In this study, a psychological experiment is conducted to investigate harm-avoidance action characteristics in humans in close contact with robotic devices. For the experiment, a situation is created in which the sharp end-effector tip of a robot suddenly approaches the eyes of a facing participant. We define three parameters that represent harm-avoidance action characteristics: the avoidance reaction time, maximum avoidance acceleration, and maximum avoidance speed. The results suggest that the avoidance reaction time depends on the initial distance between the human's eyes and the approaching object. The results show that there are individual differences in all parameters studied. The results also show that the avoidance reaction time is negatively correlated with the other two parameters, which have a strong positive correlation with each other. We conclude that the avoidance reaction time and maximum avoidance speed are considerable parameters on avoidance actions.

A vibrotactile texture display produces virtual textures by applying vibratory stimuli to finger pads. In this study, we developed a technique to alter such textures based on certain specified materials. For example, the technique allows us to alter vibrotactile textures using terms such as “wood-,” “cotton-,” or “paper-like” which are familiar to end users of displays. The altered textures feel more similar to these specified materials. We realized this technique by constructing a material space, in which the materials are located based on the features of their vibrotactile spectra. The vibrotactile textures were then modified in this space. Our experimental results show that the technique can be used to alter a virtual wood texture to feel like cloth.

For researchers of haptic interfaces, evaluation of the perceptual similarity between virtual and real haptic stimuli has long been a serious problem. One of the most commonly employed evaluation methods is an identification task where assessors identify the type of randomly presented stimuli among multiple candidates. The results of this method are summarized as confusion matrices. We developed a method that allocates all virtual and real stimuli in a perceptual space. The spatial distribution of the stimuli allows us to visually understand the perceptual relationships between the stimuli. A brief validation confirmed that the proposed method is effective in evaluating the perceptual similarity between virtual and real stimuli.

In the design of human-robot coexistence environments, human avoidance actions are not taken into consideration. We conduct a psychological experiment to observe human avoidance actions where a mechanical harm approached his or her eyes. Using the observed motions, we carry out “extrapolation simulation” of human avoidance actions in collision situations. As simulation results, first, we express collision probability using the motion range of the end effector of a robot as a variable. Second, we identify the motion conditions, which ensure the safety, of the end effector in our setup. Finally, we quantitatively compare the safety conditions between taking and not taking avoidance actions into consideration. This study contributes to providing a tool for conducting a practical risk assessment aimed at realizing a safe human-robot coexistence system.

For reasons that have not yet been investigated, some textures appeal to human touch. We investigate the relationships between the physical and sensory factors, the predictability of textures, and their appeal to human touch. We conduct sensory evaluation of 24 artificial clay textures, and the results are analyzed using factor analysis. Experimental results identify five sensory factors. We reveal that 70-80% of the textures' appeal to human touch can be described on the basis of the sensory and physical factors. We propose two models involving the predictability of textures. These models, as well as models based on the sensory and physical factors, explain the affinity to textures.

The presentation of textures is a major application of vibrotactile displays. Lossy data compression is an effective approach for reducing a data size of such textures when delivering them through the Internet. This study developed an objective index that well described material-like textures' quality changes caused by data compression processes. We introduced a linear combination of the power and amplitude spectra that were weighted by the inverse of a detection threshold curve of vibrotactile stimuli. This combination described the quality deterioration of compressed textures to an accuracy of approximately 80%. These objective indices enhance the development of the lossy data compression algorithms without psychological evaluation tests of subjective quality changes in the textures.

Remote transmission of high quality sense of touch requires the representation of multiple tactile properties and compensation of communication delay. We developed a real-time remote transmission system that can deliver multiple tactile properties using a master-slave robot system. First, we assessed what type of tactile properties should be transmitted and how to connect them in real time. Three tactile properties - roughness, friction, and softness - were transmitted on the basis of the real-time estimated physical properties of three main wavelengths, a kinetic friction coefficient, and spring constants, respectively. Tactile stimulations were generated in synchronization with hand exploration at the master side by using local tactile generation models to compensate for communication time delay. The transmission of multiple tactile properties was achieved by the integration and enhancement of our previously reported methods for vibrotactile displays and tactile sensors. A discrimination experiment using different materials showed the feasibility of the total system involving the three tactile properties.

In this study, a tactile-roughness transmission system applicable to master-slave systems with a communication time delay is developed. The master-side system constructs a local model of target objects placed in the slave-side environment. Tactile feedbacks presented to an operator at the master side are produced by combining the physical properties of target objects in the local model and the kinetic information of the operator. The time delay between the operator's motion and the tactile feedback is cancelled because the stimuli are synchronized with the exploratory motions. The proposed system is applied to the transmission of tactile-roughness. The tactile stimuli presented to the operator are vibratory stimuli whose amplitude and frequency are controlled. These stimuli are locally synthesized by combining the surface wavelength of target objects and the operator's hand velocity. Using the developed tactile-roughness transmission system, an experiment for transmitting the perceived roughness of grating scales was conducted. As a result, the roughness perceived by the operators was found to highly correlate with the roughness of the scales in the slave-side environment with a coefficient of 0.83.

The tactile display for handheld devices requires compact hardware and useful applications. To satisfy these points, we have proposed the concept of `virtual active touch' that implements virtual exploration with a cursor on a screen through a pointing-stick-type tactile interface. The objective of this study is to present three-dimensional shapes on a two-dimensional screen without force feedback devices. It is possible to present three-dimensional shapes such as a bump using a lateral force. In order to represent human perception of geometric surface shape, instead of the lateral force, we use the cutaneous sensation of friction that occurs when a human finger strokes object surfaces. First, we confirmed that the virtual active touch interface could present cutaneous sense of friction. Second, we evaluated the perception of surface height in the context of bumped shape induced by the friction display. The experimental results agreed with our expectation that faster and longer increases in friction sensation were perceived as higher bumped shapes.

Haptic interaction with handheld devices is limited by space and size constraints that inhibit free hand exploration. We developed a compact haptic interface called Vib-Touch, which is operated by fingertip via a pointing-stick input device containing a tactile feedback. A cursor on the screen could perform virtual exploration as a substitute for the finger movement. We call this technology virtual active touch. We also propose a tactile stimulation method to represent not only tactile sensations, but the whole touch experience, including kinesthetic senses and a sense of shapes perceived by a fingertip. This study reports on the first prototype of the vib-touch interface for handheld devices. We confirmed that the prototype could provide friction sensation and geometric shape information using the proposed friction display method.

Tactile display is a technology that gives an artificial sense of touch to operators of information terminals or master-slave systems. The generation of tactile stimuli in response to the hand movements of the operators is associated with active touch and is considered to be one of the effective display methods, however which inevitably causes delayed tactile feedbacks from tactile displays. The knowledge of the detection threshold of system latency between the hand movements and the stimuli is helpful in designing the tactile displays. In this study, the identification of the thresholds through psychophysical experiments of 13 participants revealed two types of thresholds. One was the time-delay at which the participants observed the existence of latency. The other was the minimal time-delay that could affect the subjective feelings of the operators while they were not conscious of the latency. The means of the thresholds were 41 ms and 59 ms, respectively. The participants reported that the time-delay stimuli caused various changes in their subjective feelings. The empirical results suggest that the two types of thresholds depend on different sensory processes. This paper also proposes a design policy for tactile display systems in terms of system latency.

In the search and rescue situation at the disaster area, the searching task should be conducted by robots in order to avoid the second disaster. We study crawler robots that can traverse rough terrains and are used in such search operations. This paper describes the designing of an online simulation environment using which we can develop control algorithms for crawler robots, especially for the robot named “Kenaf.” We use USARSim that can simulate three-dimensional space. By employing realtime simulation, we connect a real robot controller and the simulator directly and realize online simulation using the real robot controller. The tests show that the created simulation system can be used for the development of control algorithms.

Tactile telepresence has been expected to be technology which encourages operators of robotic systems to remotely maneuver objects or recognize materials being touched through robotic arms. In remote environments, temporal disparity in tactile sensations is caused by temporal latency between a touch motion of the operator and a tactile stimulus applied on him. A framework of tactile telepresence systems, which is suitable in remote environments, has been proposed in the present paper. In the framework, a tactile display locally generates the tactile stimulus in synchronization with the touch motion of the operator in order to cancel communication delay. The tactile stimulus is generated by combining the motion and tactile factors, which are sensed by a tactile sensor. The factors are characteristic parameters of objects and effect on tactile sensations. In the present paper, physical parameters of the objects are selected as the factors because the tactile stimuli can be computed using physical models involving the motions and the parameters. Based on the proposed framework, a roughness feeling telepresence system was implemented and successfully transferred a roughness factor to the operator.

To realize a telepresence system with tactile feedback and force feedback, real-time estimation of various tactile sensation must be conducted. Because several types of tactile sensation consist in touch and human tactile feeling has high time resolution. A man feels active touch extraordinary with time delayed transmission of tactile information. Our proposing human finger mimetic sensor covers three tactile factors, which are roughness, softness and friction of objects to touch. Current tactile telepresence systems represent just one tactile sensation in addition to kinesthetic information. For augmented reality, wider tactile factors must be sensed at a tactile sensor system and transferred to a tactile display system with small time delay. We realized quick estimation of vibrational frequencies of the sensor and softness of objects to regenerate touch feelings to human skin by tactile displays, which usually need time-consuming samples and make it hard to address tactile telepresence system. Quick estimation of vibrational frequency was conducted by emulating impulse emission of Meissner's corpuscles. Quick estimation of Young's modulus of objects was solved by computing strain distribution in a sensor