We propose a method by which to assist an operator in switching an internal model by visually presenting information about the dynamic characteristics of the destination in accordance with changes in dynamic characteristics when switching teleoperated hydraulic excavators. Experimental results show that when switching without presentation, the operation time significantly increased by 8:8%, compared with that observed when not switching. However, when using the proposed method, there was no increase in work time owing to switching. Therefore, the proposed method makes it possible to immediately switch the internal model of the operator.

Although the average life expectancy in Japan has been increasing in recent years, the problem of the large gap between healthy life expectancy and average life expectancy is still unresolved. Among the factors that lead to the need for nursing care, injuries due to falls account for a certain percentage of the total. In this paper, we developed boots that can provide external disturbance to the ankle with pneumatic gel muscles (PGM). We conducted an experiment using electromyographic potentials (EMG) and center of foot pressure (COP) as evaluation indices to evaluate the effectiveness of fall prevention training using this device, which is smaller and lighter than conventional devices.

In this paper, we propose a method for generating height-map images that can achieve surface texture with the haptic sensation desired by the user. A texture plate was generated based on the height-map image, and a haptic data set was generated through questionnaire evaluation when subjects freely touched the plate. Next, we constructed a haptic estimation model using features based on image decomposition methods and confirmed that haptic sensation can be estimated from image features. Then, we set up a method to calculate evaluation values for haptic and visual impressions and set up an objective function using these values. Finally, we confirmed that we can generate height-map images with the desired haptic sensation by optimizing the objective function to minimize it through verification with a test set.

Descending stairs is a difficult task for the elderly and disabled, and it is difficult to rehabilitate using real stairs because of the high degree of difficulty and the anxiety it causes. Therefore, rehabilitation using real stairs is difficult. Therefore, stair training on a flat surface using motor imagery is expected to be a new rehabilitation training method that eliminates the fear of falling. This paper proposes a new rehabilitation training method that simultaneously provides a visual presentation using virtual reality (VR) and a force sensation to the muscular senses using a pneumatic gel muscle (PGM). The system is designed to allow users to perceive the sensation of descending stairs. To verify the effectiveness of the system, we confirmed the muscle activity of the quadriceps muscle using electromyography (EMG) and evaluated the presence of the sensation of descending stairs using a psychological evaluation.

The construction industry in Japan has a shortage of workers and skilled operators because of the decrease in the working population due to the low birth rate and aging population. The number of operators of hydraulic excavators, which account for the largest proportion of construction machinery, is also decreasing and it has become difficult to employ operators with expert skills. Teleoperated hydraulic excavators can address this problem. It is difficult, however, to carry out safe and efficient construction work using a teleoperated excavator because of the lack of information available to the operator, such as the posture of the machinery and condition of the work object, in comparison with operating an actual excavator. Therefore, we propose a machine instability feedback system that calculates the attachment posture and digging reaction force and feeds the hydraulic excavator power margin directly back to the operator. This becomes an index of the margin until the excavator body, not the attachment, starts to move by the digging reaction force. We conducted a subject study using an excavator simulator to verify the effect of presenting the machine instability information to the operator. The results confirm that the operator could safely carry out digging work with acceptable work efficiency when machine instability information was provided.

Sense of agency (SoA) has long been a focus area in the field of human-machine interactions, because it is thought to relate to progress and improvements in developing interface systems that users can control with intention. However, there is currently no systematic SoA evaluation method within the context of heavy machinery operation. With our research, we intended to develop a SoA scale for heavy machinery operation (SoAS-HMO). The scale evaluated the psychological experiences of operators while they voluntarily operated heavy machinery. To build this scale, we conducted two surveys: a semi-structured interview survey to initially prepare the scale items, and a questionnaire survey to explore and confirm the scale structure. Subsequent factor analysis revealed one factor and twenty-one items that were related to the SoA experienced during operation, with sufficient reliability and validity, specifically with goodness of fit index (GFI) = 0.933, adjusted GFI (AGFI) = 0.911, comparative fit index (CFI) = 0.909., and Cronbach's a = 0.97. Therefore., this scale can successfully measure the subjective SoA of the operator and be applied to evaluate machinery efficiency. Our study provides progress towards improving heavy machinery systems that can be intentionally controlled by operators.

This paper ponders the effects of introducing an assistive force visual display on a subject's force perception. Correct and incorrect displays were tested and were found to significantly affect the subject's force perception as compared to no visual feedback case. The elbow joint was considered for this study and assisted with four wearable artificial muscles called pneumatic gel muscles (PGMs). It was observed that subjects tend to underestimate the assistive force. The introduction of a visual display could significantly reduce this force underestimation. Moreover, the level of force underestimation was higher in case of higher force values. In other words, the higher the actual force provided, the higher the level of force underestimation in all three conditions: correct, incorrect and no visual feedback. It was also observed that the perceived error (perceived force - actual force) was directly in proportion with the visual error (visual force - actual force).

We developed a system to evaluate the skill of operating a hydraulic excavator using a remotely controlled (RC) excavator and virtual reality (VR) technology. We re-modeled the RC excavator such that it can be operated in the same manner as an actual excavator, and proceeded to measure the excavator's state. To evaluate the skill of operating this system, we calculated several indices from data recorded during excavation work. We calculated the same indices from data recorded during excavation performed by an actual excavator, and verified that there exists a high correlation between the indices of the RC excavator and those of the actual excavator.

The increasing use of virtual reality (VR) and augmented reality (AR) systems has opened the possibility of providing immersive experiences to the general population around the world. However, most of the existing systems do not provide highly effective force-feedback experiences to the user. To provide such augmented systems in combination with force-feedback, novel ideas must be introduced that can be easily integrated with the existing VR and AR technologies. This work proposes a first-person VR game integrated with a soft exoskeleton that enhances the quality of interaction between the subject and the virtual environment (VE) through an additional force-feedback element. The effect of introducing the force-feedback element on the user is analyzed by using biosensors and questionnaire feedback. The biosensors are used to measure the level of anxiety induced in the subject during interaction with the virtual environment. Conditions in which this interaction occurs with and without force-feedback are compared. The questionnaire analyzes the perceived change in emotions of users as a result of the introduction of the force-feedback element. This game can be played by most individuals, regardless of age and physical fitness.

In this study, we evaluated the effects of different object parameters on fingertip deformation by examining the contact area deformation that occurs when a finger is pushed against a flat plate with a film. An experiment was conducted under various object parameters and forced displacements. First, the eccentricity, which is an index of the contact area deformation of a fingertip, was measured for different objects. As a result, it was confirmed that the eccentricity depends on the object characteristics. Next, the results of the tactile texture evaluation were modeled through a multiple regression analysis using the contact area deformation and object parameters. The relationship between the eccentricity and tactile texture evaluation was then investigated using the coefficients of determination in a multiple regression analysis. The results showed that the eccentricity can be used to evaluate a property differently from the material parameter with respect to the expression of feeling. A prediction of the tactile texture items was conducted using the eccentricity and object parameters. It was shown that it is possible to estimate the sensitivity with a correlation coefficient of 0.8 or more for smooth, granular, sticky, dry, and slippery textures.

Studies suggest that the level of muscle activation may influence force reproduction. The purpose of this study was to develop a sensorimotor-enhancing suit (SEnS)-unpowered assistive clothing without actuators or electrical devices-and to examine its efficacy for reducing voluntary muscle activation and improving force reproduction in the upper limb in healthy young adults. The SEnS was made of elastic fabric and designed to produce assistive shoulder flexion moment that can partially mitigate the required activation of the user's shoulder flexor muscles. A series of human experiments were then conducted in healthy young adults. As a proof of concept, reduction in force reproduction performance of the shoulder moment was confirmed with mitigation of the shoulder moment by using an external string. To examine the efficacy of the SEnS, voluntary muscle activation and force reproduction performance was examined with and without using the SEnS, by comparing the amount of activation in the shoulder flexor muscles and the error in force reproduction in the upper limb. The results showed that wearing SEnS reduced muscle activation and force reproduction error. These results suggest that the accuracy of force reproduction can be improved substantially by partially mitigating the shoulder moment and associated voluntary muscle activation, using the SEnS.

Human's skeletal muscles have sensory systems. Human-machine interfaces that consider muscle effort should improve the usability and manipulability of daily products. This paper presents an interface design method that considers muscle effort and illustrates possible applications. A three-dimensional musculoskeletal model is developed based on the upper extremity model with 29 muscles and 16 joints. Muscle activity is estimated by using the OpenSim. We define the muscle effort as the mean of each muscle activity, and the muscle effort is used to determine the physical design or the physical response of the human interface. We applied the proposed physical design modification method to the design of the key height of a keyboard. The key height is changed proportional to the normalized muscle effort. The estimated muscle effort suggests that users can type every keys of the modified keyboard with a similar muscle effort. We also explore to improve the manipulation feeling of the key touching of mobile devices by changing the feedback response according to the estimated muscle effort when users touch keys. The experimental results indicate that the manipulation feeling can be improved by changing the feedback response according to the muscle effort.

This paper proposes a probabilistic neural network (NN) developed on the basis of time-series discriminant component analysis (TSDCA) that can be used to classify high-dimensional time-series patterns. TSDCA involves the compression of high-dimensional time series into a lower dimensional space using a set of orthogonal transformations and the calculation of posterior probabilities based on a continuous-density hidden Markov model with a Gaussian mixture model expressed in the reduced-dimensional space. The analysis can be incorporated into an NN, which is named a time-series discriminant component network (TSDCN), so that parameters of dimensionality reduction and classification can be obtained simultaneously as network coefficients according to a backpropagation through time-based learning algorithm with the Lagrange multiplier method. The TSDCN is considered to enable high-accuracy classification of high-dimensional time-series patterns and to reduce the computation time taken for network training. The validity of the TSDCN is demonstrated for high-dimensional artificial data and electroencephalogram signals in the experiments conducted during the study.

The paper discusses the arterial stiffness during the flow-mediated dilation (FMD) test The FMD test is a method of evaluating the vascular endothelial function and has been popular as it is non-invasive and readily performed by a skillful ultrasound technician. The FMD test, however, evaluates only the maximal increase in vascular diameter mediated by the increases in blood flow after the release of the occlusive cuff and does not evaluate the arterial viscoelastic properties. This paper thus estimates the log-linearlized stiffness, to evaluate the arterial stiffness properties using the arterial diameter and blood pressure measured in a beat-to-beat manner during the FMD test. To six healthy volunteers, we performed the FMD test to measure the arterial diameter and blood pressure with ultrasound diagnostic imaging equipment and non-invasive continuous arterial blood pressure monitor, respectively. As a result, the maximal vasodilatation ratio of FMD (FMD) was obtained after cuff occlusion. In comparison with the arterial stiffness before the FMD test, the stiffness of the arterial wall is temporarily decrease and increase. It was concluded the the arterial stiffness can be estimated on a beat-to-beat basis during the FMD test.

The present study clarifies the human characteristics of motion and perception, and develops a force perception model in vehicle operation. As the first step, this paper analyzes experimentally human force perception properties during steering operation, in which multiple joint motions and large postural changes of arms are necessary. The experimental results reveal the following three points: a) force perception follows the Weber-Fechner law under the constant arm posture, b) force perception changes depending on the steering wheel angle and is affected by arm weight and force perception properties, and c) the same tendency of force perception properties is observed even if the steering direction changes. Finally, a force perception model in sending a steering wheel is proposed.

This paper reports on the results of an experiment involving a wearable sensorimotor enhancer intended to improve tactile sensitivity in human fingertips. As briefly exposing tactile receptors to subsensory vibration is known to enhance tactile sensitivity thanks to a phenomenon called stochastic resonance in the somatosensory system, applying white-noise vibration to a fingertip is expected to improve the sense of touch and associated motor skills. Against such a background, a prototype of a wearable device called a sensorimotor enhancer is proposed here. The device is attached to the radial side of the fingertip and stimulates tactile receptors by applying vibration from a lead zirconate titanate piezoelectric stack actuator. This design keeps the palmar region free, thereby helping to maintain the wearer's manipulative ability. Sensory and motor tests involving 11 human subjects were conducted to determine the efficacy of the device, and the results confirmed its usefulness.

This paper reports the initial experimental results of a wearable sensorimotor enhancer for a fingertip. A short-time exposure of tactile receptors to sub-sensory white-noise vibration is known to improve the tactile sensitivity. This phenomenon, called “noise-enhanced tactile sensation” or stochastic resonance (SR) in the somatosensory system, is expected to enhance the sense of touch when white-noise vibration is applied to a fingertip, and thereby improve associated motor skills. A prototype sensorimotor enhancer has been developed in this research. This wearable device is to stimulate tactile receptors by applying vibration from a compact lead zirconate titanate (PZT) piezoelectric stack actuator attached at the radial side of the fingertip. This design keeps the palmar region free and maintains the wearer's manipulability. Sensory and motor tests have been conducted for health subjects to confirm the efficacy of the device. Statistical significance has been observed in most of the tests.

In this paper, we propose a novel haptic device that gives weight and friction illusions. Sensing a slip condition at a fingertip plays an important role to estimate the weight and friction properties of an object. This fact suggests that controlling the contact condition could yield weight and friction illusions. Based on this idea, a prototype device was developed that controls the contact surface between a fingertip and a rigid plate based on a camera-based eccentricity control. The desired eccentricity profile was given from the mathematical relationship between the eccentricity and the surface deformation, which is obtained by known material parameters, vertical/lateral forces applied on the contact area, and static friction coefficient. The performance of the developed device was evaluated through human experiments. The experimental results show the statistically significant difference between presented weight and friction conditions.

In this paper, we focus on the hand posture when a human subject pinches an object, and the correlation between the results of sensory evaluation by the subject and the tendon force of the subject's fingers are investigated. Surface electromyogram (surface EMG) and pinching force are measured during the pinching motion of human subjects. Experimental results show that subjects feel comfortable pinching a 60 [mm] cylinder. The surface EMGs are lower in the vicinity of 60 [mm]. Furthermore, a tendon-driven robot hand is developed as a sensing hand prototype. The surface EMGs and the motor torque are compared in a pinching experiment using the tendon-driven robot hand. Experimental results show that the tendon-driven robot hand is effective for quantitatively evaluating pinching effort. Simulation of the pinching motion is conducted using a simple finger model. The simulation results show the importance of finger posture and the moment arm in estimating tendon force. Taken together, these results demonstrate the possibility of conducting quantitative evaluation using the tendon-driven robot hand.

In this paper, we propose a tendon skeletal finger model and discuss which finger postures the human feels easy to pinch based on the tendon forces and the human experimental results. The finger model mimics a human tendon skeletal structure. The tendon forces during the pinching motion were simulated using the finger model. Simulation results show that the tendon forces closely mirror the human muscle activity. Sensory evaluation of subjective pinching effort was conducted with five subjects. The subject pinched five kinds of cylinders, from 20 [mm] to 100 [mm]. The pinching force and the surface EMGs were simultaneously measured in the experiment. Based on the human questionnaire tests, we investigated which finger postures the human feels easy to pinch a cylinder. The results show that the pattern of the EMGs measured by the experiment is very similar to that of the tendon forces calculated by the finger model simulation. This indicates that the tendon force is a useful index of the subjective pinching effort and it can be used for the quantitative evaluation instead of EMGs.

Humanoid robots and robotic manipulators with good dexterity are promising to enhance the factory productivity in next generation. Dexterity of robotic manipulators can be achieved by equipping an anthropomorphic robot hand with multiple fingers. Additionally, in order to manipulate various tools that humans are using in daily life, the size and the exerting force of the robot hand should be similar to that humans are. In this paper, we proposed a human-sized multi-fingered robot hand with detachable mechanism at the wrist. The robot hand can be split at the wrist into the hand part and the actuator part. The fingers are driven by wires and are controlled by actuators embedded in the arm part. The driving force from the arm part is transmitted to the hand part by gear mechanism at the wrist. The developed robot hand has the size of 200[mm](length) × 78[mm](width) × 24.6[mm](thickness) and can exert 10[N] at the fingertip. The performance of the developed robot hand was shown by a motion control experiment.

This paper presents the haptic interface system that consists of a base object and a haptic device. The desired force response is achieved by the combination of the real force response of the base object and the virtual force exerted by the haptic device. The proposed haptic augmented reality (AR) system can easily generate the force response of a visco-elastic object with a cheap haptic device and a base object that has the similar visco-elastic property to the target object. In the demonstration, the force response of the target object was generated by using a haptic device only (VR) and using both a haptic device and a base object (AR), respectively. The evaluation experiments by participants show that the AR method has better performance than the VR method. This result indicates the potential of the proposed haptic AR interface.

This paper addresses pose estimation of a hand in motion through a model-based vision-based system. Previous research on human hand tracking and pose estimation usually suffers from limited number of degrees-of-freedom estimated, camera orientation (i.e., the hand is restricted to a particular pose while moving) and occlusion. We describe and evaluate a system that allows several DOF of the hand to be estimated without hindering its motion and while minimizing occlusion. To allow for a complete 3D motion, a voxel model and a skeletal model of the hand are used. The system uses multiple viewpoint cameras to obtain information of the hand motion. Due to the non-linear characteristics of the system, Unscented Kalman Filter (UKF) is used to track the hand motion. UKF estimates the hand pose by minimizing the difference between the skeletal model and the voxel model. Estimation results from different hand motions of up to 15DOF show the feasibility of the system.

The contact tonometer is commonly used for measuring the internal eye pressure to diagnose glaucoma. However, the conventional eye pressure measurement is valid only under the assumption that all subjects have the same structural eye stiffness. This paper challenges to measure the contact area between the probe and the cornea in addition to the corneal deformation for considering the individual differences in the structural eye stiffness. Prior to the experiment, a spherical model of an eye is developed and the analytical eye stiffness is introduced. The experiment is conducted based on the contact method where a contact probe is pressed on an anesthetized cornea. The deformation of the cornea and the contact area are captured by cameras during the experiment. The experimental results show that the measured stiffness nicely matches the analytical solution based on the constructed model. However, some subjects have different relationships between the displacement and the contact area even with similar estimated eye pressures. This suggests that the structural eye stiffness should be considered for more precise diagnoses of glaucoma.

The internal eye pressure is an important index for judging whether an eye suffers from glaucoma or not. The conventional eye pressure measurement is valid only under the condition that all subjects have the same structural eye stiffness. This paper challenges stiffness sensing of the human eye by pressing it with a contact probe. The displacement of the eye is captured by a camera with high resolution. Experimental results show that the stiffness is roughly constant for each subject, while it differs by up to a factor of five between subjects. The method can detect subjects whose eye stiffness is smaller than average for further careful medical examination

In this paper, the CPG-based control method for the rotating manipulation that can adaptively change the issuing cycle of motion-triggers is proposed. The CPG model consists of the mutual inhibition network and the rotating manipulation is performed by the motion-triggers that are issued from the CPG model. The smooth change of the desired velocity can be given by the output of neurons that keep smooth increase and decrease. Experimental results using 4-fingered hand system suggest that the issuing cycle of the motion-triggers can be adaptively changed by the feedback of joint angle according to an object size.

In this paper, a novel pointing device is proposed that utilizes the deformation of the fingertip. When a fingertip is pressed and slightly slid on a rigid plate, a partial slip, called an "incipient slip", occurs on the contact surface. While the deformation around the center of the contact area is small during the incipient slip, the boundary region moves to the sliding direction of the fingertip. The deformation changes depending on the sliding distance of the fingertip and the exerted force on the contact surface. The velocity of the pointer can be determined by the estimated distance and force based on the measurement of the deformation. In this study, the correlation between the sliding distance of the fingertip and the deformation and between the exerted force and the deformation are investigated. The degree of the deformation due to the sliding motion can be estimated based on the detected fingerprint center. The group delay spectrum tracking method is proposed for the detection of the fingerprint center. A prototype pointing device is developed to evaluate the operationality of the proposed device. Comparative experiments with conventional pointing devices are conducted. The validity of the proposed device is confirmed by the experiments.

A number of researches for the dexterous manipulation have been studied in robotics and physiologic fields independently. Multi-fingered manipulation for dexterous robot hands should be based on analysis of human skills. The purpose of this study is to apply CPG-based control to the dexterous manipulation. In this paper, CPG-based manipulation is proposed for the rotating manipulation of the dexterous hands. Finger manipulation patterns are investigated when a subject rotates a cylindrical object. Typical contact patterns during rotating manipulation are measured by FSRs. Based on the observation, the neural circuit model which generates a similar contact pattern is constructed. The experimental results by the computer simulation suggest that rotating manipulation can be performed using the constructed CPG model.

In this paper, a grip-force control of an elastic object is proposed based on a visual slip margin feedback. When an elastic object is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip" occurs on the contact surface. The slip margin between an elastic object and a rigid plate is estimated based on the analytic solution of Hertzian contact model. A 1 DOF gripper consists of a camera and a force sensor is developed. The slip margin can be estimated from the tangential force measured by a force sensor, the deformation of the elastic object and the radius on the contact area both measured by a camera. In the proposed method, the friction coefficient is not explicitly needed. The grip force is controlled by a direct feedback of the estimated slip margin, whose stability is analytically guaranteed. As a result, the slip margin is maintained to a desired value without occurring the gross slip against a disturbance load force to the object.

In this paper, a novel pointing device is proposed which utilizes the deformation of the fingerprint. When the fingertip is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip", occurs on the contact surface. Since the deformation around the center of the contact surface is small, the center of the fingerprint can be detected during the incipient slip. The group delay spectrum (GDS) tracking is utilized to detect the center of the fingerprint. The velocity of the pointer is controlled by the changes of the contact area on the basis of the tracked center. A prototype pointing device is developed to examine the operationality of the proposed method. The experimental results show that an operator can control the pointer with high accuracy.