In blind soccer, players rely solely on auditory cues for navigation and gameplay execution. This study investigates the sound source localization abilities of blind soccer athletes under conditions of background noise. Utilizing the Visual Space Acoustic System developed in our laboratory, we assessed participants’ capacity to track moving sound sources while simultaneously recording head movement data. The results indicate that visually impaired participants with blind soccer experience exhibited significantly enhanced sound source tracking capabilities compared to sighted participants. Moreover, they demonstrated greater adaptability to noisy environments.Post-experimental interviews revealed that visually impaired participants employed a distinct adaptation strategy in noisy conditions, initially localizing both the target sound and background noise before selectively focusing on the target source.Future research will expand the participant pool to include visually impaired individuals without blind soccer experience, enabling an assessment of auditory localization independent of sport-specific training. These findings contribute to the development of optimized training programs for blind soccer athletes and may inform rehabilitation strategies for individuals with visual impairments.

This study developed a high-precision paint deposition model that considers the position and direction of a spray-painting gun. Our angle-specific paint deposition model focused on the change in paint deposition due to the change in the painting angle; however, there was a problem with its versatility. We analyzed this problem, and the solution was achieved by separately modeling changes in the film thickness distribution using impact angle and spray distance, which were previously modeled together. For higher accuracy, a special function was proposed to convert the three-dimensional vector into two-dimensional coordinate values in the distribution function upper plane. To confirm the validity of our model, a painting test on an L-shaped surface was conducted, and the measured and predicted values were compared. The L-shaped surface is a typical shape in which the film thickness distribution changes with the angle; a complex path with varying distances and angles was employed. The results confirmed that the predicted values agreed well with the measured values in the L-shaped surface painting test, validating the developed model.

In the domain of blind soccer, proficient sound localization, utilizing auditory cues, is essential for player performance. This study explores the capacity of individuals to accurately localize static and dynamic auditory cues in an environment that allows free neck movement and investigates the strategies of head movement that contribute to optimal sound localization. Through enhancements to our virtual reality system, we conducted comprehensive verification tests to assess these abilities. Our analysis reveals that sighted players with experience in blind soccer tend to augment their sound localization capability for static sources through more pronounced and frequent head movements. Conversely, visually impaired players exhibit superior localization accuracy for both types of sound sources with minimal, yet efficient, head movements. A critical observation from this study is the differential performance in localizing dynamic sound sources; sighted players, despite active head movements, were unable to match the localization precision of their visually impaired peers. This disparity underscores the necessity for sighted players to refine their head movement techniques for improved sound localization. Importantly, this research also introduces a comparative analysis of simultaneous static and dynamic sound source localization, highlighting the complexities and adaptive strategies required for effective auditory perception in blind soccer.

In the present study, we used computational fluid dynamics to analyze the sag caused by spray painting, considering the change in paint shape due to flow. We focused on the paint adhering to the target surface because this behavior has not been previously examined. The particle method was adopted for the calculation because it enabled a stable analysis of the paint droplets and the complex uneven surface of the paint film. A high-speed camera and image analysis were used to capture the spray painting and identify the parameter values. Using the developed model, we analyzed the change in the film thickness distribution for painting on a flat plate in the vertical direction. It was confirmed that the numerical and experimental data correlated for two conditions of the target distance. In addition, we proposed a new index, Degree of Sagging (DSG), to evaluate the amount of sag based on the physical properties of the paint rheology and the geometry of the target. In the painting tests on flat plates with different angles, a strong positive correlation of 0.95 was observed between the sum of the calculated DSG values and the measured paint flow distance due to sag. In the painting tests on L-shaped surfaces, the predicted sag appearance by DSG agreed with the measured results at 14 of the 15 measurement points under all conditions. Overall, a computational model was developed for field implementation that could predict painting thickness distribution and sag occurrence. Note to Practitioners—The present work is motivated by the problem that teaching painting robots in a painting site is particularly time consuming. To address this problem, off-line programming methods have been proposed to computationally predict paint quality and film thickness and determine the painting route. Conventional approaches have approximated the paint deposition model with a function, which determines the paint thickness distribution, or simulated this model with high accuracy by considering electrostatic forces and wind velocity fields using computational fluid dynamics (CFD) modeling methods. However, two major problems exist in the field implementation of these studies: The first problem is the inability to predict defects, including sagging, that may occur after painting. In the conventional approach, the behavior of the paint after it adheres to the target is outside the scope of the calculation. The second problem is the high computational resource cost of CFD simulation, which renders its application to robotic painting challenging. In painting sites, the painting thickness must be accurately predicted and routed in a short period of time. In this paper, we present a method for analyzing film thickness distributions by considering time series changes of flow and propose a function-based model for predicting sagging based on the simulation results. The model for sag prediction was shown to be computationally inexpensive enough to be implemented in the field. The results of several painting experiments with a real robot confirmed the validity of the predictions made by this method. Our method could be implemented in the near future as a prediction system for off-line evaluation of painting routes, which could significantly reduce teaching cost. Since the target in this paper was a flat plate, it is necessary to conduct evaluation tests using various target shapes and painting conditions. Also, integration with the painting route optimization method has not yet been achieved, which is a topic that will be addressed in the future.

The purpose of this study is to determine the intervention method among individuals by analyzing and categorizing how the performance on a dual task decreases when the required attentional load increases through a validation test. The validation test consisted of a dual task with two different tasks. As a result of the validation test, we hypothesized that there exists a one-sided state in which response delays occur in the low-priority task during the transition from the dual-task state, where the task is performed stably, to the saturation state, where all tasks are disrupted. Therefore, considering the one-sided state, we classified participants' attention distribution state into four states during the dual-task and analyzed the transition of the collapse model for individual. As a result, we were able to identify the one-sided state, classify the collapse models of the dual-task into three patterns, and propose appropriate timing of intervention methods for each pattern.

Body language is an essential component of communication. The amount of unspoken information it transmits during interpersonal interactions is an invaluable complement to simple speech and makes the process smoother and more sustainable. On the contrary, existing approaches to human-robot collaboration and communication are not as intuitive, especially when it comes to robots transmitting information to their users. This is an issue that needs to be addressed if we aim to continue using artificial intelligence and machines to increase our capabilities, or as collaboration partners.

Post stroke patients may suffer from a gait disorder which is called foot drop. They may experience a toedrag, slow walking speed and a high risk of tripping. Although physical therapy exercises can be a remedy to such cases, their high cost and the lack of professional caregivers arise the need for robotic rehabilitation systems. On the other hand, the existing robotic systems for foot-drop rehabilitation are heavy and bulky. Therefore, in this paper, a lightweight foot-drop rehabilitation system based on a wire-driven redundant spherical parallel manipulator (RSPM) is developed. The proposed RSPM can provide 3-DOF movements to exercise the dorsiflexion, plantarflexion, supination, lateral and medial rotations. Kinematic and static analyses are performed for the proposed rob RSPM. In order to guarantee permissible positive tensions for the wires of the RSPM, a new performance index is introduced, called permissible tension index. Optimum design based on optimizing certain performance indices is carried out to obtain the proper dimensions of the movable platform design parameters of the proposed RSPM. A new index called permissible tension index is introduced to guarantee permissible positive tensions for the wires of the RSPM. A weighted sum of the global kinematic dexterity, global minimized transmission and the global permissible tension indices is considered for the objective function of the optimization process. The RSPM performance is evaluated over its range of motion and the reaction forces exerted on the ankle are shown to be tolerable by an injured ankle. Additionally, the Jacobian-based kinematic controller is shown to be capable of tracking the normal gait trajectories with acceptable tracking errors.

During auscultation, patients in difficult age often feel embarrassed and uncomfortable when exposing their chests to doctors of the different gender and being touched physically by doctors. We assume that an auscultation with robot technology can address the aforementioned gender-related issue. Toward eliminating gender bias during auscultation exam, this paper proposes a robotic platform which enables to perform the automated auscultation over clothes. Our developed system is comprised of two folds: a depth image-based estimation system of the listening positions over clothes with RGB-D camera and a contact force adjustment system for minimizing the acoustic attenuation due to the clothes with a passive-actuated end-effector. Our preliminary results demonstrated the robotic platform enables to estimate the listening locations to hear the sounds of four cardiac valves over the clothes by combining the estimated skeletal structure with statistical anatomical data and acquire the maximized acoustic quality over the clothes by adjusting the contact force. The developed robotic platform has the potential to address the gender-related issues in auscultation.

Unilateral spatial neglect (USN) is a higher cognitive dysfunction that can occur after a stroke. It is defined as an impairment in finding, reporting, reacting to, and directing to stimuli presented opposite to the damaged side of the brain. So far, we have developed an immersive virtual reality system that releases the attention biased to the non-neglected side and guides it to the neglected side to improve the symptoms of neglect. However, it improved neglect symptoms evaluated using a paper-and-pencil test on a desk (i.e., Behavioral Inattention Test), and it did not improve neglect in daily life evaluated using a standardized evaluation tool [i.e., Catherine Bergego Scale (CBS)], which is a rating index of neglect in actual daily life. Therefore, establishing a method for improving neglect in daily life is necessary. This study was designed to develop a platform for improving neglect in daily life as an initial investigation of rehabilitation methods for patients with USN. To improve neglect, the following three elements were derived from previous studies that should be included in rehabilitation: object search by moving around, reproduction of the complexity of daily life, and own body movement in the neglected side with visual confirmation. The patient moves around the room and visually explores and touches the objects presented in the mixed reality (MR) space. Using this system, we conducted a validity test on a patient with USN. The results showed that the patient improved from moderate to mild neglect in the CBS. Thus, the applicability of proposed MR system was suggested in terms of improving neglect in patients with USN in daily life environment.

The term “esports” is used to refer to game-based competitions considered as sports events. Esports currently involves a qualitative instruction that relies on individual experience and intuition, such as instruction from professional gamers. By contrast, sports observe a shift from qualitative to high-quality instruction using quantitative evaluation indices. Therefore, in esports, evaluation indices must be quantified to improve the training quality. This study focuses on crossing situations in soccer games. Crossing is a play in which a player passes from the side to the goal. It is important in soccer because of its high possibility of leading to a goal. Therefore, the quality of cross training in soccer must be improved to increase the scoring probability by crossing. As a feasibility study, this work aims at a quantitative evaluation of crossing in soccer games. The important factors for a successful cross are the positional relationship and the speed of each player at the time of the cross. A crossing scene is modeled to obtain a cross score based on these factors. First, when the velocity and trajectory of the cross ball are determined, the area is defined in which the offensive and defensive players can touch the ball. This area is calculated, and the value of how close to the center the ball trajectory passes in relation to the area is determined for each player. Using these values, the Cross Score (CS) was obtained by constructing a cross evaluation formula. To confirm the validity of the obtained evaluation values, a system that can obtain the evaluation values using image processing was developed. Then, 264 videos of crossing scenes in a soccer game were obtained, and CS was calculated for each of them. The correlation coefficient between the cross score and the percentage of successful crosses is 0.923, showing a strong correlation and confirming CS validity. As an example of application of the CS, it is possible to judge whether a cross is good or bad by displaying the CS on a heatmap using quantitative values. This enables visual and quantitative feedback using CS, suggesting the possibility of improving the quality of training.

Autonomous fetal ultrasonography with a robotic ultrasound (US) can potentially solve the issue of the shortage of ob-gyn physicians in prenatal care. In fetal cardiac diagnosis, acoustic shading derived from the fetal skeleton makes the scanning procedure using a robotic US cumbersome. We hypothesize that fetal bone distribution can be used for determining the fetal position and for subsequently estimating the heart position. This paper serves a method for estimating fetal heart position for autonomous cardiac diagnosis. Our proposed system is comprised of following three processes. First, the fetal bone distribution is generated by following steps: detect shadow areas derived from fetal bones; calculate the three-dimensional (3D) position of the bones; and identify bone types (skull, spine, and others) based on its shadow features. Next, the fetal head position and body axis are estimated based on the generated 3D bone distribution. Finally, the fetal chest position and thoracic sagittal axis of the fetus are estimated based on anatomical structures, and the position of the heart is determined. We validated the proposed method by evaluating the accuracies of bone detection and heart position estimation with three pregnant volunteers. The false positive rate of bone detection is less than 10%, which is sufficient for the proposed method; further, the estimation error for the heart position was within an acceptable error of 15 mm. These results demonstrate the sufficient accuracy of the proposed estimation method and suggest the potential of autonomous fetal cardiac diagnosis using robotic US scanning.

To reduce burdens on obstetricians and pregnant women, we developed a robotic fetal ultrasonography system, which can safely and remotely perform ultrasound scans on pregnant women’s abdomen. However, obstetricians must still control the robot in real time and learning the remote operation is time consuming. To solve this problem, the robot system must be improved to perform automatic ultrasound examination. A technical difficulty is that the robot must recognize the position and angle of the fetus in the uterus. In this study, we propose a fetal position estimation method based on existence probabilities of fetal body parts. The proposed method comprises 1) the classification of different fetal body parts based on the acquired ultrasound images and 2) the estimation of the appropriate position and angel of the probe to allow the robot to automatically collect the three-dimensional volume data of the fetus.

Fine needle insertions into a lung are challenging in terms of the needle deflection due to the breathing motion. Although previous related works neglected the effect for the needle deflection due to the breathing motion by patients stopping the breath during the insertion, they have to suffer from the discomfort. This paper proposes the intermittent insertion control method to decrease needle deflection adapting the lung deformation due to the breathing motion. The novelty of this method is to allow for accurate needle insertion without stopping the breath, which will contribute to decreasing the discomfort and the amount of radiation exposure for patients. The intermittent insertion is to move forward the fine needle during a certain time frame that the needle deflection barely occurs since the lung is not deformed by the diaphragm motion. The feasibility of the proposed method was validated through a polyvinyl chloride (PVC) phantom and ex vivo experiments. The results showed that the deflection can be suppressed up to 1.3 mm and 3.9 mm in the PVC phantom and ex vivo experiments, respectively.

Most stroke patients with hemiplegia have difficulty on heel rocker function due to weak dorsiflexors at their paralyzed side. The proposed high-dorsiflexion assistive robotic technology (RT) supports ankle dorsiflexion during gait rehabilitation. Aside from the McKibben-type artificial muscle's contraction to assist swing phase dorsiflexion, a tension spring is aligned in series to support heel rocker function upon heel strike. Because of different requirements of dorsiflexion torque during loading response phase, selection of suitable spring for corresponding user could be important. Springs with either excessively large or small coefficients might the normal heel rocker function. In this research, an identification method based on observing tension on the spring in loading response phase was derived. We aim for simple clinical setting and availability on patients with various pathological gait patterns, especially gait asymmetry. A case study on a stroke patient with hemiplegia was conducted to evaluate effect with the identified spring. Increase of knee angle (before intervention: mean 2.118°, SD 0.418°; during intervention: mean 12.814°, SD 1.764°) and shank angular velocity to the ground (before intervention: mean 117.300°/s, SD 9.491°/s; during intervention: mean 255.875°/s, SD 34.130°/s) indicates sufficient heel rocker function could be supported during intervention of the identified tension spring in this specific case.

Hand-eye coordination (HEC) is an important ability in a variety of sports, as it involves the ability to quickly and accurately control motion using visual information. This ability is extensively involved in badminton smash reception. In existing studies on HEC evaluation, this ability was evaluated according to information collected after the motion had been performed. Accordingly, it is impossible to determine the specific phase of perception or motion that causes problems in HEC. The purpose of this study was to develop a system for identifying the abilities of perception involved in HEC as it relates to badminton smash it is considered as badminton's most effective shot. Two systems were developed to identify HEC perceptual abilities, one focused on perception in a static state (i) and the other focused on perception-to-motion comprehensiveness (ii). Next, two types of tests were conducted, one involving (i) and the other using (ii). The tests were randomly administered for a total of 18 participants in three groups: six people who had previously played badminton, six people who had played ball sports other than badminton, and six people who were inexperienced at ball sports. Participants were healthy adults. The tests aimed to identify factors of perceptual ability by comparing them across systems. The results showed no significant difference between experienced badminton players and others according to the conducted tests, where test (i) was used for measuring the distance between the shuttlecock and gaze. A significant difference between the group of experienced badminton players and the other two groups was, however, observed in the test using (ii). The results suggested the ability to follow the shuttlecock with the eyes to be a perceptual ability related to HEC in badminton smash receptions. The results of this study can help evaluate perceptual ability during smash receptions.

Unilateral spatial neglect (USN) is a higher cognitive dysfunction that can occur after a stroke. It is defined as an impairment in finding, reporting, reacting to, and directing stimuli opposite the damaged side of the brain. We have proposed a system to identify neglected regions in USN patients in three dimensions using three-dimensional virtual reality. The objectives of this study are twofold: first, to propose a system for numerically identifying the neglected regions using an object detection task in a virtual space, and second, to compare the neglected regions during object detection when the patient's neck is immobilized (`fixed-neck' condition) versus when the neck can be freely moved to search (`free-neck' condition). We performed the test using an immersive virtual reality system, once with the patient's neck fixed and once with the patient's neck free to move. Comparing the results of the study in two patients, we found that the neglected areas were similar in the fixed-neck condition. However, in the free-neck condition, one patient's neglect improved while the other patient's neglect worsened. These results suggest that exploratory ability affects the symptoms of USN and is crucial for clinical evaluation of USN patients.

In this paper, we propose an automatic method for estimating fetal position based on classification and detection of different fetal parts in ultrasound images. Fine tuning is performed in the ultrasound images to be used for fetal examination using CNN, and classification of four classes leg and other is realized. Based on the obtained learning result, binarization that thresholds the gradient of the feature obtained by Grad Cam is performed in the image so that a bounding box of the region of interest with large gradient is extracted. The center of the bounding box is obtained from each frame so that the trajectory of the centroids is obtained; the position of the fetus is obtained as the trajectory. Experiments using 2000 images were conducted using a fetal phantom. Each recall ratiso of the four class is 99.6% for head, 99.4% for body, 99.8% for legs, 72.6% for others, respectively. The trajectories obtained from the fetus present in “left”, “center”, “right” in the images show the above-mentioned geometrical relationship. These results indicate that the estimated fetal position coincides with the actual position very well, which can be used as the first step for automatic fetal examination by robotic systems.

This paper presents an experimental evaluation of a collaborative task for a design assessment of a wearable robot arm. Wearable robotic devices have been recently proposed as a new concept of the human robot collaboration. In particular, wearable robot arms have been expected to complement our physical capabilities and perform multiple tasks simultaneously. However, a design principle of a wearable robot arm based on the cooperativeness and collision safety has not been discussed sufficiently. The design factors of the wearable robot arm are considered to have a dominant influence on the cooperativeness and collision safety. We therefore conducted an experiment to evaluate time required for a collaborative task and number of collisions with a robot arm device. Arm lengths and attachment positions were compared in the conducted experiment as a design factor of a wearable robot arm. The experimental results indicated a correlation between evaluation indexes and design factors. As a result, we suggest that the concept design of a wearable robot arm is recommended to consist of an extended arm length without constraints of the attachment positions.

Unilateral spatial neglect (USN) is defined as failure to report, respond, or adjust to novel or meaningful stimuli presented to the side opposite a brain lesion. This symptom deteriorates the rehabilitation efficiency and hinders activities of daily living. USN is classified into 2 sub-categories: body-centered neglect and object-centered neglect. No optimal intervention has been proposed for treating object-centered neglect. Thus, we developed a visual cueing system using immersive virtual reality to improve object-centered neglect. A feasibility study showed 2 typical measurements for object-centered neglect tended to improve the symptom, these results provide preliminary evidence for designing successful rehabilitation programs using a visual cueing system in patients with object-centered neglect.

Active ankle dorsiflexion in swing phase to ensure foot clearance, and resistive dorsiflexion for heel rocker function to prevent foot slap are important for a normal human gait. Stroke patients with hemiplegia have motor deficiency in their paralyzed side, and thus could not voluntarily generate enough dorsiflexion force. In this paper, we present our design of a light-weighted high dorsiflexion supportive robotic technology. It supports swing phase dorsiflexion with a McKibben-type artificial muscle, and supports heel rocker function with a tension spring. A pilot clinical trial shows the immediate improvement of dorsiflexion in swing phase by supporting high dorsiflexion angle in intervention, and the other pilot clinical trial shows the effects of heel rocker function support with lessened angular velocity of ankle plantarflexion and knee flexion.

Maintaining gait performance is important as a strategy for health promotion and fall prevention in older adults. In the present study, we introduced a haptic-based biofeedback (BF) system designed to augment foot pressure information with a wearable vibrotactile BF device attached to the pelvis. This device provided information regarding an older adult's foot pressure pattern to the person (trainee) and coach (trainer) to refine the interpersonal feedback. In this project, a feasibility laboratory study was first conducted to clarify the validity of the BF system (i.e., kinematic changes and cognitive burden of the device). Our preliminary study showed that the system had potential to modify the kinematic pattern in some older adults, but not walking speed (comprehensive evaluation of walking). Moreover, it is likely that the device required, to some extent, a cognitive burden for specific older people; thus, this aspect may have interfered with the usability of the BF in the initial practice phase. These results provided essential preliminary knowledge for designing a successful BF system and future trial with BF devices.

A new type of wearable robot that assists the user with additional arms has been presented in recent years. Previous studies have reported that wearable robot arms allow wearer to perform multiple tasks simultaneously. Although concept designs have been qualitatively discussed, quantitative consideration in the design phase of wearable robot arms is necessary to improve their safety and operational efficiency. In order to evaluate the man-machine cooperativeness between users and wearable robot arms, the present paper proposes three evaluation indexes:workspace extensiveness, cooperativeness and invasiveness. The proposed evaluation indexes were defined according to the calculations of the common domain of the human workspace and the robot arm workspace. The calculation results suggest a design tradeoff between the work efficiency and the workspace invasiveness. The proposed indexes can be utilized as a new parameter for evaluating human-robot interactions when designing wearable robot arms.

Accurate insertion of fine needles is difficult due to needle deflection. Needle deflection in the lower abdomen is particularly complex, as the needle has to pass through various tissues. As the area of the bowel is dominant in lower abdominal insertion, it is important to analyze the deflection during bowel insertion and to control the needle to minimize deflection. Few studies have focused on bowel insertion. We performed a fundamental deflection analysis of needle insertion in the bowel. Moreover, we have proposed an insertion method for minimizing the needle deflection based on our analysis. First, we performed needle insertion at various insertion positions and insertion angles into the hollow-shaped bowel, and determined the trend of the needle deflection during bowel insertion. The results revealed that the needle deflection was increased due to an increase in insertion angle, and therefore insertion angle should be minimized as much as possible. However, during actual bowel needle insertion, there are many situations in which the ideal path cannot be selected due to the arrangement of the bowel loops. We proposed an insertion method that can eliminate the total needle deflections during bowel insertion by controlling the needle tip direction at the breaching of each bowel wall. The results suggest that the needle deflection can be minimized by selecting the insertion path in which the sum of each insertion angle is zero. We verified the results of this insertion method in multiple bowels in in vivo experiments, and showed that it has the potential to be used in clinical practice.

In unmanned construction, work efficiency is lower than that in manned construction due to lack of visual information. Thus, we previously developed an autonomous camera control system to provide various visual information suited to work states through multiple displays. However, that system increased the cognitive load on operators, and required them to have much experience to choose appropriate views for various situations. Next, we should investigate the degree of effectiveness for each view in a certain state. Thus, in this study, we analyzed gaze patterns to clarify which are the displays that operators often watch in work states, i.e., moving, grasping, transport, and releasing. We then derived which gaze patterns have higher work performance, including time efficiency and safeness. We clustered gaze patterns using Ward's method, which is a criterion applied in hierarchical clustering. To evaluate the objective of this study, we conducted experiments involving debris transport tasks, using a virtual reality simulator. The results indicated that gaze patterns differed in operators and we found that better time efficiency related to specific gaze patterns for each work state.

This paper presents the design, development and magnetic resonance imagining (MRI) compatibility evaluation of a small size, compact and adjustable different finger phalange lengths rehabilitation device. This device employs ultrasonic motor as its actuator and adopts a novel six-link mechanism to drive the finger. The final system enables to provide two joints (the MCP and the PIP) in each finger to do flexion and extension motion with one degree of freedom (DOF). The MRI compatibility of the robot was also evaluated. The results demonstrate that there is neither an effect from the MRI environment on the robot performance, nor significant degradation on MRI images by the introduction of the robot in the MRI scanner. Finally, an fMRI study with subject was carried out, the result shows a stable brain activation was observed when the middle finger of the subject was driven to implement passive rehabilitation motion inside the MRI scanner.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. We have developed an autonomous camera control system to supply appropriate visual information. In order to inform operators the potential data in the views, we introduced several augmented reality (AR) elements to induce visual attention to suitable images in a scenario. The purpose of this study is to develop an attention inducement system using AR technique and evaluate it under different conditions. We first improve our autonomous camera control system and then implement several kinds of AR items suitable for each work situation. The experimental results conducted using our virtual reality simulator confirm that AR items can supply a better positional relationship between machine and objects in the environment which makes operator handle the conditions as experienced. The results from eye-tracker data also indicate that AR items can induce visual attention to images suitable for situations.

A method to autonomously control multiple environmental cameras, which are currently fixed, for providing more adaptive visual information suited to the work situation for advanced unmanned construction is proposed. Situations in which the yaw, pitch, and zoom of cameras should be controlled were analyzed and imaging objects including the machine, manipulator, and end-point and imaging modes including tracking, zoom, posture, and trajectory modes were defined. To control each camera simply and effectively, four practical camera roles combined with the imaging objects and modes were defined as the overview-machine, enlarge-end-point, posture-manipulator, and trajectory-manipulator. A role assignment system was then developed to assign the four camera roles to four out of six cameras suitable for the work situation, e.g., reaching, grasping, transport, and releasing, on the basis of the assignment priority rules, in the real time. Debris removal tasks were performed by using a VR simulator to compare fixed camera, manual control, and autonomous systems. Results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and error contacts and increasing the subjective usability while improving the time efficiency.

A method to tune a basic input/output gain (BIOG) according to usage conditions for human-machine systems is proposed for improving work performance. Adapting a BIOG is effective in terms of improving operability and work efficiency, but frequent changes using a gain scheduling strategy degrade the operability by confusing the machine dynamics. The proposed tuning system adjusts a BIOG at long intervals on the basis of comprehensive features in operator and work content obtained from the histogram of control lever input. The target value is set to the normal distribution, meaning that all ranges of the control lever are evenly used in a spring-type lever, as leveling the histogram independent of an operator and work content provides the consistent operational feeling, which leads to comfortable operability. To ensure practicality and effectiveness, a BIOG curve is set to a polygonal line involving a break point and a saturation point that are tuned by equivalent transform of area differences between the obtained histogram and normal distribution curves. Two types of experimental task were performed using a hydraulic arm system. Results showed that the proposed BIOG tuning system improves time efficiency by reducing the area difference while increasing the subjective usability compared with a conventional fixed BIOG system.

This paper proposes a framework for visualizing comprehensive work tendency using a frequency plot map of the end-point of a manipulator as a fundamental study of work analysis using long-term data for human-operated work machines. A visualization system requires high generality and commonality to enable to extract various characteristics to be extracted on an arbitrary time scale independently of the work machine specifications, work contents and environment, and machine operator. The proposed framework meets this requirement by first creating a two-dimensional end-point plot map with its origin fixed at the yaw joint of the manipulator to deal with arbitrary usage conditions. It then extracts arbitrary characteristics by using hierarchical feature extraction filters, including binary, quantity, and advanced filters, defined by three kinds of essential data: operation, movement, and load. Finally, it quantifies the filtered map by gridding and normalization to visually grasp its frequency distribution. Two experiments in which the work environments and completion time differed were conducted using an instrumented hydraulic arm. Results indicated that the comprehensive work tendency revealed by using the proposed framework corresponds to the actual work results independently of the various conditions.

This paper proposes a practical framework to estimate whether or not a grapple installed in demolition machines is in a grasp state. Object grasp is a highly difficult task that requires safe and precise operations, so identifying a grasp or non-grasp state is important for assisting an operator. These types of outdoor machines lack visual and tactile sensors, so the proposed framework adopts practically available lever operation and cylinder pressure sensors. The grasp is formed by a grasp motion, which is operations to make the grapple pinch an object, and the grasp state, where the grapple holds the object in any manipulator movements. Thus, the framework determinately confirms the grasp motion through the requisite conditions defined by using sequential changes of binarized operation and pressure data for the grapple and the manipulator, and stochastically confirms the grasp state through the enhancement conditions defined by using force and movement vectors including vertical downward force, movement in the longer direction, and horizontal reciprocating movement. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the proposed framework is effective for identifying grasp/non-grasp with high accuracy, independently of various operators and environments.

Handling objects with a single hand without dropping the object is challenging for a robot. A possible way to aid the motion planning is the prediction of the sensory results of different motions. Sequences of different movements can be performed as an offline simulation, and using the predicted sensory results, it can be evaluated whether the desired goal is achieved. In particular, the task in this paper is to roll a sphere between the fingertips of the dexterous hand of the humanoid robot TWENDY-ONE. First, a forward model for the prediction of the touch state resulting from the in-hand manipulation is developed. As it is difficult to create such a model analytically, the model is obtained through machine learning. To get real world training data, a dataglove is used to control the robot in a master-slave way. The learned model was able to accurately predict the course of the touch state while performing successful and unsuccessful in-hand manipulations. In a second step, it is shown that this simulated sequence of sensor states can be used as input for a stability assessment model. This model can accurately predict whether a grasp is stable or whether it results in dropping the object. In a final step, a more powerful grasp stability evaluator is introduced, which works for our task regardless of the sphere diameter.

This paper proposes a quantification method for a comprehensive work flow in construction work for describing work states in more detail on the basis of analyzing state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector. On the basis of the state transition rules derived from a transition-condition analysis, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined. PST can be classified into essential (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. To evaluate comprehensive work flows modeled by combing EST and NST, work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and work analysis using EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

This paper proposes an organ boundary determination method for detecting internal bleeding. Focused assessment with sonography for trauma (FAST) is important for patients who are sent into shock by internal bleeding. However, the FAST has a low sensitivity, approximately 42.7%, and delays of lifesaving treatment due to internal bleeding being missed have become a serious problem in emergency medical care. This study aims, therefore, to construct an automatic internal bleeding detection robotic system on the basis of ultrasound (US) image processing to improve the sensitivity. Internal bleeding has two key features: it is extracted from low-brightness areas in US images and accumulates between organs. We developed method for extracting low-brightness areas and determining algorithms of organ boundaries by low-brightness set analysis, and we detect internal bleeding by combining these two methods. Experimental results based on clinical US images of internal bleeding between Liver and Kidney showed that proposed algorithms had a sensitivity of 77.8% and specificity of 95.7%.

This paper proposes a practical framework for detecting (identifying the on-off state of) the external force applied to a construction manipulator (front load) by using a hydraulic sensor. Such a detection system requires high accuracy and robustness considering the uncertainty in pressure-based force measurement. Our framework is thus organized into (i) identifying the dominant error force component (self-weight and driving force) using theoretical and experimental estimation and binarizing the analog external cylinder force, (ii) evaluating detection conditions to address indeterminate conditions such as stroke-end, singular posture, and impulsive or oscillatory force and redefining three-valued outputs such as on, off, or not de terminate (ND), and (iii) outputting the front load decision by combining all the cylinder decisions to improve robustness through priority analysis. Experiments were conducted using an instrumented hydraulic arm. Results indicate that our frame work adequately detects on, off, and ND outputs of the front load in various detection conditions without misidentification.

This paper proposes a practical framework for measuring the external force applied to a construction manipulator (front load vector) by using a hydraulic sensor. Such a force measurement system requires high accuracy and robustness considering the uncertainty in the construction machinery field, but it inevitably includes measurement errors owing to difficult-to-reduce modeling errors. Our framework thus adopts a relative accuracy improvement strategy without correcting the models for the practicality. It comprises (i) quantifying the internal error range (IER) by using the sum of the maximal measurement errors of static and dynamic friction forces, which change in postural and kinematic conditions, (ii) calculating the error force vector by using IER to select cylinders (sensors) that have less error, and (iii) outputting the front load vector using the cylinders whose error force vector is minimum. Experiments were conducted using an instrumented hydraulic arm. The results indicate that our framework can enhance the relative accuracy of external force measurement independently of various postural and kinematic conditions.

This paper proposes a state identification framework to support the complicated dual-arm operations in construction work. The operational support in construction machinery filed requires the compatibility with different types of support and the commonality among various operator skill levels. The proposed framework is therefore organized into two functions: real-time task phase identification and time-series attentional condition identification. The task phase is defined by utilizing the joint load applied according to the environment constraint condition. The attentional condition is defined as one of the internal work-state condition classified by the necessity level of operational support, and is dependent on the vectorial or time-series date selected by the identified task phase. Experiments are conducted using the hydraulic dual arm system to perform transporting and removing tasks. Results show that the number of error contacts, internal force applied, and mental workload is decreased without time-consumption increase. The result confirmed that the proposed framework greatly contribute to improving each operator's work performance.

The purpose of this report is to propose a diagnosis and treatment scenario by assistance of bystander and echography robot for trauma patient. Quick treatment is important for patients who have shock by internal bleeding. Therefore, focused assessment with sonography for trauma (FAST), which is a simple and quick diagnostic method, was developed as a first lifesaving step in a hospital. However, a shock patient has little time, and transportation to a hospital may take too long. Therefore, we aim at development of a system which enables FAST at injury scene by assistance of bystander. To develop the system, life-saving flow and a FAST device are significant issues. First, we constructed a diagnosis and treatment scenario. Then, we developed a tele-echography robot system which has 4-DOF that a bystander could attach. This robot is attached to each roughly FAST areas of patient body by a bystander and remotely fine-tuned position by a doctor in a hospital. In this way, a bystander may not do an exact positioning. In addition, the robot has a mechanism to generate contact force between echo probe and patient body surface by two springs. This mechanism not only fit in patient body motion but also reducing the number of controlled axis. To confirm the medical applications of the scenario and the robot, we performed experiments with some examinees and doctors. We confirmed effectiveness of the mechanism and that a bystander could attach the robot to each roughly FAST areas of patient body. We also confirmed that a doctor could do FAST with the robot by remote-controlled on the roughly FAST areas in approximately three minutes. These results show that the robot would enable FAST by assistance of bystander, and the scenario would make FAST faster than the time required transporting the patient to the hospital.

Human-symbiotic humanoid robots that can perform tasks dexterously using their hands are needed in our homes, welfare facilities, and other places as the average age of the population increases. To improve the task performance of human-symbiotic humanoid robots, a motion-planning method with active body-environment contact was developed. Taking into account the positive and negative effect of mechanical passive elements implemented in joints, this motion-planning method can enables the hand-arm system to establish the active BE contact at the appropriate body-site and to select the joints that perform the movement for executing the given task. Control algorithms for the tool operation, namely, writing with a pen, were also constructed. The motion-planning method was validated through actual experiments on a prototype human-symbiotic humanoid robot.

In this paper, we propose a sophisticated design of human symbiotic robots that provide physical supports to the elderly such as attendant care with high-power and kitchen supports with dexterity while securing contact safety even if physical contact occurs with them. First of all, we made clear functional requirements for such a new generation robot, amounting to fifteen items to consolidate five significant functions such as ldquosafetyrdquo, ldquofriendlinessrdquo, ldquodexterityrdquo, ldquohigh-powerrdquo and ldquomobilityrdquo. In addition, we set task scenes in daily life where support by robot is useful for old women living alone, in order to deduce specifications for the robot. Based on them, we successfully developed a new generation of human symbiotic robot, TWENDY-ONE that has a head, trunk, dual arms with a compact passive mechanism, anthropomorphic dual hands with mechanical softness in joints and skins and an omni-wheeled vehicle. Evaluation experiments focusing on attendant care and kitchen supports using TWENDY-ONE indicate that this new robot will be extremely useful to enhance quality of life for the elderly in the near future where human and robot co-exist.

Advanced operated-work machines, which have been designed for complicated tasks and which have complicated operating systems, requires intelligent systems that can provide the quantitative work analysis needed to determine effective work procedures and that can provide operational and cognitive support for operators. Construction work environments are extremely complicated, however, and this makes state identification, which is a key technology for an intelligent system, difficult. We therefore defined primitive static states (PSS) that are determined using on-off information for the lever inputs and manipulator loads for each part of the grapple and front and that are completely independent of the various environmental conditions and variation in operator skill level that can cause an incorrect work state identification. To confirm the usefulness of PSS, we performed experiments with a demolition task by using our virtual reality simulator. We confirmed that PSS could robustly and accurately identify the work states and that untrained skills could be easily inferred from the results of PSS-based work analysis. We also confirmed in skill-training experiments that advice information based on PSS-based skill analysis greatly improved operator's work performance. We thus confirmed that PSS can adequately identify work states and are useful for work analysis and skill improvement.

Human-symbiotic humanoid robots that can perform tasks dexterously using their hands are needed in our homes, welfare facilities, and other places. To improve their task performance, we propose a motion control scheme aimed at appropriately coordinated hand and arm motions. By observing human manual tasks, we identified active body-environment contact as a kind of human manual skill and devised a motion control scheme based on it. We also analyzed the effectiveness of active body-environment contact in glass-placing and drawer-opening tasks. We validated our motion control scheme through actual tests on a prototype human-symbiotic humanoid robot.

In this paper, we describe the design method of an anthropomorphic 4-DOF tactile interaction manipulator with mechanically passive joints allowing robots to accomplish high performance force-following and tactile-based contact state recognition despite physical interference and contact with humans occurring at links and at joints. The shape and appearance of this manipulator was based on statistical data on the Japanese male physique to allow for human-like contact interaction recognition. Joint are passively compliant, consisting of a mechanical leaf spring and rotary damper giving the manipulator high passivity. Cylindrical or spherical surfaces of the body including the elbow and wrist are overlaid with distributed tactile sensors acquiring accurate contact information. Evaluation experiments indicate that the manipulator provides high performance of force-following and tactile stimulation measurement during physical interference and contact with humans and confirmed its effectiveness in improving human/robot symbiosis.

It is important for human symbiotic robots working near humans to have the adaptability to reliably follow force from humans while maintaining task performance despite unexpected disturbances. Thus, in the current study we propose a coordination control method of concurrently accomplishing task execution and human following even when physical interference and contact (PIFACT) occur with humans. First, functional requirements for the control method are specified from the viewpoints of motion-phase transition capability, time management characteristic of respective motion phase, and forms of task-performable human-following motion. Next, a control system architecture satisfying the requirements is presented. In addition, we describe a method of quantitatively representing a rule of task to process PIFACT adapting motions that allow achieving both human following and task performance according to the attributes of imposed tasks. Finally, experiments were carried out in which PlFACT was induced between humans and a full-size anthropomorphic robot equipped with the control architecture. We evaluated the results in terms of the comparison of variations of hand orientation and position while following humans during PIFACT among conditions where tasks with diverse rules were imposed on the robot. Evaluation of experiments demonstrates the proposed control architecture is useful for coordinating task execution and human following necessary for elevating human symbiotic robots.

In this paper, we propose a method for designing an identification system of human-robot contact states based on tactile recognition. First, a method of quantifying tactile cognition of a human (receiver) touched by other people (toucher) using a neural network called MCP (modified counter propagation) is presented, which matches the verbal response by the receiver with tactile stimulation detected during physical interference and contact utilizing tactile interface. It is incorporated that the probability of corresponding contact state is determined, based on the degree of similarity of the characteristics between new input data and reference data patterns stored in advance. Referring to the SOM (self-organizing maps) formed through learning, which contains the relationship between contact states and tactile stimulation detected, a robot that comes into contact with a human can recognize and infer contact states from tactile stimulation like the receiver. Next, in order to accomplish high-performance of contact state identification by improving the learning performance, an evaluation criterion to quantify the discriminatability of contact states is proposed. Finally, the experimental results confirm that the proposed method is useful for identifying contact states, based on only tactile sensing, as represented by the receiver.

In this paper, we propose a design method of wholebody tactile interface for elevating human-robot coordination. The proposed surface cover sensor as the tactile interface enables robots to detect accurate 3D force vector applied on various parts of the body during physical interaction with humans. First, after analysis of a variety of contact situations between humans and robots, tactile and force information needed are specified, and the specifications are decided based on knowledge of human engineering. Next, a design method of a cover sensor that can detect accurate force vector and contact position is described. The cover sensor utilizes a force-torque sensor and is surrounded with several touch sensors. The mechanism was implemented on a humanoid robot, WENDY. Finally, evaluation experiments of locus tracking on the surface and force-following to humans were carried out. The experimental results demonstrate the high capabilities of the proposed cover sensor as human-robot interface, and indicate that the proposed design method of whole-body tactile interface is capable of enhancing human-robot coordination.

The study described aims to develop human symbiotic robots, which have the abilities of carrying out physical, informational, and psychological interaction, and support daily work in a human's living space. We mainly discuss two essential design requirements for realizing human-robot symbiosis, such as safety and dexterity. First, through the development of human symbiotic robot WENDY (Waseda ENgineering Designed sYmbiont), a mechanical design method is proposed. Next, the effectiveness of the method is evaluated by several basic experiments, such as object grasping by using visual information, impact safety motion, and pressure control on the fingertip. Finally, performances of WENDY are also exhibited from several experiments that require high level integration of whole body mechanisms.

The authors describe the control methods used for a humanoid's compliant behavior following human force and motions while fulfilling given tasks under various constraint conditions during Physical InterFerence (PIF) with a human. PIF is a form of physical interaction viewed from the robot's point of view. In cases of PIF occurrence, PIF Adapting Behaviors for attenuating physical influences caused by PIF on both a human body and a robots' task are required. First, a base control method for compliantly following PIF by coordinating multiple joints of the arms and trunk is presented. By utilizing this method, PIF force produced on several areas of the robot's entire body is efficiently reduced. Next, the control methods for fulfilling given tasks, as well as following PIF at the same time, are proposed. In these methods, the idea is incorporated that if the utilization of redundancy is needed for task fulfillment or constraint conditions and attributes of the given task are changed, the role of each joint needs to change as well. Adapting to the diversity of task attributes and also the necessity of utilizing redundancy, the proposed control methods enable the robots to realize both force following and task fulfillment at the same time. Finally, from the evaluation of the experiments, it was confirmed that the proposed methods realize a humanoid's capability of compliantly adapting to human motions while fulfilling tasks by efficiently utilizing redundancy.

An objective of this study is to find out design requirements for developing human symbiotic robots, which share working space with humans, and have the ability of carrying out physical, informational, and psychological interaction. The paper mainly describes design strategies of the human symbiotic robots, through the development of a test model of the robots, WENDY (Waseda ENgineering Designed sYmbiont). In order to develop WENDY, mobility and dexterity of a humanoid robot Hadaly-2, which was developed in 1997, are improved on. The performances of WENDY are evaluated by experiments of object transport and egg breaking, which require high level integration of the whole body system.

In this paper, a comprehensive new concept concerning situations where robots physically contact with human such as tactile contacts and collisions is proposed, which is named physical interference (PIF). In order to realize PIF adaptation with human, it is required for robots to recognize PIF with human as well as secure human safety. First, information of PIF with human for recognizing PIF in detail is specified. Next, a hybrid PIF adapting hardware system composed of a mechanical passive compliant arm and PIF recognizing covers with viscoelastic materials is proposed and was developed. Finally, in order to evaluate the hardware system, experiments of adapting to PIF with human, using an arm equipped with MIA (mechanical impedance adjuster) and the developed surface covers, were carried out. From the results of the evaluation experiments, the validity of the developed hardware system for basic PIF adaptation with human was confirmed.