The application of humanoid robots is gaining attention as a solution to the caregiver shortage caused by an aging population. In this study, we automated the process of changing a patient’s body position from supine to lateral, a key aspect of nursing care. We proposed a method for recognizing a patient’s 3D posture at close range by simultaneously using a fisheye camera and a RGBD camera. For robot motion, we developed a trajectory generation method that adapts to the patient’s posture by converting measurement data into a mathematical model. Additionally, we identified the optimal timing for the movement of robot arms with minimal physical strain by considering human body dynamics. In all experiments using mannequins of different body shapes, the robot successfully reached the target joint and lifted one side of the body by more than 48 degrees. Future work will include detection of joints unaffected by body bulges and application the method to other repositioning movements.

Caregiving is a vital role for domestic robots, especially the repositioning care has immense societal value, critically improving the health and quality of life of individuals with limited mobility. However, repositioning task is a challenging area of research, as it requires robots to adapt their motions while interacting flexibly with patients. The task involves several key challenges: (1) applying appropriate force to specific target areas; (2) performing multiple actions seamlessly, each requiring different force application policies; and (3) motion adaptation under uncertain positional conditions. To address these, we propose a deep neural network (DNN)-based architecture utilizing proprioceptive and visual attention mechanisms, along with impedance control to regulate the robot's movements. Using the dual-arm humanoid robot Dry-AIREC, the proposed model successfully generated motions to insert the robot's hand between the bed and a mannequin's back without applying excessive force, and it supported the transition from a supine to a lifted-up position. The project page is here: https://sites.google.com/view/caregiving-robot-airec/repositioning

Tactile sensors are indispensable in robotic systems because they deliver vital contact information during environmental interactions. In our work, we leverage the variable compliance of a porous material—where different interaction forces induce varying degrees of compliance—to achieve self-adapting tactile sensing. This distinctive non-linear characteristic allows its sensitivity to be automatically tuned over a range from 0.008 mT/N to 0.045 mT/N. After coating with a magnetic polymer, the porous material functions as a 3-axis magnetic sensing medium. Its length and width are set at 30 mm and 35 mm respectively to accommodate the printed circuit board. To preserve the overall measuring range, it is designed with a thickness of 15 mm. This thickness enables monitoring of the volumetric changes due to the enhanced compliance, which is suitable for three-dimensional shape recognition. In this work, we present the design, fabrication, experimental characterization, and applications of an lightweight 3-axis magnetic sponge sensor with overall dimensions of 30 mm (width) × 35 mm (length) × 17 mm (height) and a detection range of 60 N. Notably, the sensing material weighs only 2 g, thanks to its porous structure.

Dialogue generation plays a key role in responding to user and providing transparency in motion execution in human-robot interaction. As motion planning is generally performed in terms of discrete motions, previous studies have focused on dialogue generation at the boundaries between motions. Recently, continuous motion generation was proposed to enable adapting actions to unique characteristics of the objects for domestic robots. Since a continuous motion generally involves physical and nonphysical phases, providing dialogues when the phase changes is crucial for decreasing users’ anxiety and guaranteeing safety. However, continuous motions lack clear phase boundaries, posing challenges for dialogue generation between phases. For this problem, we segmented continuous motions into discrete phases, and constructed a system to enable the robot to autonomously generate dialogues by detecting phase boundaries. To do so, we built phase estimation models using robot sensor data and designed system modules. Specifically, we collected data in the scenario of a robot assisting to lift the user up from bed. We segmented the continuous motion into three phases based on the user’s posture and whether the robot applied force to the human. The best phase estimation model achieved a macro F1 score of 0.894, demonstrating that phases can be estimated from sensor data. The evaluation results of our system demonstrated that the system accurately detects phase boundaries and generates appropriate dialogues corresponding to phases. Furthermore, we conducted simulations with a user agent to investigate system behaviors when the phase estimation was incorrect. The results suggested that explicitly stating the phase is important for avoiding misunderstandings and safety issues.

Traffic accidents, mostly caused by human errors, pose a significant challenge in automotive safety. This paper introduces TUN-DAS, a novel Time-series analysis and Unsupervised learning-based Driving behavior Assessment System, designed to systematically evaluate drivers’ behavior and identify potential risks. Unlike conventional in-cabin driver monitoring systems, which generally rely on rule-based or supervised methods, TUN-DAS utilizes a unique time-series clustering approach. This method combines a k-means algorithm with Dynamic Time Warping (DTW) and DTW Barycenter Averaging (DBA), enabling a more nuanced and comprehensive assessment of driving behavior. The modification of conventional DTW in our system allows for effective computation of temporal differences between driving data. TUN-DAS stands out in its ability to detect both sporadic human errors and consistent unsafe driving patterns by analyzing anomalies within individual drivers and across a broader driver dataset. Our validation with bus driver data shows TUN-DAS’s effectiveness in driving behavior assessment, significantly achieving better accuracy than traditional rule-based evaluation methods and allowing for quantitative and temporal analysis of a series of driving behavior. This system holds promise for enhancing road safety by facilitating the early identification of high-risk drivers, thereby contributing to the development of safer driving environments.

Mixed reality (MR) technologies have a high potential to enhance obstacle negotiation training beyond the capabilities of existing physical systems. Despite such potential, the feasibility of using MR for obstacle negotiation on typical training treadmill systems and its effects on obstacle negotiation performance remains largely unknown. This research bridges this gap by developing an MR obstacle negotiation training system deployed on a treadmill, and implementing two MR systems with a video see-through (VST) and an optical see-through (OST) Head Mounted Displays (HMDs). We investigated the obstacle negotiation performance with virtual and real obstacles. The main outcomes show that the VST MR system significantly changed the parameters of the leading foot in cases of Box obstacle (approximately 22 cm to 30 cm for stepping over 7cm-box), which we believe was mainly attributed to the latency difference between the HMDs. In the condition of OST MR HMD, users tended to not lift their trailing foot for virtual obstacles (approximately 30 cm to 25 cm for stepping over 7cm-box). Our findings indicate that the low-latency visual contact with the world and the user's body is a critical factor for visuo-motor integration to elicit obstacle negotiation.

Direct teaching is a straightforward way of teaching new motion to robots. Active methods with torque sensors, for example, can be used so that the robot can follow the movements of the human, but such methods introduce delays. Alternatively, series clutch actuators are easily backdrivable without delay. However, vertical joints are subject to gravity torques, which need to be compensated when disengaging the clutch. We implemented passive gravity compensation to counteract the robot’s weight, but this mechanism cannot compensate for varying payloads, as adjustable passive gravity compensation is relatively slow and mechanically complex. The varying payload causes an unintended joint movement, i.e. the arm falls down on its own, which is unacceptable during direct teaching. Therefore, this paper demonstrates how the torque output controlled with series clutch actuators can be used to compensate for varying payloads while maintaining high backdrivability. The proposed method is evaluated on a collaborative robot with a clutch in series for each actuator. Real-world experiments with payloads from 0 to 3 kg are conducted. During the experiments, the operator force is measured to evaluate the proposed method.

Handovers play a key role in human-robot interactions. However, current research focuses on visible-hand handovers, thereby heavily relying on hand detection. Large objects in human-robot interactions present a unique challenge: they inherently block the person’s hands and arms from the robot's view. This occlusion raises the robot’s risk of unintended physical contact with the person, leading to discomfort and safety concerns. This study aims to develop a model that can determine a pose for the robot that ensures a handover that avoids physical contact with the person, especially in scenarios when hands and arms are occluded. Toward this goal, a three-branch multimodal Deep Neural Network (DNN) regression model was implemented. First, a robust human-pose keypoints detection to calculate shoulder-elbow angles is applied. Secondly, we extract the refined object’s segmented mask. Thirdly, we compute two intrinsic object properties. The concatenated outputs from these branches pass through extra dense layers, resulting in the prediction of the robot's 14 arms-joint angles. Compared to an only keypoint data processed-based model, our multimodal approach made a 17.7% accuracy improvement. The experiments highlight each pipeline step’s significance, showing important results even when hands and arms were heavily occluded, adjusting to different variations.

Tactile sensors are crucial in robotics as they enable robots to perceive and interact with their environment through touch, akin to the human sense of touch. Adjustable sensors that can adapt to various tasks by functional or structural modification have not been extensively explored. In terms of sensing adjustability of a sensor, two important aspects are the sensor’s sensitivity and compliance. This paper proposes a novel design for an adjustable compliance and sensitivity sensor composed of a silicone base, a permanent magnet elastomer (PME), and a printed circuit board (PCB) with magnetic transducers installed. Its adjustability is achieved by varying the pneumatic pressure. This paper presents the design, manufacturing process, and experimental characterization of such an adjustable compliance and sensitivity sensor. This paper thoroughly investigates how altering the pressure of the sensor influences its sensing properties. The results show that it can achieve adjustability in all three axes. For the current design, the sensitivity can be varied from 0.093 to 0.125 mT/N (34.41%), 0.089 to 0.13 mT/N (31.54%), and 0.169 to 0.45 mT/N (62.44%) in the X-, Y-, and Z-axis, respectively. The deformation it undertakes varies from 3.20 to 3.79 mm (18.44%), indicating the compliance change.

Multi-fingered hands can be suitable for stable object manipulation. Furthermore, abundant tactile information can be acquired with multi-fingered hands, useful to recognize the object’s properties, which is beneficial to adapt the motion to the object. However, generating dexterous manipulation motions with multi-fingered hands with high density tactile sensors is challenging due to complex touch states. Hence, tasks that conventionally require a high level of active tactile sensing simultaneously with motion generation, such as pulling in the hand while recognizing the posture of an object are difficult to accomplish. In this letter, we propose a novel deep predictive learning approach using Vision-Transformer (ViT) and Long-Short Term Memory (LSTM). The ViT’s attention mechanism can spatially focus on specific fingers represented by distributed 3-axis tactile sensors (uSkin). The LSTM can preserve long time-series information of the manipulation which can realize changing the desired motion according to the initial touching position and orientation for the target object. Results showed that the ViT-LSTM is effective in performing adaptive finger movements according to the properties of the object, i.e. its hardness and relative posture.

Intelligent robots are expected to do manipulation tasks relying on real-time sensing feedback. Especially, tactile sensing plays a more and more important role in precise manipulation tasks. For example, a 1 mm error while inserting a USB stick, which is hard to perceive visually, will result in a failed insertion or even break the USB stick. In this paper, to estimate and compensate residual position uncertainties during robotic insertion tasks, an exploration motion is introduced to acquire environment information by tactile sensing and a state-of-the-art transformer-based neural network is proposed to estimate the error distance from long-duration tactile sensing data. Our system is trained on over 2000 insertion trials with basic geometry shaped 3D printed objects. Without any prior knowledge, we achieve an 85% insertion success rate with average 5 attempts on 4 unseen daily objects relying only on tactile feedback acquired from our proposed exploratory motion. It is noteworthy that our designed exploration motion can provide insightful information about extrinsic contact information and our proposed learning model exceeds previous baselines in extracting useful information regarding the contact interaction between the grasped object and the environment.

To promote rapid recovery and quality of life after a musculoskeletal disorder, rehabilitation exercises that are suitable for each individual's physical condition are important. In cases of disuse muscle atrophy of the quadriceps, inappropriate training can cause injury. Although resistance-training robotic systems have been developed and could adjust resistance load, a systematic detection method with appropriate force strength for automatic adjustment for each individual has not yet been established. In the current study, we developed an electromyogram (EMG) based method that determines the minimum effective resistance load for muscle growth. Using an integrated EMG (IEMG) model of incremental resistance load focused, we constructed a method to determine the minimum effective resistance load with logarithmic functions. The feasibility of our method was tested with a slow training protocol using a wire-driven leg extension training robot to measure the relationship between IEMG and resistance load by applying the incremental resistance load. The proposed model was found to be suitable for six young and four elderly subjects with different levels of muscle mass, and the load derived for each person was shown to induce effectively acute thigh circumference expansion, which is a factor leading to future muscle hypertrophy.

Multifingered robot hands can be extremely effective in physically exploring and recognizing objects, especially if they are extensively covered with distributed tactile sensors. Convolutional neural networks (CNNs) have been proven successful in processing high dimensional data, such as camera images, and are, therefore, very well suited to analyze distributed tactile information as well. However, a major challenge is to organize tactile inputs coming from different locations on the hand in a coherent structure that could leverage the computational properties of the CNN. Therefore, we introduce a morphology-specific CNN (MS-CNN), in which hierarchical convolutional layers are formed following the physical configuration of the tactile sensors on the robot. We equipped a four-fingered Allegro robot hand with several uSkin tactile sensors; overall, the hand is covered with 240 sensitive elements, each one measuring three-axis contact force. The MS-CNN layers process the tactile data hierarchically: at the level of small local clusters first, then each finger, and then the entire hand. We show experimentally that, after training, the robot hand can successfully recognize objects by a single touch, with a recognition rate of over 95%. Interestingly, the learned MS-CNN representation transfers well to novel tasks: by adding a limited amount of data about new objects, the network can recognize nine types of physical properties.

A fall on stairs can be a dangerous accident. An important indicator of falling risk is the foot clearance, which is the height of the foot when ascending stairs or the distance of the foot from the step when descending. We developed an augmented reality system with a holographic lens using a visual illusion to improve the foot clearance on stairs. The system draws a vertical striped pattern on the stair riser as the participant ascends the stairs to create the illusion that the steps are higher than the actual steps, and draws a horizontal striped pattern on the stair tread as the participant descends the stairs to create the illusion of narrower stairs. We experimentally evaluated the accuracy of the system and fitted a model to determine the appropriate stripe thickness. Finally, participants ascended and descended stairs before, during, and after using the augmented reality system. The foot clearance significantly improved, not only while the participants used the system but also after they used the system compared with before.

For assisting humans in their daily lives, robots need to perform long-horizon tasks, such as tidying up a room or preparing a meal. One effective strategy for handling a long-horizon task is to break it down into short-horizon subgoals, that the robot can execute sequentially. In this paper, we propose extending a predictive learning model using deep neural networks (DNN) with a Subgoal Proposal Module (SPM), with the goal of making such tasks realizable. We evaluate our proposed model in a case-study of a long-horizon task, consisting of cutting and arranging a pizza. This task requires the robot to consider: (1) the order of the subtasks, (2) multiple subtask selection, (3) coordination of dual-arm, and (4) variations within a subtask. The results confirm that the model is able to generalize motion generation to unseen tools and objects arrangement combinations. Furthermore, it significantly reduces the prediction error of the generated motions compared to without the proposed SPM. Finally, we validate the generated motions on the dual-arm robot Nextage Open. See our accompanying video here: https://youtu.be/3hYS2knRm50

Skill transfer from humans to robots is challenging. Presently, many researchers focus on capturing only position or joint angle data from humans to teach the robots. Even though this approach has yielded impressive results for grasping applications, reconstructing motion for object handling or fine manipulation from a human hand to a robot hand has been sparsely explored. Humans use tactile feedback to adjust their motion to various objects, but capturing and reproducing the applied forces is an open research question. In this paper we introduce a wearable fingertip tactile sensor, which captures the distributed 3-axis force vectors on the fingertip. The fingertip tactile sensor is interchangeable between the human hand and the robot hand, meaning that it can also be assembled to fit on a robot hand such as the Allegro hand. This paper presents the structural aspects of the sensor as well as the methodology and approach used to design, manufacture, and calibrate the sensor. The sensor is able to measure forces accurately with a mean absolute error of 0.21, 0.16, and 0.44 Newtons in X, Y, and Z directions, respectively.

These days, value structure and social structure are changing with the background of immigration, globalization, and diversification. In many countries, such as Japan, a foreign workforce is introduced due to ageing citizens and labour shortages. Such diversification would be a great opportunity to create innovation. Innovation is often achieved when ideas from different perspectives synergize with each other. In this paper, we focus on the job hunting scene and suggest recruiting diverse and cooperative teams as one form of recruitment. Conventional job-hunting services and Social networking services (SNSs) are limited to matching labour supply and demand or attracting people with similar values. To help build and recruit diverse and cooperative teams, we propose a system to build a platform to connect people with different backgrounds, skills, and values and promote their cooperation. With the system, job hunters could find a team where members can leverage each other’s strengths and compensate for each other’s weaknesses. Furthermore, companies could effectively evaluate the diversity and cooperativeness of teams for hiring decisions. The evaluation scenario demonstrated that the proposed system could increase both job hunter and recruiter satisfaction levels and social impact.

Driver inattention is a significant contributor to fatal car crashes, leading to a need for accurate driver gaze zone classification methods. However, these classifications are particularly challenging under unconstrained conditions, such as when a driver’s face is partially occluded by masks or scarves, when the environment has significant lighting differences, or when the driver’s eyeglasses have reflections. This paper presents a framework that addresses these challenges by combining computer vision techniques and different deep-learning models to robustly recognize a driver’s gaze zone under highly unconstrained conditions. The framework uses a Contrast-Limited Adaptive Histogram Equalization (CLAHE) to adjust the color space of the frame, making it easier to recognize features under varying light conditions. It then employs dense landmark detection techniques to achieve robust recognition of the face, eyes, and pupils, including the use of optical flow estimation methods for tracking pupil and eyelid movement. The framework considers two facial poses and trains individual Deep Neural Network (DNN) models for each pose. As facial structure varies among individuals, the feature vector parameters for these DNN models are based on different relations between pupil and eye landmarks proportional to the driver’s face. The method has demonstrated its outstanding performance under a dataset involving highly unconstrained driving conditions.

Detecting the onset/ongoing of slip, i.e. if a grasped object is slipping or will slip from the gripper while being lifted, is crucial. Conventionally, it is regarded as a tactile sensing related problem. However, recently multi-modal robotic learning has become popular and is expected to boost the performance. In this paper we propose a novel CNN-TCN model to fuse tactile and visual information for detecting the onset/ongoing of slip. In our experiments, two uSkin tactile sensors and one Realsense435i camera are used. Data is collected by randomly grasping and lifting 35 daily objects 1050 times in total. Furthermore, we compare our CNN-TCN model with the widely used CNN-LSTM model. As a result, our proposed model achieves a 88.75% detection accuracy and outperforms the CNN-LSTM model combined with different pretrained vision networks.

In both teleoperation in real space and exploration in virtual space, ‘passive’ and ‘active’ haptic feedback can help to improve the performance of the task, especially in object handover and exploring. However, the current wearable haptic devices are hard to display continuous omnidirectional motion feedback simultaneously, which makes it not yet achieved. In this study, we thus propose a cutaneous haptic device, which enables continuous omnidirectional motion feedback for exhibiting ‘active’ and ‘passive’ haptic feedback. By applying small smart actuators (i.e., piezo actuators), the device can obtain contact force and be wearable. By arranging the closed loop with a plain-woven structure, our device makes continuous omnidirectional motion feedback possible. Our 35 g device can generate 0.94 N contact force and 0.5 N shear force. The passive and active haptic evaluations also proved its haptic capability. In conclusion, our proposed device with ‘active’ and ‘passive’ haptic feedback can provide continuous omnidirectional motion making it possible to be used for precise teleoperation.

A treadmill system has a large potential to provide humans with an augmented walking experience in real-life without a spatial limitation. However, a treadmill gait is different from walking on level ground. In previous studies, the adaptive belt speed control of a treadmill was developed to achieve a self-paced walking for making the users' treadmill gait similar to their level ground gait. Such studies have focused on steady-state walking and regulating the user's position on the treadmill. A normal gait can be divided into an acceleration state after gait initiation, a steady state, and a deceleration state for stopping. The objective of this study is to develop a treadmill system with a wire tension application enabling a human to experience a similar gait to a level ground gait during the transition phase from an acceleration state to a steady state. We developed a treadmill 4 m long × 1 m wide. To allow a user to move on the treadmill during the gait acceleration phase, an insensitive zone where a user can move without the treadmill belt drive was set. In addition, the treadmill was equipped with a wire traction system to apply a traction force canceling the effect of the belt floor acceleration of the treadmill when the belt speed of the treadmill changes. Through an experiment with six participants, the proposed treadmill system allowed the users to move in an acceleration state with the same head acceleration pattern as with level ground walking and cancel the inertial effect with the wire traction, which enabled the users to transition to a steady state from an acceleration state.

A surface microstructure that mimics the surface of a gecko's foot can exert a large gripping force with a small contact force. If such a structure is applied to the fingertips of a two-fingered parallel gripper, stable grasping can be achieved independent of the wetting and frictional state of the contact surface. However, the adhesive force of the microstructure is large while releasing the object, which hinders the release of the object. In this study, we developed a release method using a conveyor mechanism that easily peels off in the direction of rotation with a focus on the characteristics of the micro-protrusion structure. This mechanism is driven in conjunction with the gripper's grasping and releasing motions. Our experiments confirmed that the gripper can stably release the object using the proposed mechanism. The proposal in this paper is a mechanism that dynamically changes the adhesive force on a fingertip by mechanically switching the surface state in accordance with the gripper's grasping and releasing states. This idea can be applied to not only surface microstructure such as gecko-inspired surfaces but also adhesive surfaces such as adhesive tape, and provides novel knowledge in the field of robotics as a method of mechanically changing the fingertip adhesive force.

Drowsy drivers cause many deadly crashes. As a result, researchers focus on using driver drowsiness classifiers to predict this condition in advance. However, they only consider constraint situations. Under highly unrestricted scenarios, this categorization remains extremely difficult. For example, several studies consider the driver’s mouth closure crucial for detecting drowsiness. However, the mouth closure cannot be seen when the driver wears a mask, which is a potential failure for these classifiers. Moreover, these works do not make experiments under unconstrained situations as environments with considerable light variation or a driver with eyeglasses reflections. As a result, this paper proposes a video-based novel pipeline that employs new parameters, computer vision and deep-learning techniques to identify drowsiness in drivers under unconstrained situations. First, we alter the Lab color space of the frame to ease strong light changes. Then, we achieve a robust recognition of the face, eyes and body-joints landmarks using dense landmark detection that includes optical flow estimation methods for 3D eyelid and facial expression movement tracking and an online optimization framework to build the association of cross-frame poses. After this, we consider three important landmarks: eyes, lower-face contour, and chest. We performed several pre-processing and combinations using these landmarks to compare the efficiency of three alternative feature vectors. Finally, we fuse spatiotemporal features using a Gated Recurrent Units (GRU) model. Results over a dataset with highly unconstrained driving conditions demonstrate that our method outperforms classifying the driver’s drowsiness correctly in various challenging situations, all under mask-wearing scenarios.

Vision plays an important role in motion planning for mobile robots which coexist with humans. Because a method predicting a pedestrian path with a camera has a trade-off relationship between the calculation speed and accuracy, such a path prediction method is not good at instantaneously detecting multiple people at a distance. In this study, we thus present a method with visual recognition and prediction of transition of human action states to assess the risk of collision for selecting the avoidance target. The proposed system calculates the risk assessment score based on recognition of human body direction, human walking patterns with an object, and face orientation as well as prediction of transition of human action states. First, we investigated the validation of each recognition model, and we confirmed that the proposed system can recognize and predict human actions with high accuracy ahead of 3 m. Then, we compared the risk assessment score with video interviews to ask a human whom a mobile robot should pay attention to, and we found that the proposed system could capture the features of human states that people pay attention to when avoiding collision with other people from vision.

Recently, tactile sensing has attracted great interest for robotic manipulation. Predicting if a grasp will be stable or not, i.e. if the grasped object will drop out of the gripper while being lifted, can aid robust robotic grasping. Previous methods paid equal attention to all regions of the tactile data matrix or all time-steps in the tactile sequence, which may include irrelevant or redundant information. In this paper, we propose to equip Convolutional Neural Networks with spatial-channel and temporal attention mechanisms (SCT attention CNN) to predict future grasp stability. To the best of our knowledge, this is the first time to use attention mechanisms for predicting grasp stability only relying on tactile information. We implement our experiments with 52 daily objects. Moreover, we compare different spatio-temporal models and attention mechanisms as an empirical study. We found a significant accuracy improvement of up to 5% when using SCT attention. We believe that attention mechanisms can also improve the performance of other tactile learning tasks in the future, such as slip detection and hardness perception.

Object picking with two-fingered grippers is widely used in practice. However, the deformability and slipperiness of the target object still remain a challenge, and not resolving them might lead to breaking or dropping of the grasped objects. To prevent such instances, tactile sensing plays an important role because it can directly detect even the subtle changes that occur during grasping. Mechanoreceptors in the human skin detect such events by the change in the skin shape and/or vibration. Using a similar approach, a combined deformation and slip detection system using a distributed 3axis tactile information with various time-scales is proposed. Specifically, the tactile information includes the z-axis data, which denotes the deformation of the skin perpendicular to the finger’s surface and the x- and y-axes, which measure deformations tangential to the surface. The perpendicular and tangential tactile information are used to determine the deformation and slip, respectively. The system is based on a multilayer perceptron (MLP) that outputs detection results from a 3-axis tactile information. Results showed that, the perpendicular and tangential tactile information with an appropriate timescale were effective for deformation and slip detection with over 89% and 95% recognition rates, respectively, measured for 40 different objects. Moreover, 195 out of 200 real-time untrained grasping states were successful detected. Finally, 10 untrained objects were successfully picked.

The skin is an important organ which enables humans to interact with the unstructured environment around. It is perfectly soft and covers the entire body providing immediate feedback even when that part is not directly in the field of vision. With the human skin as an inspiration, in this paper, we develop a novel completely soft robot skin for tactile sensing. The skin utilizes a new type of material called as Permanent Magnet Elastomer (PME) to replace the traditionally used hard permanent magnet for hall effect based tactile sensors. PME is formed by mixing Neodymium particles in a polymer base and using strong magnetization (up to 6 T) for anisotropy and to achieve strong and complete magnetization. The 6-axis soft PME is a perfect replacement for powerful hard magnets. We also do a thorough analysis of this material by infusing it in different types of silicone and as a result the most suitable combinations are selected. Performance tests show that the sensor can detect minute forces like 0.1 N. Moreover, the hysteresis test is carried out and the hysteresis error for our skin is found to be only 1.402%. An overloading test is also performed by loading the skin up to 64 N to check the robustness. In conclusion, the skin can produce reliable Triaxial force measurements and we present two models of it for smaller and large force range measurements respectively.

This paper presents the design and evaluation of the low-cost capacitive proximity-force-fusion sensor "safe skin", which can measure simultaneously the proximity of humans as well as the contact force. It was designed such that the force and proximity sensing functions can work concurrently without interfering with each other. Moreover, active shielding, on-chip digitization and ground isolation are implemented for the sensor to minimize the influence from stray capacitance and electromagnetic interference (EMI) from the environment, which ensures that the sensor has a high system robustness for industrial applications. The prototype version has the capability of detecting a grounded human hand sized object from a distance of 400 mm. Moreover, forces in the range of 5 to 40 N can be measured, with 43.7% hysteresis and 6.7% nonlinearity. Due to its sensing characteristics, when used on a robot, the sensor could be used to ensure the safety of nearby humans in the future. The sensor could also potentially be used as an interface for human-robot interaction (HRI).

Although exists several drivers' gaze direction classifiers to prevent traffic accidents caused by inattentive driving, making this classification while the driver's face is temporarily or permanently occluded remains exceptionally challenging. For example, drivers using masks, sunglasses, or scarves and daily light variations are non-ideal conditions that recurrently appear in an everyday driving scenario and are frequently overlooked by the existing classifiers. This paper presents a single camera framework gaze zone classifier that operates robustly even during non-uniform lighting, non-frontal face pose, and faces undergo temporal or permanent occlusions. The usage of a normalized dense aligned face pose vector, the classification result of a pre-processed right eye area pixels, and the classification result of a pre-processed left eye area pixels is the cornerstone of the feature vector used in our model. The key of this paper is double-folded: firstly, the usage of a normalized dense alignment for a robust face, landmark, and head-pose direction detection and secondly, the processing of the right and left eye images using computer vision and deep learning techniques for refining, modifying, and finally labeling eyes information. Experiments on a challenging dataset involving non-uniform lighting, non-frontal face pose, and faces with temporal or permanent occlusions show each feature's importance towards making a robust gaze zone classifier under unconstrained driving situations.

Synthetic fibre ropes have high tensile strength, a lower friction coefficient and are more flexible than steel ropes, and are therefore increasingly used in robotics. However, their characteristics are not well studied. In particular, previous work investigated the long-term behaviour only under static loading. In this paper, we investigate the elongation behaviour of synthetic fibre ropes under cyclic loading. In particular, we use ropes made from Dyneema DM20 (UHMWPE) and Zylon AS (PBO), which according to prior work have low creep. While Dyneema is more widely used, Zylon has higher tensile strength. We could show that under cyclic loading the Dyneema DM20 rope elongated more than 9% and kept on extending even after 500 cycles. Zylon exhibited a more stable and lower elongation of less than 3%.

The use of tactile information is one of the most important factors for achieving stable in-grasp manipulation. Especially with low-cost robotic hands that provide low-precision control, robust in-grasp manipulation is challenging. Abundant tactile information could provide the required feed-back to achieve reliable in-grasp manipulation also in such cases. In this research, soft distributed 3-axis skin sensors ("uSkin") and 6-axis F/T (force/torque) sensors were mounted on each fingertip of an Allegro Hand to provide rich tactile information. These sensors yielded 78 measurements for each fingertip (72 measurements from the uSkin and 6 measurements from the 6-axis F/T sensor). However, such high-dimensional tactile information can be difficult to process because of the complex contact states between the grasped object and the fingertips. Therefore, a convolutional neural network (CNN) was employed to process the tactile information. In this paper, we explored the importance of the different sensors for achieving in-grasp manipulation. Successful in-grasp manipulation with untrained daily objects was achieved when both 3-axis uSkin and 6-axis F/T information was provided and when the information was processed using a CNN.

Performing various in-hand manipulation tasks, without learning each individual task, would enable robots to act more versatile, while reducing the effort for training. However, in general it is difficult to achieve stable in-hand manipulation, because the contact state between the fingertips becomes difficult to model, especially for a robot hand with anthropomorphically shaped fingertips. Rich tactile feedback can aid the robust task execution, but on the other hand it is challenging to process high-dimensional tactile information. In the current paper we use two fingers of the Allegro hand, and each fingertip is anthropomorphically shaped and equipped not only with 6-axis force-torque (F/T) sensors, but also with uSkin tactile sensors, which provide 24 tri-axial measurements per fingertip. A convolutional neural network is used to process the high dimensional uSkin information, and a long short-term memory (LSTM) handles the time-series information. The network is trained to generate two different motions ("twist" and "push"). The desired motion is provided as a task-parameter to the network, with twist defined as -1 and push as +1. When values between -1 and +1 are used as the task parameter, the network is able to generate untrained motions in-between the two trained motions. Thereby, we can achieve multiple untrained manipulations, and can achieve robustness with high-dimensional tactile feedback.

Future robots are expected to share their workspace with humans. Controlling and limiting the forces that such robots exert on their environment is crucial. While force control can be achieved actively with the help of force sensing, passive mechanisms have no time delay in their response to external forces, and would therefore be preferable. Series clutch actuators can be used to achieve high levels of safety and backdrivability. This work presents the first implementation of a linear series clutch actuator. It can exert forces of more than 110N while weighing less than 2kg. Force controllability and safety are demonstrated. Static balancing, which is important for the application in a vertical joint, is also implemented. The power consumption is evaluated, and for a payload of 3kg and with the maximum speed of 94mm/s, the power consumed by the actuator is 11W. Overall, a practical implementation of a linear series clutch actuator is reported, which can be used for future collaborative robots.

Adaptive assistance of gait training robots has been determined to improve gait performance through motion assistance. An important control role during walking is to avoid tripping by controlling minimum toe clearance (MTC), which is an indicator of tripping risk, to avoid its decrease among gait cycles. No conventional gait training robots can adjust assistance timing based on MTC. In this paper, we propose a system that applies force intermittently based on the MTC prediction algorithm to encourage people to avoid lowering the MTC. This prediction algorithm is based on a radial basis function network, the input data of which include the angles, angular velocities, and angular accelerations of the hip, knee, and ankle joints in the sagittal and coronal planes at toe-off. The cable-driven system that can switch between assistance and non-assistance modes applies force when the predicted MTC is lower than the mean value. Nine participants were asked to walk on a treadmill, and we tested the effect of the system. The MTC data before, during, and after the assistance phase were analyzed for 120 s. The results showed that the minimum and first quartile values of MTC could be increased after the assistance phase.

We propose a deep learning model for a robot to wipe 3D-objects. Wiping of 3D-objects requires recognizing the shapes of objects and planning the motor angle adjustments for tracing the objects. Unlike previous research, our learning model does not require pre-designed computational models of target objects. The robot is able to wipe the objects to be placed by using image, force, and arm joint information. We evaluate the generalization ability of the model by confirming that the robot handles untrained cube and bowl shaped-objects. We also find that it is necessary to use both image and force information to recognize the shape of and wipe 3D objects consistently by comparing changes in the input sensor data to the model. To our knowledge, this is the first work enabling a robot to use learning sensorimotor information alone to trace various unknown 3D-shape.

Measuring the forces that humans exert with their fingers could have many potential applications, such as skill transfer from human experts to robots or monitoring humans. In this paper we introduce the "Exo-Glove" system, which can measure the joint angles and forces acting on the human finger without covering the skin that is in contact with the manipulated object. In particular, 3-axis sensors measure the deformation of the human skin on the sides of the finger to indirectly measure the 3-axis forces acting on the finger. To provide a frame of reference for the sensors, and to measure the joint angles of the human finger, an exoskeleton with remote center of motion (RCM) joints is used. Experiments showed that with the exoskeleton the quality of the force measurements can be improved.

We consider the design of under-actuated articulated mechanism that are able to maintain stable static balance. Our method augments an user-provided design with counter-weights whose mass and attachment locations are automatically computed. The optimized counterweights adjust the center of gravity such that, for bounded external perturbations, the mechanism returns to its original configuration. Using our sketch-based system, we present several examples illustrating a wide range of user-provided designs can be successfully converted into statically-balanced mechanisms. We further validate our results with a set of physical prototypes.

Transmission systems that enable speed, torque, and direction conversion with high responsiveness and back-drivability are strongly required for higher performance robotic systems. Engagement states can be changed by clutches, but traditional clutches cannot change the direction and provide sufficient backdrivability. On the other hand, direction conversion can be realized by gear box such as bevel gears, but its backdrivability is poor. Thus, we newly develop a prototype transmission system with backdrivablity and responsiveness integrating clutch and direction converter using magnetorheological fluid (MRF). MRF is a functional fluid consisted of magnetic particles and carrier fluids which can change its viscosity rapidly and continuously according to the strength of magnetic field. MRF clutch consists of driving and driven shaft connected by vanes and coils for controlling MRF. MRF direction converter consists of bevel gears, brake, and MRF for reversing the output direction. In addition to traditional functions, such as speed and torque conversion, the proposed power transmission system provides five working modes: forward direction, reverse direction, and three kinds of free, according to states of the MRF clutch, MRF direction converter, and brake. Preliminary experiments revealed that the proposed transmission system could adequately realize implemented functions.

Distracted driving is one of the most common causes of traffic accidents around the world. Recognizing the driver's gaze direction during a maneuver could be an essential step for avoiding the matter mentioned above. Thus, we propose a gaze zone classification system that serves as a base of supporting systems for driver's situation awareness. However, the challenge is to estimate the driver's gaze inside not ideal scenarios, specifically in this work, scenarios where may occur self-occlusions or non-aligned head and eyes direction of the driver. Firstly, towards solving miss classifications during self-occlusions scenarios, we designed a novel protocol where a 3D full facial geometry reconstruction of the driver from a single 2D image is made using the state-of-the-art method PRNet. To solve the miss classification when the driver's head and eyes direction are not aligned, eyes and head information are extracted. After this, based on a mix of different data pre-processing and deep learning methods, we achieved a robust classifier in situations where self-occlusions or non-aligned head and eyes direction of the driver occur. Our results from the experiments explicitly measure and show that the proposed method can make an accurate classification for the two before-mentioned problems. Moreover, we demonstrate that our model generalizes new drivers while being a portable and extensible system, making it easy-adaptable for various automobiles.

In this paper, we present a lightweight Deformable Surface Mechanism (DSM) by applying shape-memory alloy (SMA) and sponge for moving objects as a soft actuator. The SMA is driven by heating and cooling processing with the cur-rent flowing. For the SMA, cooling is a process for recovering to original length which consumes time. In order to decrease the recovering time and making the surface deformable, a sponge sheet is applied in the mechanism. We used the cotton thread to sew the SMA into the sponge to manufacture the mechanism. The DSM contains a multi-triangle structure, and each triangle works as an individual actuation unit. By applying this structure and special sewing technique, the sponge sheet can be deformed in a vertical direction when the SMA contracted. While, when the current is turned off, the SMA can be stretched to the original length by the pushing force generated by the sponge. Therefore, a deformable surface mechanism with a rapid response can be achieved. We simulated the changing of uni-Deformable Surface Mechanism (uniDSM), and the experiments were followed to compare with the analyzed results. Additionally, different objects were examined on the DSM to test the conveyance ability.

In past research, in-hand object manipulation for various sized and shaped objects has been achieved. However, the network had to be trained for each different motion. Training data takes time to acquire and increases the hardware load, thereby increasing the cost for training data. Four-fingered in-hand manipulation is especially difficult as a high number of joints need to be controlled in synchrony. This paper presents a method that reduces the required training data for in-hand manipulation with the idea of pretraining and mutual finger motions. The Allegro Hand is used with soft fingertips and integrated 6-axis F/T sensors to evaluate the proposed method. To make the network more versatile, the training data included objects of various sizes and shapes. When pretraining the network, one shot learning suffices to learn a new task; mutual finger motions can be exploited to use three-fingered pretraining data for four-fingered manipulation. Both data-sharing and weight-sharing were used and show similar results. Crucially, pretraining data from fingers with the same kinematic chain has to be used, showing the importance of morphology specific learning. Moreover, objects with untrained sizes and shapes could be manipulated.

Distributed tactile sensors on multi-fingered hands can provide high-dimensional information for grasping objects, but it is not clear how to optimally process such abundant tactile information. The current paper explores the possibility of using a morphology-specific convolutional neural network (MS-CNN). uSkin tactile sensors are mounted on an Allegro Hand, which provides 720 force measurements (15 patches of uSkin modules with 16 triaxial force sensors each) in addition to 16 joint angle measurements. Consecutive layers in the CNN get input from parts of one finger segment, one finger, and the whole hand. Since the sensors give 3D (x, y, z) vector tactile information, inputs with 3 channels (x, y and z) are used in the first layer, based on the idea of such inputs for RGB images from cameras. Overall, the layers are combined, resulting in the building of a tactile map based on the relative position of the tactile sensors on the hand. Seven different combination variations were evaluated, and an over-95% object recognition rate with 20 objects was achieved, even though only one random time instance from a repeated squeezing motion of an object in an unknown pose within the hand was used as input.

The sense of touch is often crucial for humans to perform manipulation tasks. Providing tactile feedback during teleoperation or for users of prosthetic devices would be beneficial. However, the representation of tactile information constitutes a major technical challenge, since the numerous and possibly multimodal sensor readings are massive compared to the available tactile display technology. We introduce an algorithm that deploys two stages of K-means clustering along and across tactile image frames that render tactile sensor information at each time instant. In this manner, the massive tactile information is adaptively compressed in real-time while preserving its physical meaning, thus, remains intuitive and direct. We experimentally verify and examine the characteristics of our algorithm by evaluating the original and compressed tactile data. The data was gathered during the active tactile exploration of several objects of daily living by an Allegro robot hand that was covered with 15 uSkin sensor modules providing 2403-axis force vector measurements at each time instant. Our novel algorithm is straight forward enough to be implemented into tactile feedback systems. Finally, our algorithm allows for the direct feedback of massive tactile sensor data for a broad variety of tactile sensors and tactile displays, thereby, enables the compressed yet intuitive representation of massive tactile sensor information for real-time applications.

An end-effector such as a gripper or multi-fingered hand is essential to enable robots to grasp and manipulate objects of various size and shape. Soft skin increases the grasp stability and can provide space for tactile sensors. However, covering the joints with skin is challenging, typically causing a considerable surface area of multi-segment robot fingers not to be covered by skin. This also creates the risk that objects get pinched in the joints when flexing the fingers. The current paper suggests using a remote center motion (RCM) mechanism to move the center of joint rotation to the surface of a thick skin layer. In particular, a 6-bar mechanism is used. Thereby, a thick soft skin layer with a continuous surface can be realized. Furthermore, adaptive joint coupling with linkages is implemented. In the current paper a 2-fingered gripper is realized, and objects of various size and shape are grasped (from thin paper to objects of 135 mm diameter). The current gripper was manufactured with 3D-printed material to enable rapid prototyping, therefore the payload was limited to only 1 kg for this version. Overall, this paper shows the feasibility of an RCM for a robot finger and discusses the benefits and limitations of such a mechanism.

Although conventional service providers are independent from each other when attending most of the population, demanded services are changing along with the social structure. Especially in the case of Taiwan, the number of co-working families has been increasing, and self-employed households occupy a large proportion of all working forms. Due to their diverse lifestyles and work styles, services that are suitable for personal objectives and that optimize the use of time are required. To meet this demand, it is important to connect people and city facilities to make it easier to provide suitable services. Based on those backgrounds, an innovated personal service platform in Taiwan is proposed, focusing on three factors, including time, place and personal information to connect people and city service facilities. Among various kinds of services, we targets services purchased in cities such as sales, mobility services, health services, government services and so on. It aims to link these services flexibly and dynamically to achieve personal objectives according to each situation. And, it can provide suitable services for a variety of every-day living situations. With this system, people can increase satisfaction and free time, improving life quality while making the economy more dynamic.

Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the automated driving (AD) system to a human driver when they encounter system limitations. In such `unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.

Continuous, easy-to-implement, accurate inference of intended joint angles is important for effectively controlling powered prosthetic devices that can improve the lives and capabilities of upper-limb amputees. Estimation of intended joint angles is difficult because conventional biosignals are not directly related to the intended angle motion. In previous work, we began to address this issue by confirming that both transra-dial amputees and intact subjects, the measured deformation of the muscle bulge on the skin surface change according to the intended wrist joint angle. This paper presents a continuous prosthesis wrist joint control method using this deformation signal. We here verify the effectiveness of the distribution of the muscle bulge for accurate and stable wrist joint angle control in real time. The wrist joint angles were calculated in real time from a muscle viscoelastic model using the previously determined algorithm. We compared the error between measured and estimated angles with a conventional method, the Voigt model, and the KelvinVoigt model. Experimental results obtained for three intact people over three trials of wrist movement tasks gave the accuracy and stability of 7.96±6.16°when using the Voigt model; this is a similar performance compared to related work using a surface electromyogram. A method for continuously controlling the wrist joint angle for a prosthesis using the distribution of the muscle bulge was thus successfully established.

This paper investigates tactile object recognition with relatively densely distributed force vector measurements and evaluates what kind of tactile information is beneficial for object recognition. The uSkin tactile sensors are embedded in an Allegro Hand, and provide 240 triaxial force vector measurements in total in all fingers. Active object sensing is used to gather time-series training and testing data. A simple feedforward, a recurrent, and a convolutional neural network are used for recognizing objects. Evaluations with different number of employed measurements, static vs. time series data and force vector vs. only normal force vector measurements show that the high-dimensional information provided by the sensors is indeed beneficial. An object recognition rate of up to 95% for 20 objects was achieved.

Typically, the range and sensitivity of force sensors are determined during production. However, to be able to do both delicate and high-force demanding work, adjustable force sensitivity would be beneficial. The current paper proposes such a sensor by implementing a planar electromagnet above a 3-axis magnetic sensor, separated by soft foam. Furthermore, the sensor has digital output with an integrated microcontroller. The magnetic field strength with varying currents is examined in simulation, and the field changes according to displacements are investigated both in simulation and with the actual sensor. A prototype 3-axis force sensor is implemented and the relationship between the magnetic field change and the corresponding applied force is also investigated. It could be shown that the sensitivity of the sensor to displacements, as well as force, can indeed be adjusted.

Some electric-powered wheelchairs are recently redefined as personal mobility devices. Their users are not only elderly or handicapped people, but also passengers with large baggage or pedestrians going from station to destination, i.e., last-mile transport. Consequently, people with different operation skills and expectations on personal mobility would become new users of this kind of devices. Safe and comfort travel in human co-existing environment such as sidewalks and airports is a social expectation for personal mobility. In order to realize this, understanding the operation skill of each user by a practical and simple method is essential. This paper thus introduced a skill level classification method by machine learning using only joystick data as input. In order to determine the number of skill level clusters, basic 26 features of joystick operation data are used for unsupervised clustering (single-linkage). We then made evaluation indexes by using speed, speed control, and direction control. For a five-level classification by using gradient boosting as supervised learning, we achieved a 67% accuracy (tolerance: 0) and a 98% accuracy (tolerance: 1). Further analysis of the feature importance of gradient boosting revealed key features to a good operation. Results also show that skill level differed among people with different driving experiences.

Gas pipeline requires to be inspected regularly for leakages caused by natural disaster. Robots are widely used for pipeline inspection since they are more convenient than manual inspection. Several problems, however, exist due to the restriction by complex pipe networks. The most significant one is limited inspection range caused by restriction of cable length or wireless signal attenuation. In this paper, we proposed a concept of wireless relay communication to assist robot to extend the inspection range, and we newly developed a tracked robot chain system. In this system, each robot serves as a relay communication node. Leakage information of pipes are transmitted via these relay nodes. To ensure the stability of relay communication between adjacent robots, we adopted RSSI (received signal strength indication)-based evaluation method for cooperative and coordinated movement of robot chain system. Moreover, wireless application layer communication protocol (WALCP) was used to increase the stable performance of wireless relay communication. Each robot can self-navigate based on distance measurement module, which enables robots to pass through an elbow junction. Multiple experiments to evaluate relay transmission efficiency, RSSI-based cooperative movement, and comprehensive performance were conducted. Results revealed that our proposed system could realize relatively accurate relay transmission and RSSI-based coordinated movement.

Human sensing enables intelligent vehicles to provide driver-adaptive support by classifying perceived workload into multiple levels. Objective of this study is to classify driver workload associated with traffic complexity into five levels. We conducted driving experiments in systematically varied traffic complexity levels in a simulator. We recorded driver physiological signals including electrocardiography, electrodermal activity, and electroencephalography. In addition, we integrated driver performance and subjective workload measures. Deep learning based models outperform statistical machine learning methods when dealing with dynamic time-series data with variable sequence lengths. We show that our long short-term memory based recurrent neural network model can classify driver perceived-workload into five classes with an accuracy of 74.5%. Since perceived workload differ between individual drivers for the same traffic situation, our results further highlight the significance of including driver characteristics such as driving style and workload sensitivity to achieve higher classification accuracy.

Smooth and efficient robot navigation among humans is a crucial requirement for successful integration of robots in human society. Towards this end, an indispensable characteristic of robot action is legibility while communicating its intention. However, unlike humans, present robots cannot convey its intention through human-like non-verbal communication. This paper explores the use of projection indicators for communicating directional intent of a robot across three different `crossing scenarios' as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. The results of the study show statistically significant improvement in perceived feelings of the measured attributes when using the auxiliary communication method. The studied method also improves cooperation from the participants. Nevertheless, the improvement in perceived feeling does not necessarily replicate in terms of smoothness across all the scenarios.

Efficient navigation in a congested environment is immensely difficult for the current state-of-the-art mobile robots owing to the challenges related to both sensing and actuation. Consequently, conventional approaches in mobile robotics research prioritize safety by slowing down or stopping robot's movement under challenging circumstances. This paper considers an alternative approach in which the robot utilizes forearm contact to create space for itself and to act as a safety buffer, during very close or congested navigation interactions. First, two categories of contact methods are defined based on different navigation scenarios. These contact methods are analyzed through different contact positions and force directions in a set of comparative experiments. The results of the comparative experiments are then incorporated into a contact-based framework that outputs the necessity, and the appropriate form of forearm contact. Finally, a set of evaluation experiments are performed to test the usability and effectiveness of the constructed framework. The results indicate that the proposed framework effectively outputs appropriate contact according to the situation, and aids the robot while navigating through space-constrained scenarios.

Disaster response crawler robot OCTOPUS has four arms and four flippers for better adaptability to disaster environments. To further improve the robot mobility and terrain adaptability in unstructured terrain, we propose a new locomotion control method called compound motion pattern (CMP) for multi-limb robots like OCTOPUS. This hybrid locomotion by cooperating the arms and flippers would be effective to adapt to the unstructured terrain due to combining the advantages of crawling and walking. As a preliminary study on CMP, we proposed a fundamental and conceptual CMP while clarifying problems in constructing CMP, and developed a semi-autonomous control system for realizing the CMP. Electrically-driven OCTOPUS was used to verify the reliability and correctness of CMP. Results of experiments on climbing a step indicate that the proposed control system could obtain relatively accurate terrain information and the CMP enabled the robot to climb the step. We thus confirmed that the proposed CMP would be effective to increase terrain adaptability of robot in unstructured environment, and it would be a useful locomotion method for advanced disaster response robots.

Intelligent vehicles operating in different levels of automation require the driver to fully or partially conduct the dynamic driving task (DDT) and to conduct fallback performance of the DDT, during a trip. Such vehicles create the need for novel human-machine interfaces (HMIs) designed to conduct high-level vehicle control tasks. Multimodal interfaces (MMIs) have advantages such as improved recognition, faster interaction, and situation-adaptability, over unimodal interfaces. In this study, we developed and evaluated a MMI system with three input modalities; touchscreen, hand-gesture, and haptic to input tactical-level control commands (e.g. lane-changing, overtaking, and parking). We conducted driving experiments in a driving simulator to evaluate the effectiveness of the MMI system. The results show that multimodal HMI significantly reduced the driver workload, improved the efficiency of interaction, and minimized input errors compared with unimodal interfaces. Moreover, we discovered relationships between input types and modalities: location-based inputs-touchscreen interface, time-critical inputs-haptic interface. The results proved the functional advantages and effectiveness of multimodal interface system over its unimodal components for conducting tactical-level driving tasks.

We suggest that different behavior generation schemes, such as sensory reflex behavior and intentional proactive behavior, can be developed by a newly proposed dynamic neural network model, named stochastic multiple timescale recurrent neural network (S-MTRNN). The model learns to predict subsequent sensory inputs, generating both their means and their uncertainty levels in terms of variance (or inverse precision) by utilizing its multiple timescale property. This model was employed in robotics learning experiments in which one robot controlled by the S-MTRNN was required to interact with another robot under the condition of uncertainty about the other's behavior. The experimental results show that self-organized and sensory reflex behavior-based on probabilistic prediction-emerges when learning proceeds without a precise specification of initial conditions. In contrast, intentional proactive behavior with deterministic predictions emerges when precise initial conditions are available. The results also showed that, in situations where unanticipated behavior of the other robot was perceived, the behavioral context was revised adequately by adaptation of the internal neural dynamics to respond to sensory inputs during sensory reflex behavior generation. On the other hand, during intentional proactive behavior generation, an error regression scheme by which the internal neural activity was modified in the direction of minimizing prediction errors was needed for adequately revising the behavioral context. These results indicate that two different ways of treating uncertainty about perceptual events in learning, namely, probabilistic modeling and deterministic modeling, contribute to the development of different dynamic neuronal structures governing the two types of behavior generation schemes.

Human's ability to perceive intent plays a crucial role in achieving smooth and efficient navigation. At the present state, even with the state-of-the-art anthropomorphic robots, displaying human-like non-verbal communication (kinesics) is a challenging task. This poses a significant difficulty in performing legible navigation behavior for robots. In this paper, we look into light (turn indicator) and screen (arrow indicator) indicators as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. Our results show a statistically significant improvement in perceived comfort, predictability, and performance with the use of light indicators.

We developed a four-arm four-crawler advanced disaster response robot called OCTOPUS. Disaster response robots are expected to be capable of both mobility, e.g., entering narrow spaces over very rough unstable ground, and workability, e.g., conducting complex debris-demolition work. However, conventional disaster response robots are specialized in either mobility or workability. Moreover, strategies to independently enhance the capability of crawlers for mobility and arms for workability will increase the robot size and weight. To balance environmental applicability with the mobility and workability, OCTOPUS is equipped with a mutual complementary strategy between its arms and crawlers. The four arms conduct complex tasks while ensuring stabilization when climbing steps. The four crawlers translate rough terrain while avoiding toppling over when conducting demolition work. OCTOPUS is hydraulic driven and teleoperated by two operators. To evaluate the performance of OCTOPUS, we conducted preliminary experiments involving climbing high steps and removing attached objects by using the four arms. The results showed that OCTOPUS completed the two tasks by adequately coordinating its four arms and four crawlers and improvement in operability needs.

A combined method of Doppler positioning and carrier-based hyperbolic positioning with a multi-channel GPS-pseudolite is proposed for indoor localization. This method uses carrier-phase output from a GPS/pseudolite receiver. The carrier-phase observable is precise but does not provide range information between the pseudolite and receiver antennas necessary for position calculation. This is because of the existence of carrier ambiguity. This problem can be solved by using the proposed combined method. In the present work, the positioning theory is established and experimentally evaluated with actual devices including a robot. The experimental result shows that a positioning accuracy of more than 10 cm is achievable.

The advantages of mechanical compliance have driven the development of devices using new smart materials. A new kind of magnetorheological piston based on a toroidal array of magnetorheological valves, has been previously tested to prove its feasibility. However, being an initial prototype its potential was still limited by its complex design, and low output force. This study presents the revisions done to the design with several improvements targeting key performance parameters. An improved annular piston design is also introduced as comparison with conventional devices. The toroidal and annular piston head prototypes are built and tested, and their force performance compared with the previous iteration. The experimental results show an overall performance improvement of the toroidal assembly. However, the force model used in the study still fails to accurately predict the magnetic flux at the gaps of the piston head. This deviation is later verify and corrected using a FEM analysis. The force performance of the new toroidal assembly is on par with the commonplace annular design. It also displays a more linear behaviour, at the expense of lower energy efficiency. Finally, it also shows potential for a greater degree of customisation to meet different system requirements.

In-hand manipulation is often needed to accomplish a practical task after grasping an object. In-hand manipulation of variously sized and shaped objects in multi-fingered hands without dropping the object is challenging. In this paper we suggest a combined strategy of force control and passive adaptation through soft fingertips with simple interpolation control to achieve in-hand manipulation between various postures and with various objects. While passive compliance can be achieved in numerous ways, this paper uses soft skin, as it does not require complex mechanisms and was easy to integrate in the robot hand (Allegro hand). Softness has proven to significantly ease object grasping, and the current paper shows the importance of softness also for in-hand manipulation. In particular, the simple interpolation strategy between various postures is successful when combined with soft fingertips, with or without force control, but fails with hard fingertips. Objects of varying size, shape and hardness were reliably manipulated. While the soft fingertips enabled good results in our experiments, a sufficiently precise definition of the postures and object size was required. When combining the interpolation control with a force control strategy, bigger errors in defining the posture and object size are possible, without deforming or dropping the object, and the resultant force is lower. As a result, we achieved robust in-hand manipulation between various postures and with objects of different size, shape and hardness.

A novel joint angle estimation method is proposed using a new bio-signal for an amputee subject. We used the muscle bulge movement on the forearm skin surface as a new bio-signal for estimating the extent of motion in a previous study. We found that it is feasible to estimate the intended wrist joint angle using the distribution of the muscle bulge for intact subjects. Thus, in the present paper, we validate the feasibility of our method for an amputee. In applying our method to an amputee subject, we improved our distance sensor device so that it can accommodate the position of the muscle, which is variable for an amputee subject. In addition, we improved the algorithm that estimates the wrist joint angle using linear multiple regression for calculating the relationship between the intended wrist joint angle and the distribution of the muscle bulge. As a result, we found that the distribution of the muscle bulge changes for the amputee as for intact subjects. The movement of the position of the muscle bulge on the forearm skin corresponded to the extent of the intended wrist joint angle. According to the result of the estimation of the wrist joint angle, the root-mean-square error of the estimated angle with respect to the measured angle for the amputee was slightly larger than the error for intact subjects. Nevertheless, the root-mean-square error for the amputee was smaller than that when employing the previous method for intact subjects. Finally, it is feasible to use the muscle bulge movement on the forearm skin to estimate the intended wrist joint angle for the upper limb amputee.

Autonomous vehicles would make the future roads safer by keeping the human driver out of the loop. However, reduced degree of human-control could result in loss of the feeling of driving for some drivers. Therefore, in this study we proposed a method of interaction between the driver and autonomous vehicle by allowing the driver to control the vehicle's lateral and longitudinal motions. We adopted hand gestures as input modality because it can reduce driver's visual and cognitive demands. We first derived seven fundamental vehicle maneuvers to improve driver experience, and related them to seven independent hand gestures. We then created a hand gesture interface to control an autonomous vehicle, using Leap Motion as the gesture recognition platform. We conducted driving experiments involving twenty drivers in a virtual reality driving simulator to investigate the effectiveness of this interface for vehicle control. We evaluated the driving experience and drivers' opinions regarding the gestural interface. The results proved that semi-autonomous controlling using the hand gesture interface significantly reduced drivers' perceived workload.

Elderly people are at risk of falling because of their low toe clearance. Gait training to improve toe clearance could be instrumental in avoiding tripping. We propose using a gait-training robot that applies torque during the pre-swing phase to achieve this goal. It is still possible to revert to their original trajectory after the training, however, depending on the magnitude of the applied torque. We investigated the relation between the magnitude of the applied torque and the change in toe clearance before and after application of torque. We developed a robot and carried out an experiment in which a motor pulls a string embedded on the robotic frame worn by the participants, thereby applying torque during the pre-swing phase. The experimental task included walking on a treadmill for 50 s. We applied torque to the knee during the pre-swing phase for 20 s. The phases before and after applying torque were 15-s normal walking phases with no interference from the robot. We compared toe clearance during the phases before and after applying torque. We found that the toe clearance increased after applying a torque of 8 Nm. We were thus able to verify the influence of torque on toe clearance.

In unmanned construction, work efficiency is lower than that in manned construction due to lack of visual information. Thus, we previously developed an autonomous camera control system to provide various visual information suited to work states through multiple displays. However, that system increased the cognitive load on operators, and required them to have much experience to choose appropriate views for various situations. Next, we should investigate the degree of effectiveness for each view in a certain state. Thus, in this study, we analyzed gaze patterns to clarify which are the displays that operators often watch in work states, i.e., moving, grasping, transport, and releasing. We then derived which gaze patterns have higher work performance, including time efficiency and safeness. We clustered gaze patterns using Ward's method, which is a criterion applied in hierarchical clustering. To evaluate the objective of this study, we conducted experiments involving debris transport tasks, using a virtual reality simulator. The results indicated that gaze patterns differed in operators and we found that better time efficiency related to specific gaze patterns for each work state.

A new design of a magnetorheological piston prototype intended for passive or active force control in robotic applications for human robot interaction is introduced. It is based in a novel toroidal array of valves, contained within the piston head, which are used to control the output force of the actuator in order to achieve a high degree of reliability, size efficiency, and safety, by exploiting the material properties of magnetorheological fluids and permalloy metals. This paper describes the main points in the development of the magnetorheological piston prototype, the mathematical modelling of the magnetic circuit, and the results of the experiments conducted using a universal testing machine to evaluate the passive performance of the prototype. Results show the feasibility and performance of the new toroidal magnetic circuit of the magnetorheological hydraulic piston prototype. Improvements in order to be able to test the active performance of the design together with a pump setup are proposed.

Moving objects within the hand is challenging, especially if the objects are of various shape and size. In this paper we use machine learning to learn in-hand manipulation of such various sized and shaped objects. The TWENDY-ONE hand is used, which has various properties that makes it well suited for in-hand manipulation: a high number of actuated joints, passive degrees of freedom and soft skin, six-axis force/torque (F/T) sensors in each fingertip, and distributed tactile sensors in the skin. A dataglove is used to gather training samples for teaching the required behavior. The object size information is extracted from the initial grasping posture. After training a neural network, the robot is able to manipulate objects of untrained sizes and shape. The results show the importance of size and tactile information. Compared to interpolation control, the adaptability for the initial posture gap could be greatly extended. Final results show that with deep learning the number of required training sets can be drastically reduced.

We propose a method for realizing effective dynamic motion learning in a flexible-joint robot using motor babbling. Flexible-joint robots have recently attracted attention because of their adaptiveness, safety, and, in particular, dynamic motions. It is difficult to control robots that require dynamic motion. In past studies, attractors and oscillators were designed as motion primitives of an assumed task in advance. However, it is difficult to adapt to unintended environmental changes using such methods. To overcome this problem, we use a recurrent neural network (RNN) that does not require predetermined parameters. In this research, we propose a method for facilitating effective learning. First, a robot learns simple motions via motor babbling, acquiring body dynamics using a recurrent neural network (RNN). Motor babbling is the process of movement that infants use to acquire their own body dynamics during their early days. Next, the robot learns additional motions required for a target task using the acquired body dynamics. For acquiring these body dynamics, the robot uses motor babbling with its redundant flexible joints to learn motion primitives. This redundancy implies that there are numerous possible motion patterns. In comparison to a basic learning task, the motion primitives are simply modified to adjust to the task. Next, we focus on the types of motions used in motor babbling. We classify the motions into two motion types, passive motion and active motion. Passive motion involves inertia without any torque input, whereas active motion involves a torque input. The robot acquires body dynamics from the passive motion and a means of torque generation from the active motion. As a result, we demonstrate the importance of performing prior learning via motor babbling before learning a task. In addition, task learning is made more efficient by dividing the motion into two types of motor babbling patterns.

This paper presents the design, development and magnetic resonance imagining (MRI) compatibility evaluation of a small size, compact and adjustable different finger phalange lengths rehabilitation device. This device employs ultrasonic motor as its actuator and adopts a novel six-link mechanism to drive the finger. The final system enables to provide two joints (the MCP and the PIP) in each finger to do flexion and extension motion with one degree of freedom (DOF). The MRI compatibility of the robot was also evaluated. The results demonstrate that there is neither an effect from the MRI environment on the robot performance, nor significant degradation on MRI images by the introduction of the robot in the MRI scanner. Finally, an fMRI study with subject was carried out, the result shows a stable brain activation was observed when the middle finger of the subject was driven to implement passive rehabilitation motion inside the MRI scanner.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. We have developed an autonomous camera control system to supply appropriate visual information. In order to inform operators the potential data in the views, we introduced several augmented reality (AR) elements to induce visual attention to suitable images in a scenario. The purpose of this study is to develop an attention inducement system using AR technique and evaluate it under different conditions. We first improve our autonomous camera control system and then implement several kinds of AR items suitable for each work situation. The experimental results conducted using our virtual reality simulator confirm that AR items can supply a better positional relationship between machine and objects in the environment which makes operator handle the conditions as experienced. The results from eye-tracker data also indicate that AR items can induce visual attention to images suitable for situations.

As robots progressively continue to enter human lives, it becomes important for robots to navigate safely and efficiently in crowded environments. In fact, efficient navigation in crowded areas is an important prerequisite for successful coexistence between humans and robots. In this paper, we explore an unconventional idea wherein a robot tries to achieve a more efficient navigation by influencing an obstructing human to move away by means of contact. First, preliminary human reaction experiments were conducted wherein we established that we can successfully induce a human to move in a desired direction. Following this result, we have proposed a novel motion planning approach which considers inducement by contact. The system is then verified through simulation and real experiments. The results show us that the proposed method can be utilized for safer and more efficient navigation in a crowded, but relatively static environment.

Eye-tracking technology has been applied to surgical skill improvement and can observe differences in gaze data between the experienced and novice surgeon. Some eye trackers can even detect pupil diameter. We hypothesized that differences in technical proficiency among surgeons affect the extent of changes in pupil diameter, because the higher mental workload a laparoscopic task requires, the larger the person's pupil diameter dilates. In this study, we recorded the pupil diameters of pediatric surgeons while they performed a suturing task in a dry box environment. The more proficient surgeons exhibited a smaller distribution in pupil diameter than did the less proficient surgeons. Our data indicate that an individual's suturing proficiency is closely related to the degree of dispersion of pupil diameter.

A method to autonomously control multiple environmental cameras, which are currently fixed, for providing more adaptive visual information suited to the work situation for advanced unmanned construction is proposed. Situations in which the yaw, pitch, and zoom of cameras should be controlled were analyzed and imaging objects including the machine, manipulator, and end-point and imaging modes including tracking, zoom, posture, and trajectory modes were defined. To control each camera simply and effectively, four practical camera roles combined with the imaging objects and modes were defined as the overview-machine, enlarge-end-point, posture-manipulator, and trajectory-manipulator. A role assignment system was then developed to assign the four camera roles to four out of six cameras suitable for the work situation, e.g., reaching, grasping, transport, and releasing, on the basis of the assignment priority rules, in the real time. Debris removal tasks were performed by using a VR simulator to compare fixed camera, manual control, and autonomous systems. Results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and error contacts and increasing the subjective usability while improving the time efficiency.

A method to tune a basic input/output gain (BIOG) according to usage conditions for human-machine systems is proposed for improving work performance. Adapting a BIOG is effective in terms of improving operability and work efficiency, but frequent changes using a gain scheduling strategy degrade the operability by confusing the machine dynamics. The proposed tuning system adjusts a BIOG at long intervals on the basis of comprehensive features in operator and work content obtained from the histogram of control lever input. The target value is set to the normal distribution, meaning that all ranges of the control lever are evenly used in a spring-type lever, as leveling the histogram independent of an operator and work content provides the consistent operational feeling, which leads to comfortable operability. To ensure practicality and effectiveness, a BIOG curve is set to a polygonal line involving a break point and a saturation point that are tuned by equivalent transform of area differences between the obtained histogram and normal distribution curves. Two types of experimental task were performed using a hydraulic arm system. Results showed that the proposed BIOG tuning system improves time efficiency by reducing the area difference while increasing the subjective usability compared with a conventional fixed BIOG system.

This paper proposes a framework for visualizing comprehensive work tendency using a frequency plot map of the end-point of a manipulator as a fundamental study of work analysis using long-term data for human-operated work machines. A visualization system requires high generality and commonality to enable to extract various characteristics to be extracted on an arbitrary time scale independently of the work machine specifications, work contents and environment, and machine operator. The proposed framework meets this requirement by first creating a two-dimensional end-point plot map with its origin fixed at the yaw joint of the manipulator to deal with arbitrary usage conditions. It then extracts arbitrary characteristics by using hierarchical feature extraction filters, including binary, quantity, and advanced filters, defined by three kinds of essential data: operation, movement, and load. Finally, it quantifies the filtered map by gridding and normalization to visually grasp its frequency distribution. Two experiments in which the work environments and completion time differed were conducted using an instrumented hydraulic arm. Results indicated that the comprehensive work tendency revealed by using the proposed framework corresponds to the actual work results independently of the various conditions.

This paper proposes a practical framework to estimate whether or not a grapple installed in demolition machines is in a grasp state. Object grasp is a highly difficult task that requires safe and precise operations, so identifying a grasp or non-grasp state is important for assisting an operator. These types of outdoor machines lack visual and tactile sensors, so the proposed framework adopts practically available lever operation and cylinder pressure sensors. The grasp is formed by a grasp motion, which is operations to make the grapple pinch an object, and the grasp state, where the grapple holds the object in any manipulator movements. Thus, the framework determinately confirms the grasp motion through the requisite conditions defined by using sequential changes of binarized operation and pressure data for the grapple and the manipulator, and stochastically confirms the grasp state through the enhancement conditions defined by using force and movement vectors including vertical downward force, movement in the longer direction, and horizontal reciprocating movement. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the proposed framework is effective for identifying grasp/non-grasp with high accuracy, independently of various operators and environments.

Handling objects with a single hand without dropping the object is challenging for a robot. A possible way to aid the motion planning is the prediction of the sensory results of different motions. Sequences of different movements can be performed as an offline simulation, and using the predicted sensory results, it can be evaluated whether the desired goal is achieved. In particular, the task in this paper is to roll a sphere between the fingertips of the dexterous hand of the humanoid robot TWENDY-ONE. First, a forward model for the prediction of the touch state resulting from the in-hand manipulation is developed. As it is difficult to create such a model analytically, the model is obtained through machine learning. To get real world training data, a dataglove is used to control the robot in a master-slave way. The learned model was able to accurately predict the course of the touch state while performing successful and unsuccessful in-hand manipulations. In a second step, it is shown that this simulated sequence of sensor states can be used as input for a stability assessment model. This model can accurately predict whether a grasp is stable or whether it results in dropping the object. In a final step, a more powerful grasp stability evaluator is introduced, which works for our task regardless of the sphere diameter.

This paper proposes a quantification method for a comprehensive work flow in construction work for describing work states in more detail on the basis of analyzing state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector. On the basis of the state transition rules derived from a transition-condition analysis, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined. PST can be classified into essential (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. To evaluate comprehensive work flows modeled by combing EST and NST, work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and work analysis using EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

This paper proposes a novel hybrid-structured model for the adaptive localization of robots combining a stochastic localization model and a rhythmic action model, for avoiding vacant spaces of landmarks efficiently. In regularly arranged landmark environments, robots may not be able to detect any landmarks for a long time during a straight-like movement. Consequently, locally diverse and smooth movement patterns need to be generated to keep the position estimation stable. Conventional approaches aiming at the probabilistic optimization cannot rapidly generate the detailed movement pattern due to a huge computational cost; therefore a simple but diverse movement structure needs to be introduced as an alternative option. We solve this problem by combining a particle filter as the stochastic localization module and the dynamical action model generating a zig-zagging motion. The validation experiments, where virtual-line-tracing tasks are exhibited on a floor-installed RFID environment, show that introducing the proposed rhythm pattern can improve a minimum error boundary and a velocity performance for arbitrary tolerance errors can be improved by the rhythm amplitude adaptation fed back by the localization deviation.

A Doppler positioning method with orientation estimation for an indoor messaging system (IMES) is proposed. With this method, both position and orientation of a receiver are estimated simultaneously by using Doppler shifts produced by moving a receiver antenna under the use of two or more IMES transmitters. The proposed method is evaluated through an experiment in which the interval of two transmitters is varied. The results of the experiment demonstrate that centimeter- to decimeter-level positioning accuracy and orientation-estimation accuracy of ±3 degrees are achieved; these results were largely consistent with the theoretical values calculated from dilution of precision. In addition, magnetic-compass error indoors was experimentally investigated; the results show that a magnetic-compass is a large error source if it is used indoors. Lastly, which initial values of a nonlinear least-square used for the proposed method converge to appropriate position and orientation solutions is analyzed; the results of the analysis suggest that if an initial position is set to the midmost of two transmitters, a proper solution is obtained except in the case that initial orientation is 180 degrees opposite from the correct orientation.

This paper proposes an organ boundary determination method for detecting internal bleeding. Focused assessment with sonography for trauma (FAST) is important for patients who are sent into shock by internal bleeding. However, the FAST has a low sensitivity, approximately 42.7%, and delays of lifesaving treatment due to internal bleeding being missed have become a serious problem in emergency medical care. This study aims, therefore, to construct an automatic internal bleeding detection robotic system on the basis of ultrasound (US) image processing to improve the sensitivity. Internal bleeding has two key features: it is extracted from low-brightness areas in US images and accumulates between organs. We developed method for extracting low-brightness areas and determining algorithms of organ boundaries by low-brightness set analysis, and we detect internal bleeding by combining these two methods. Experimental results based on clinical US images of internal bleeding between Liver and Kidney showed that proposed algorithms had a sensitivity of 77.8% and specificity of 95.7%.

This paper addresses the rhythmic reference in physical human-robot interaction. Human refers to a rhythm from multiple sensing modalities when turning a rope with another human synchronously. This study verifies a hypothesis that some humans mix several rhythms of the modalities into a rhythm (rhythmic reference). Six participants, four males and two females, 21-23 years old, took part in eight experiments which examined the hypothesis. In each experiment, we masked the perception of each participant using eight combination of three kinds of masks, an eye-mask, headphones, and a force mask. Each participant interacted with an operator that turned a rope with a constant frequency. As a result of the experiments, a participant increased the controlling error as the number of masks was increased regardless the types of masked modalities. The result strongly supported our hypothesis.

This paper proposes a practical framework for detecting (identifying the on-off state of) the external force applied to a construction manipulator (front load) by using a hydraulic sensor. Such a detection system requires high accuracy and robustness considering the uncertainty in pressure-based force measurement. Our framework is thus organized into (i) identifying the dominant error force component (self-weight and driving force) using theoretical and experimental estimation and binarizing the analog external cylinder force, (ii) evaluating detection conditions to address indeterminate conditions such as stroke-end, singular posture, and impulsive or oscillatory force and redefining three-valued outputs such as on, off, or not de terminate (ND), and (iii) outputting the front load decision by combining all the cylinder decisions to improve robustness through priority analysis. Experiments were conducted using an instrumented hydraulic arm. Results indicate that our frame work adequately detects on, off, and ND outputs of the front load in various detection conditions without misidentification.

A method to improve positioning accuracy of an indoor messaging system (IMES) was developed. This method uses Doppler shifts (produced by moving a receiver antenna) and three-dimensional attitude to determine the position of the receiver. A rotation-type Doppler-measurement system applying this method was developed. To evaluate the system, two experiments were conducted: in one, rotation radius of a movable receiver antenna was varied; in the other, position where positioning is conducted was varied. The experimental results show that the method can achieve centimeter- to decimeter-level positioning accuracy.

This paper proposes a practical framework for measuring the external force applied to a construction manipulator (front load vector) by using a hydraulic sensor. Such a force measurement system requires high accuracy and robustness considering the uncertainty in the construction machinery field, but it inevitably includes measurement errors owing to difficult-to-reduce modeling errors. Our framework thus adopts a relative accuracy improvement strategy without correcting the models for the practicality. It comprises (i) quantifying the internal error range (IER) by using the sum of the maximal measurement errors of static and dynamic friction forces, which change in postural and kinematic conditions, (ii) calculating the error force vector by using IER to select cylinders (sensors) that have less error, and (iii) outputting the front load vector using the cylinders whose error force vector is minimum. Experiments were conducted using an instrumented hydraulic arm. The results indicate that our framework can enhance the relative accuracy of external force measurement independently of various postural and kinematic conditions.

This paper presents a general method for creating an approximately optimal online stabilization system. An optimal stabilization system is an ideal online system that can calculate each optimal motion leading to a stable mechanical goal state depending on the current state. We propose a system that selects each semi-optimal motion according to the current state from a reverse-time tree. To create the reverse-time tree, we applied rapid semi-optimal motion planning method (RASMO) to a reverse-time search from a stable state. We also developed an online motion selection technique. To validate the proposed method, we simulated the stabilization of a double inverted pendulum. When we used an optimization criteria, time optimal, the system quickly stabilized the pendulum's posture and velocity. When we used higher resolution RASMO, the time approached the optimal time. The general framework proposed here is applicable to a variety of machines.

This paper proposes a state identification framework to support the complicated dual-arm operations in construction work. The operational support in construction machinery filed requires the compatibility with different types of support and the commonality among various operator skill levels. The proposed framework is therefore organized into two functions: real-time task phase identification and time-series attentional condition identification. The task phase is defined by utilizing the joint load applied according to the environment constraint condition. The attentional condition is defined as one of the internal work-state condition classified by the necessity level of operational support, and is dependent on the vectorial or time-series date selected by the identified task phase. Experiments are conducted using the hydraulic dual arm system to perform transporting and removing tasks. Results show that the number of error contacts, internal force applied, and mental workload is decreased without time-consumption increase. The result confirmed that the proposed framework greatly contribute to improving each operator's work performance.

The purpose of this report is to propose a diagnosis and treatment scenario by assistance of bystander and echography robot for trauma patient. Quick treatment is important for patients who have shock by internal bleeding. Therefore, focused assessment with sonography for trauma (FAST), which is a simple and quick diagnostic method, was developed as a first lifesaving step in a hospital. However, a shock patient has little time, and transportation to a hospital may take too long. Therefore, we aim at development of a system which enables FAST at injury scene by assistance of bystander. To develop the system, life-saving flow and a FAST device are significant issues. First, we constructed a diagnosis and treatment scenario. Then, we developed a tele-echography robot system which has 4-DOF that a bystander could attach. This robot is attached to each roughly FAST areas of patient body by a bystander and remotely fine-tuned position by a doctor in a hospital. In this way, a bystander may not do an exact positioning. In addition, the robot has a mechanism to generate contact force between echo probe and patient body surface by two springs. This mechanism not only fit in patient body motion but also reducing the number of controlled axis. To confirm the medical applications of the scenario and the robot, we performed experiments with some examinees and doctors. We confirmed effectiveness of the mechanism and that a bystander could attach the robot to each roughly FAST areas of patient body. We also confirmed that a doctor could do FAST with the robot by remote-controlled on the roughly FAST areas in approximately three minutes. These results show that the robot would enable FAST by assistance of bystander, and the scenario would make FAST faster than the time required transporting the patient to the hospital.

Reader antennas' configuration for two wheeled robots were evaluated to estimate their posture from floor-installed RFID tags. RFID systems where IC tags are installed under/on floors have widely been utilized in recent years as the next positioning infrastructure. The reader antennas should be properly placed on a robot so that such an environment can give full play to its potential capabilities of positioning the robot. This problem calls for guidelines in designing the configuration of reader antennas. Experiments using actual robots cannot offer sufficient data because of time and physical limitations, which prevent helpful and reproducible evaluations. We overcame this problem by constructing a simulation environment using a localization model and by evaluating the effects of configurations on positioning accuracy using computations. Of particular note, we found a “forward-backward configuration effect” from the results and had a detailed a discussion on how it occurred. Finally, a simple experiment using an actual robot validated the effect.

Human-symbiotic humanoid robots that can perform tasks dexterously using their hands are needed in our homes, welfare facilities, and other places as the average age of the population increases. To improve the task performance of human-symbiotic humanoid robots, a motion-planning method with active body-environment contact was developed. Taking into account the positive and negative effect of mechanical passive elements implemented in joints, this motion-planning method can enables the hand-arm system to establish the active BE contact at the appropriate body-site and to select the joints that perform the movement for executing the given task. Control algorithms for the tool operation, namely, writing with a pen, were also constructed. The motion-planning method was validated through actual experiments on a prototype human-symbiotic humanoid robot.

Safety issue has become the primary concern in wheelchair mounted robotic arm applications. We introduced mechanical gravity canceller in the design to realize a light weight manipulator, which also yield the greatly simplify of manipulator dynamics. Based on the simplified dynamics, sensor based contact detection can be easily implemented to enable safety. Contact reaction schemes are also applied through a impedance control law to achieve desired backdrivability. Experiments are conducted to illustrate the proposed method.

In this paper, we propose a sophisticated design of human symbiotic robots that provide physical supports to the elderly such as attendant care with high-power and kitchen supports with dexterity while securing contact safety even if physical contact occurs with them. First of all, we made clear functional requirements for such a new generation robot, amounting to fifteen items to consolidate five significant functions such as ldquosafetyrdquo, ldquofriendlinessrdquo, ldquodexterityrdquo, ldquohigh-powerrdquo and ldquomobilityrdquo. In addition, we set task scenes in daily life where support by robot is useful for old women living alone, in order to deduce specifications for the robot. Based on them, we successfully developed a new generation of human symbiotic robot, TWENDY-ONE that has a head, trunk, dual arms with a compact passive mechanism, anthropomorphic dual hands with mechanical softness in joints and skins and an omni-wheeled vehicle. Evaluation experiments focusing on attendant care and kitchen supports using TWENDY-ONE indicate that this new robot will be extremely useful to enhance quality of life for the elderly in the near future where human and robot co-exist.

Advanced operated-work machines, which have been designed for complicated tasks and which have complicated operating systems, requires intelligent systems that can provide the quantitative work analysis needed to determine effective work procedures and that can provide operational and cognitive support for operators. Construction work environments are extremely complicated, however, and this makes state identification, which is a key technology for an intelligent system, difficult. We therefore defined primitive static states (PSS) that are determined using on-off information for the lever inputs and manipulator loads for each part of the grapple and front and that are completely independent of the various environmental conditions and variation in operator skill level that can cause an incorrect work state identification. To confirm the usefulness of PSS, we performed experiments with a demolition task by using our virtual reality simulator. We confirmed that PSS could robustly and accurately identify the work states and that untrained skills could be easily inferred from the results of PSS-based work analysis. We also confirmed in skill-training experiments that advice information based on PSS-based skill analysis greatly improved operator's work performance. We thus confirmed that PSS can adequately identify work states and are useful for work analysis and skill improvement.

We have developed a novel way for robots to estimate their pose dynamically in an environment in which RFID tags have been arranged. We previously developed a method for localizing robots using a particle filter. Testing in a room equipped with a lattice of RFID tags at 300-mm intervals revealed that the estimation fails when the robot's RFID readers are near the center of the robot's rotation because the reader could not detect enough tags by rotating movements when the robot's positions are not suitable. We have overcome this problem by developing an active localization algorithm that generates an entropy map from the RFID arrangement information, predicts the pose using a particle filter, and attracts the robot to the target using a dynamic model, the fundamental unit of which is rotation-based angular velocity. Testing demonstrated that a robot using this algorithm and an entropy map can estimate its pose robustly without falling into a dead zone by moving only about 20 cm at most.

Human-symbiotic humanoid robots that can perform tasks dexterously using their hands are needed in our homes, welfare facilities, and other places. To improve their task performance, we propose a motion control scheme aimed at appropriately coordinated hand and arm motions. By observing human manual tasks, we identified active body-environment contact as a kind of human manual skill and devised a motion control scheme based on it. We also analyzed the effectiveness of active body-environment contact in glass-placing and drawer-opening tasks. We validated our motion control scheme through actual tests on a prototype human-symbiotic humanoid robot.

Wabot-House Research Laboratory is working on a project that will enable integrating robots into our everyday life. We believe that a “structured environment” (SE) will be one of most important concepts for this project. An SE generally means that objects near people that have some database or intelligence provide certain information to those people. An SE will also assist robot recognition or movement planning. Now, we focus on a global positioning system (GPS), which is a global SE that gives users or robots their positions whenever and wherever they are outdoors all over the world. GPS will strongly support robot self-positioning. However, GPS has the problem that it cannot be used when the robots are indoors. To solve this problem, we experimentally mounted four pseudolites (‘pseudo’ means imitated and ‘lite’ means satellite) in our laboratory and developed indoor GPS. The system worked well unless the robot was near the wall, where cycle slip often occurred. To examine the characteristics and reason for cycle slip, we measured the radio-wave environment in the laboratory. The first half of this paper introduces this system. The last reports results and findings about this experiment.

A method of estimating pose of a robot on a lattice of RFID tags is described. In recent years, radio frequency identification (RFID) technology has become a very popular method for localizing robots because it is robust to disturbances such as lighting and obstacles, which adversely affect the conventional methods that use cameras, supersonic waves and so on. Despite the advantage that RFID tags, especially passive tags, can be inexpensively mass-produced, previous studies using RFID have not targeted the detailed work of robots because they have made use of RFID tags dotted over a wide area as landmarks. Therefore, it is still difficult to use the technology at home. There is a model room in WABOT-HOUSE Laboratory of Waseda University where the floor is equipped with a lattice of RFID tags at 300 mm intervals, simulating a future home environment where robots interact symbiotically with humans. We speculate that such an environment, where the tags are distributed at regular intervals, is one of the most probable infrastructures of the near future and propose a method that use Monte Carlo localization to estimate the pose of robot on the lattice. Our experiments show that robots can localize their position more precisely than the interval of tags and also estimate their orientation successfully by using the proposed method when two readers are placed in appropriate positions.

We have proposed self-organizing network elements (SONE) as a learning method for robots to meet the requirements of autonomous exploration of effective output, simple external parameters, and low calculation costs. SONE can be used as an algorithm for obtaining network topology by propagating reinforcement signals between the elements of a network. Traditionally, the analysis of fundamental features in SONE and their application to supervised learning tasks were difficult because the learning method of SONE was limited to reinforcement learning. Here the abilities of generalization, incremental learning, and temporal sequence learning were evaluated using a supervised learning method with SONE. Moreover, the proposed method enabled our SONE to be applied to a greater variety of tasks.

We developed a learning system for autonomous robots that allows for autonomous exploration of the effective output, and has simple external parameters and a low calculation cost. We propose the concept of self-organizing network elements (SONE) for creating learning systems with these characteristics. We created and evaluated a self-organizing logic circuit by using this concept. Our results indicated this learning system had the characteristics

We created a human-robot communication system that can adapt to user preferences that can easily change through communication. Even if any learning algorithms are used, evaluating the human-robot interaction is indispensable and difficult. To solve this problem, we installed a machine learning algorithm called interactive evolutionary computation (IEC) into a communication robot named WAMOEBA-3. IEC is a kind of evolutionary computation like a genetic algorithm. With IEC, the fitness function is performed by each user. We carried out experiments on the communication learning system using an advanced IEC system named HMHE. Before the experiments, we did not tell the subjects anything about the robot, so the interaction differed among the experimental subjects. We could observe mutual adaptation, because some subjects noticed the robot's functions and changed their interaction. From the results, we confirmed that, in spite of the changes of the preferences, the system can adapt to the interaction of multiple users

In this paper, we describe the design method of an anthropomorphic 4-DOF tactile interaction manipulator with mechanically passive joints allowing robots to accomplish high performance force-following and tactile-based contact state recognition despite physical interference and contact with humans occurring at links and at joints. The shape and appearance of this manipulator was based on statistical data on the Japanese male physique to allow for human-like contact interaction recognition. Joint are passively compliant, consisting of a mechanical leaf spring and rotary damper giving the manipulator high passivity. Cylindrical or spherical surfaces of the body including the elbow and wrist are overlaid with distributed tactile sensors acquiring accurate contact information. Evaluation experiments indicate that the manipulator provides high performance of force-following and tactile stimulation measurement during physical interference and contact with humans and confirmed its effectiveness in improving human/robot symbiosis.

This paper describes how to implement interactive evolutionary computation (IEC) into a human-robot communication system. IEC is an evolutionary computation (EC) in which the fitness function is performed by human assessors. We used IEC to configure the human-robot communication system. We have already simulated IEC's application. In this paper, we implemented IEC into a real robot. Since this experiment leads considerable burdens on both the robot and experimental subjects, we propose the human-machine hybrid evaluation (HMHE) to increase the diversity within the genetic pool without increasing the number of interactions. We used a communication robot, WAMOEBA-3 (Waseda artificial mind on emotion base), which is appropriate for this experiment. In the experiment, human assessors interacted with WAMOEBA-3 in various ways. The fitness values increased gradually, and assessors felt the robot learnt the motions they desired. Therefore, it was confirmed that the IEC is most suitable as the communication learning system.

In this paper, we experimentally investigated the open-end interaction generated by the mutual adaptation between humans and robot. Its essential characteristic, incremental learning, is examined using the dynamical systems approach. Our research concentrated on the navigation system of a specially developed humanoid robot called Robovie and seven human subjects whose eyes were covered, making them dependent on the robot for directions. We used the usual feed-forward neural network (FFNN) without recursive connections and the recurrent neural network (RNN) for the robot control. Although the performances obtained with both the RNN and the FFNN improved in the early stages of learning, as the subject changed the operation by learning on its own, all performances gradually became unstable and failed. Next, we used a 'consolidation-learning algorithm' as a model of the hippocampus in the brain. In this method, the RNN was trained by both new data and the rehearsal outputs of the RNN not to damage the contents of current memory. The proposed method enabled the robot to improve performance even when learning continued for a long time (open-end). The dynamical systems analysis of RNNs supports these differences and also showed that the collaboration scheme was developed dynamically along with succeeding phase transitions.

It is important for human symbiotic robots working near humans to have the adaptability to reliably follow force from humans while maintaining task performance despite unexpected disturbances. Thus, in the current study we propose a coordination control method of concurrently accomplishing task execution and human following even when physical interference and contact (PIFACT) occur with humans. First, functional requirements for the control method are specified from the viewpoints of motion-phase transition capability, time management characteristic of respective motion phase, and forms of task-performable human-following motion. Next, a control system architecture satisfying the requirements is presented. In addition, we describe a method of quantitatively representing a rule of task to process PIFACT adapting motions that allow achieving both human following and task performance according to the attributes of imposed tasks. Finally, experiments were carried out in which PlFACT was induced between humans and a full-size anthropomorphic robot equipped with the control architecture. We evaluated the results in terms of the comparison of variations of hand orientation and position while following humans during PIFACT among conditions where tasks with diverse rules were imposed on the robot. Evaluation of experiments demonstrates the proposed control architecture is useful for coordinating task execution and human following necessary for elevating human symbiotic robots.

A novel approach to human-robot collaboration based on quasi-symbolic expressions is proposed. The target task is navigation in which a person with his or her covered and a humanoid robot collaborate in a context-dependent manner. The robot uses a recurrent neural net with parametric bias (RNNPB) model to acquire the behavioral primitives, which are sensory-motor units, composing the whole task. The robot expresses the PB dynamics as primitives using symbolic sounds, and the person influences these dynamics through tactile sensors attached to the robot. Experiments with six participants demonstrated that the level of influence the person has on the PB dynamics is strongly related to task performance, the person's subjective impressions, and the prediction error of the RNNPB model (task stability). Simulation experiments demonstrated that the subjective impressions of the correspondence between the utterance sounds (the PB values) and the motions were well reproduced by the rehearsal of the RNNPB model.

On the methodology of robotic design from the traditional view of communication which is assumed as a symbol process, robots are forced to confront the symbol grounding problem. However, if communication is assumed as the analog dynamics and robots are driven by it, robots can avoid the problem and be situated in the environment and to other agents. In this paper we introduce a new communication system constructed from the view of dynamical systems to achieve the situatedness. This system is that there is a robot in a virtual environment and the control of the robot is shared by human operation using a joystick and a robot controller. As the controller, we adopt multiple recurrent neural networks (MRNN) which are able to cope with complex environments and broad communication that single recurrent net cannot cope with. We conduct two experiments in order to evaluate the effectiveness of MRNN to a low level communication task such as nonverbal interaction. First, we examine the effect of the number of RNNs contained in MRNN. Second, we examine the effect of the context dependency of MRNN. These experiments show the capability of MRNN as a new-type controller of communication robot.

We developed an emotional communication robot, WAMOEBA, using behavior-based techniques. We also proposed motor-agent (MA) model, which is an autonomous distributed-control algorithm constructed of simple sensor motor coordination. Though it enables WAMOEBA to behave in various ways, the weight of the combinations between different motor agents is influenced by the preferences of the developer. We usually use machine-learning algorithms to automatically configure these parameters for communication robots. However, this makes it difficult to define the quantitative evaluation required for communication. We therefore used the method of interactive evolutionary computation (IEC), which can be applied to problems involving quantitative evaluation. IEC does not require to define a fitness function; this task is performed by users. But the biggest problem with using IEC is human fatigue, which causes insufficiency of individuals and generations for convergence of EC. To fix this problem, we use the prediction function that automatically calculates the fitness values of genes from some samples that have received the human subjective evaluation. Then, we carried out the behavior acquisition experiment using the IEC simulation system with the prediction function. As the results of experiments, it is confirmed that diversifying the genetic pool is an efficient way for generating a variety of behavior.

Recent studies on embodied cognitive science have shown us the possibility of emergence of more complex and nontrivial behaviors with quite simple designs if the designer takes the dynamics of the system-environment interaction into account properly. In this paper, we report our tentative classification experiments of several objects using the human-like autonomous robot, "WAMOEBA-2Ri". As modeling the environment, we focus on not only static aspects of the environment but also dynamic aspects of it including that of the system own. The visualized result of this experiment shows the integration of multimodal sensor dataset acquired by the system-environment interaction ("grasping") enable robust categorization of several objects. Finally, in discussion, we demonstrate a possible application to making "invariance in motion" emerge consequently by extending this approach.

In this paper, we propose a method for designing an identification system of human-robot contact states based on tactile recognition. First, a method of quantifying tactile cognition of a human (receiver) touched by other people (toucher) using a neural network called MCP (modified counter propagation) is presented, which matches the verbal response by the receiver with tactile stimulation detected during physical interference and contact utilizing tactile interface. It is incorporated that the probability of corresponding contact state is determined, based on the degree of similarity of the characteristics between new input data and reference data patterns stored in advance. Referring to the SOM (self-organizing maps) formed through learning, which contains the relationship between contact states and tactile stimulation detected, a robot that comes into contact with a human can recognize and infer contact states from tactile stimulation like the receiver. Next, in order to accomplish high-performance of contact state identification by improving the learning performance, an evaluation criterion to quantify the discriminatability of contact states is proposed. Finally, the experimental results confirm that the proposed method is useful for identifying contact states, based on only tactile sensing, as represented by the receiver.

In this paper, we investigated interactive learning between human subjects and robot experimentally, and its essential characteristics are examined using the dynamical systems approach. Our research concentrated on the navigation system of a specially developed humanoid robot called Robovie and seven human subjects whose eyes were covered, making them dependent on the robot for directions. We compared the usual feed-forward neural network (FFNN) without recursive connections and the recurrent neural network (RNN). Although the performances obtained with both the RNN and the FFNN improved in the early stages of learning, as the subject changed the operation by learning on its own, all performances gradually became unstable and failed. Results of a questionnaire given to the subjects confirmed that the FFNN gives better mental impressions, especially from the aspect of operability. When the robot used a consolidation-learning algorithm using the rehearsal outputs of the RNN, the performance improved even when interactive learning continued for a long time. The questionnaire results then also confirmed that the subject's mental impressions of the RNN improved significantly. The dynamical systems analysis of RNNs supports these differences.

In this paper, we propose a design method of wholebody tactile interface for elevating human-robot coordination. The proposed surface cover sensor as the tactile interface enables robots to detect accurate 3D force vector applied on various parts of the body during physical interaction with humans. First, after analysis of a variety of contact situations between humans and robots, tactile and force information needed are specified, and the specifications are decided based on knowledge of human engineering. Next, a design method of a cover sensor that can detect accurate force vector and contact position is described. The cover sensor utilizes a force-torque sensor and is surrounded with several touch sensors. The mechanism was implemented on a humanoid robot, WENDY. Finally, evaluation experiments of locus tracking on the surface and force-following to humans were carried out. The experimental results demonstrate the high capabilities of the proposed cover sensor as human-robot interface, and indicate that the proposed design method of whole-body tactile interface is capable of enhancing human-robot coordination.

Aims to investigate the intelligence which can make robots adapt to the human environment. The paper points out the problems of the behavior-based robot, and proposes the methods which can generate whole body motions based on body structure and integrates the reflection motions to make the behaviors continuous. The motion performances are compared in two kinds of environments, such as a dynamic environment and a static environment, by using a simulator of the autonomous robot WAMOEBA-2Ri developed in this research. Finally, we show that the integration parameters of the proposed method reflect the body structure of the robot and environmental structures.

This study presents a method of human cooperating systems, which can determine when support behavior is necessary by a human work model. We focus on assembly work as the target and propose a self-organizing approach of human work models by sampled human information by vision sensors. The support is determined according to the work model. Such a system would realize provision of support without a strict model of the assembly target and enable support in cases where neither the assembly process, nor the final form of the completed task is known to the system in advance. First, a method of measuring human information and extracting states where support is necessary, from the human work model is presented. Next a support system for assembly work cooperation according to the work model, with physical interaction capabilities is described. Experiments were carried out to evaluate and verify the effectiveness of the system. The results show that the constructed assembly support system is effective in both improving performance and increasing friendliness.

The study described aims to develop human symbiotic robots, which have the abilities of carrying out physical, informational, and psychological interaction, and support daily work in a human's living space. We mainly discuss two essential design requirements for realizing human-robot symbiosis, such as safety and dexterity. First, through the development of human symbiotic robot WENDY (Waseda ENgineering Designed sYmbiont), a mechanical design method is proposed. Next, the effectiveness of the method is evaluated by several basic experiments, such as object grasping by using visual information, impact safety motion, and pressure control on the fingertip. Finally, performances of WENDY are also exhibited from several experiments that require high level integration of whole body mechanisms.

The authors describe the control methods used for a humanoid's compliant behavior following human force and motions while fulfilling given tasks under various constraint conditions during Physical InterFerence (PIF) with a human. PIF is a form of physical interaction viewed from the robot's point of view. In cases of PIF occurrence, PIF Adapting Behaviors for attenuating physical influences caused by PIF on both a human body and a robots' task are required. First, a base control method for compliantly following PIF by coordinating multiple joints of the arms and trunk is presented. By utilizing this method, PIF force produced on several areas of the robot's entire body is efficiently reduced. Next, the control methods for fulfilling given tasks, as well as following PIF at the same time, are proposed. In these methods, the idea is incorporated that if the utilization of redundancy is needed for task fulfillment or constraint conditions and attributes of the given task are changed, the role of each joint needs to change as well. Adapting to the diversity of task attributes and also the necessity of utilizing redundancy, the proposed control methods enable the robots to realize both force following and task fulfillment at the same time. Finally, from the evaluation of the experiments, it was confirmed that the proposed methods realize a humanoid's capability of compliantly adapting to human motions while fulfilling tasks by efficiently utilizing redundancy.

This paper aims to clarify the cooperation intelligence of robots. This paper describes the autonomous robot named WAMOEBA-2R which can communicate with humans by both an informational and physical way. WAMOEBA-2R has two arms of which each joint has a torque sensor to realize the physical interaction with humans, the function of the voice recognition and the face recognition. The arms are controlled by a distributed agent network system. The network architecture is acquired in a neural network by the feedback-error-learning algorithm. We surveyed 150 visitors at the '99 International Robot Exhibition held in Tokyo (Oct. 1999) to evaluate their psychological impressions of WAMOEBA-2R. As a result, some factors of the human-robot emotional communication were discovered.

The study focuses on the dynamics of KANSEI information and aims to propose an algorithm of motion planning using KANSEI. Concretely, the violin playing is regarded as the target motion which will be greatly influenced by KANSEI. The study introduces a multi-agent algorithm in which four physical bowing parameters are agents to adapt the impression transition smoothly, while maintaining the relationships between the parameters. We realized the violin performance suitable for the timbre words by introducing the proposed agent algorithm into the bowing machine developed in this research. As a result of the experiments, it was confirmed that there were various playing performances according to a single impression transition.

An objective of this study is to find out design requirements for developing human symbiotic robots, which share working space with humans, and have the ability of carrying out physical, informational, and psychological interaction. The paper mainly describes design strategies of the human symbiotic robots, through the development of a test model of the robots, WENDY (Waseda ENgineering Designed sYmbiont). In order to develop WENDY, mobility and dexterity of a humanoid robot Hadaly-2, which was developed in 1997, are improved on. The performances of WENDY are evaluated by experiments of object transport and egg breaking, which require high level integration of the whole body system.

Discusses the communication between autonomous robots and humans through the development of a robot (WAMOEBA-2) which has an emotion model. The model refers to the internal secretion system of humans and it has four kinds of the hormone parameters to use to adjust various internal conditions such as motor output, cooling fan output and sensor gain. We surveyed 126 visitors at '97 International Robot Exhibition held in Tokyo, Japan (Oct. 1997) in order to evaluate psychological impressions of the robot. As a result, the human friendliness of the robot was confirmed and some factors of the human-robot emotional communication were discovered.

In this paper, a comprehensive new concept concerning situations where robots physically contact with human such as tactile contacts and collisions is proposed, which is named physical interference (PIF). In order to realize PIF adaptation with human, it is required for robots to recognize PIF with human as well as secure human safety. First, information of PIF with human for recognizing PIF in detail is specified. Next, a hybrid PIF adapting hardware system composed of a mechanical passive compliant arm and PIF recognizing covers with viscoelastic materials is proposed and was developed. Finally, in order to evaluate the hardware system, experiments of adapting to PIF with human, using an arm equipped with MIA (mechanical impedance adjuster) and the developed surface covers, were carried out. From the results of the evaluation experiments, the validity of the developed hardware system for basic PIF adaptation with human was confirmed.

This research aims to clarify the behavior intelligence and the human cooperation intelligence of robots by the emotion models which is based on the robot's hardware structure. In this paper human's mental images and language are given consideration as a method for emotional expression. The hypothesis model for the acquisition of the internal expressions of robots and the experimental results using a real autonomous robot are described.

In order for a mobile manipulator to be used in areas such as offices and houses, the mobile platform must be small-sized. In the case of a small-sized platform, the mobile manipulator may fall down when moving at high speed, or executing tasks in the presence of disturbances. Therefore, it is necessary to consider both stabilization and manipulation simultaneously while coordinating vehicle motion and manipulator motion. In this paper, we propose a method for coordinating vehicle motion planning considering manipulator task constraints, and manipulator motion planning considering platform stability. Specifically, first, the optimal problem of vehicle motion is formulated, considering vehicle dynamics, manipulator workspace and system stability. Next, the manipulator motion is derived, considering stability compensation and manipulator configuration. Finally, the effectiveness of this method is demonstrated by simulation.

The objective of this study is to investigate design and control strategies for realizing human-robot symbiosis, and to develop a human symbiotic humanoid robot, Hadaly-2, which can communicate and collaborate with human. In this paper, mechanism design methodologies, specifications, and control strategies of a mobile manipulation system of the Hadaly-2 will be described. First, mechanism design concepts of the manipulation system for the human symbiotic robot are proposed. Next, two force controlled anthropomorphic manipulators (WAM-10R, and L) and a body-vehicle mechanism are developed in consideration of the design concepts. Then, the communication and collaboration abilities of Hadaly-2 are evaluated by means of several behaviors, such as gesture motion, shaking hands with human, and block carrying tasks. From the results of evaluation experiments, it is confirmed that the Hadaly-2 can realize efficient interaction and collaboration with human.

In order to realise human cooperative machinery, the machine's ability of understanding human intention becomes an important issue. This paper proposes a method of extracting human intention from the shape of Work Triangle, which consists of head and hands of the human. A system, which samples the Work Triangle during assembly work by vision sensors, was constructed. With the system, the patterns of the Work Triangle of different skilled subjects were sampled. A Kohonen neural network was used for categorising the patterns of the Work Triangle. The categorised patterns between skilled and unskilled subjects were examined. The patterns which only appear in unskilled subjects, were extracted as states of "Needing support". A physical support prototype system, which carries out supports by the "Needing support" state, was constructed.

This study discusses the communication between autonomous robots and humans through the development of a robot which has an emotion model. The model refers to the internal secretion system of humans, and it has four kinds of hormone parameters to be used for adjusting various internal conditions such as motor output cooling fan output and sensor gain. We surveyed 126 visitors at '97 International Robot Exhibition held in Tokyo, Japan, in order to evaluate psychological impressions of the robot. As a result, the human friendliness of the robot was confirmed and some factors of the human-robot emotional communication were discovered.

Human KANSEI is based on tacit knowledge and experience has received a great deal of attention from researchers. "KANSEI" is a Japanese word that is similar in meaning to sensibility. In the engineering field, KANSEI has been researched in regard to facial expression, gestures and voice (F. Hara and K. Tanaka, 1997), and has also be used to design keyboard switches (Kajiro Watanabe and Hiroaki Kosaka, 1995). KANSEI in music has also been researched from different perspectives. KANSEI does affect human motion; however, its effects on human motion have not yet been clarified. Therefore, the goal of this study is to clarify the effects of human KANSEI on human motion. The bowing in violin playing was selected as an example. The relationship between the timbre terms that are KANSEI data and the bowing parameters that are motion parameters were analyzed via factor analysis. An algorithm to convert timbre terms into bowing parameters was constructed by using the results from factor analysis, and violin sounds were produced by using this algorithm.

This study aims to realize passive impedance control of the robot joint by mechanical elements (MIA), and to develop an anthropomorphic manipulator using this mechanism. The mechanism has the advantage of conventional method of the force control in realizing high compliance. The manipulator which employs this joint mechanism is appropriate for the human-robot cooperative tasks, because each joint can realize safety motion. This paper presents a development of the anthropomorphic manipulator (seven DOF MIA ARM) which consists of shoulder, elbow and wrist. This paper also describes a dynamic model of multiple-DOF MIA and an experimental evaluation of the seven DOF MIA ARM by means of trajectory control. The experimental results show that the seven DOF MIA ARM can realize high performance in motion control with consideration to the effect of dynamics, selfweight and damping.

In order for a mobile manipulator to move stably (not overturn) and execute the given motions of the end-effect and the vehicle simultaneously, a manipulator must have redundancy. By using this redundancy, it is possible to perform task at an optimal manipulation configuration when the robot is stable, and recovering the system's stability when the robot is unstable. The ability to recover stability by this manipulator compensation motion is limited. Thus in order to ensure the feasibility of stability compensation, the task plan or vehicle motion must be within this ability. In this paper, first the concept of stability compensation range by static posture change is proposed. Then, within the stability compensation range, the compensation motion of a redundant manipulator considering the manipulation configuration and the system stability is derived, given the motions of the end-effector and the vehicle. Finally, the effectiveness of this method is illustrated by simulation experiments.

To plan behavior procedures, it is necessary for an agent to have a world model concerning the temporal sequences information. In this paper, a temporal information learning algorithm is proposed with a three layer neural network implementing the "effectiveness of simulation accumulation" algorithm. This algorithm can construct a "behavior automaton" in the neural network. From the results of some learning experiments using a mobile robot simulation, the generated automaton expresses the complexity of the simulation environments. The robot agent acquires a behavior automaton for obstacle avoidance behavior which is influenced by the simulation environment.

The objective of our study is to realize passive impedance of the robot joint by mechanical elements (MIA), and to develop an anthropomorphic multiple DOF manipulator using this mechanism. This mechanism has the advantage of a conventional method of force control in realizing high compliance. This paper presents the development of the 4-DOF MIA ARM which is an upper arm of the manipulator. This paper also describes the experimental evaluation of the 4-DOF MIA manipulator by means of step responses and circular trajectory. The experimental results show that the 4-DOF MIA ARM has high performance not only in force control but also in motion control.

The objective of this work is to develop the technology for human-machine communication through the research of the emergence of mind in mechanical systems. In this paper, the hypothesis about the emergence of mind is proposed. First, a system chart expressing the human brain information processing and the development of an autonomous mobile robot "WAMOEBA-IR" (Waseda artificial mind on emotion base) are described. The conception of the WAMOEBA-IR design is that robots should have a self-presentation evaluation function. Further more, the method to evaluate the whole system is described from the viewpoint of the animal psychology. As a result of the experiments, WAMOEBA-IR showed specific emotional reactions with color appearances to some situations. WAMOEBA-IR has the sense of values about colors and sounds based on self-preservation as the first step of the emergence of mind.

It is desired that a vehicle-mounted mobile manipulator can move with stability and can operate tasks in various environments in the presence of disturbances. In this paper, a mobile manipulator cooperative motion planning algorithm is proposed, consisting of a rough motion planning and a local motion modification. As a step to realize the cooperative motion, the vehicle motion planning is discussed, given the end-effector trajectory. First, the vehicle path is planned. Then the optimal problem of determining the passing time of the vehicle along the planned path is formulated, considering the vehicle acceleration, the manipulator workspace and the system stability. Using a gradient projection method, the vehicle motion is derived. Finally, the effectiveness of this method is illustrated by simulation.

The object of our work is to realize passive impedance on the joints of robot fingers by mechanical elements, as leaf springs and brakes, and to develop robot fingers using the new mechanism to grasp objects softly. This paper presents a new joint mechanism, named "mechanical impedance adjuster", and an impedance control method of the robot finger joint by this mechanism. The effectiveness of this method is shown by the experiment on a 1-DOF finger model.

The stability of a vehicle-mounted mobile manipulator has a close relation with the vehicle's motion, the manipulator's posture and motion and the end-effector's force. The purpose of this study is to derive the cooperative motions of the vehicle and the manipulator for a stabilization which is compatible with task operation so that the mobile manipulator can successfully accomplish tasks in environments with various disturbances. The authors have already proposed the stability concepts based on the ZMP criterion to discuss the stabilization and the task operation, and have presented the method of ZMP moved path by a stability potential field to maintain the stability for a mobile manipulator. In this paper, based on the above-mentioned considerations, the manipulator compensatory motion is discussed for stabilizing the mobile manipulator while the vehicle is moving along a given motion.

In order to realize constraint tasks by a manipulator, it is effective to adjust the joint impedance to the appropriate value. Most of the previous studies use the active force control method that uses information from force sensors. Using this method, the performances are limited by the responses of the servo systems, the non-linear characteristics of the force sensors and the dynamics of the manipulator. The object of this study is to adjust the joint impedance of the manipulator to an ideal degree by a mechanism which consists of a spring and a damper. In a previous study, the authors proposed a compliance adjustment method using the spring mechanism, the structure and control of the pseudo-damper system, and simple control algorithms for the coordinated system. In this paper, the authors discuss the evaluation of the effect of these mechanisms and the control method using the 1-DOF Arm Model which was newly developed as a base model to build a multiple degrees of freedom manipulator.

Many future applications of robotic systems will require that manipulators perform operations while being carried by moving vehicles. However, such a vehicle mounted mobile manipulator can be unstable or even tip over. Previous work on stability control hardly considered the dynamics and environmental disturbances. The stability of a mobile manipulator has a close relation with the vehicle motion, manipulator motion and posture, and end-effector force. To evaluate the stability for a mobile manipulator, the concepts about stability, such as the stability degree and the valid stable region based on the zero moment point (ZMP) criterion have already been proposed. In this paper, as a control scheme for maintaining or recovering stability, the method of ZMP path planning by a stability potential field is presented, in which the concepts of the goal state and prohibitive state of stability are outlined. A motion planning algorithm is then formulated, which controls the manipulator in order to maintain the stability of the whole system while the vehicle is moving along a given trajectory.>

Many future applications of robotic systems will require manipulators to operate from moving vehicles. However, vehicle-mounted mobile manipulators might be unstable and even tip over. The authors assume that the stability of such a mobile manipulator has a close relationship with the vehicle's motion, the manipulator's posture and motion, and the endpoint's force. They present the concepts of the stability degree and the valid stable region based on the ZMP zero moment point criterion, which can be used as effective stability criteria in controlling mobile manipulators. Finally, the concepts are illustrated by computer simulation.

Advanced robots will have to not only have 'hard' functions but also have 'soft' functions. Therefore, the purpose of this study is to realize 'soft' functions of robots such as dexterity, speediness and intelligence by the development of an anthropomorphic intelligent robot playing keyboard instrument. This paper describes the development of keyboard playing robot WABOT-2(WAseda roBOT-2) with a focus on the mechanisms of arm-and-hand which has 21 degrees of freedom in total, their hierarchically structured control computer system, the information processing method at the high level computer and finger-arm coordination control which realizes the autonomous movement of WABOT-2.