The application of humanoid robots is gaining attention as a solution to the caregiver shortage caused by an aging population. In this study, we automated the process of changing a patient’s body position from supine to lateral, a key aspect of nursing care. We proposed a method for recognizing a patient’s 3D posture at close range by simultaneously using a fisheye camera and a RGBD camera. For robot motion, we developed a trajectory generation method that adapts to the patient’s posture by converting measurement data into a mathematical model. Additionally, we identified the optimal timing for the movement of robot arms with minimal physical strain by considering human body dynamics. In all experiments using mannequins of different body shapes, the robot successfully reached the target joint and lifted one side of the body by more than 48 degrees. Future work will include detection of joints unaffected by body bulges and application the method to other repositioning movements.

Caregiving is a vital role for domestic robots, especially the repositioning care has immense societal value, critically improving the health and quality of life of individuals with limited mobility. However, repositioning task is a challenging area of research, as it requires robots to adapt their motions while interacting flexibly with patients. The task involves several key challenges: (1) applying appropriate force to specific target areas; (2) performing multiple actions seamlessly, each requiring different force application policies; and (3) motion adaptation under uncertain positional conditions. To address these, we propose a deep neural network (DNN)-based architecture utilizing proprioceptive and visual attention mechanisms, along with impedance control to regulate the robot's movements. Using the dual-arm humanoid robot Dry-AIREC, the proposed model successfully generated motions to insert the robot's hand between the bed and a mannequin's back without applying excessive force, and it supported the transition from a supine to a lifted-up position. The project page is here: https://sites.google.com/view/caregiving-robot-airec/repositioning

Dialogue generation plays a key role in responding to user and providing transparency in motion execution in human-robot interaction. As motion planning is generally performed in terms of discrete motions, previous studies have focused on dialogue generation at the boundaries between motions. Recently, continuous motion generation was proposed to enable adapting actions to unique characteristics of the objects for domestic robots. Since a continuous motion generally involves physical and nonphysical phases, providing dialogues when the phase changes is crucial for decreasing users’ anxiety and guaranteeing safety. However, continuous motions lack clear phase boundaries, posing challenges for dialogue generation between phases. For this problem, we segmented continuous motions into discrete phases, and constructed a system to enable the robot to autonomously generate dialogues by detecting phase boundaries. To do so, we built phase estimation models using robot sensor data and designed system modules. Specifically, we collected data in the scenario of a robot assisting to lift the user up from bed. We segmented the continuous motion into three phases based on the user’s posture and whether the robot applied force to the human. The best phase estimation model achieved a macro F1 score of 0.894, demonstrating that phases can be estimated from sensor data. The evaluation results of our system demonstrated that the system accurately detects phase boundaries and generates appropriate dialogues corresponding to phases. Furthermore, we conducted simulations with a user agent to investigate system behaviors when the phase estimation was incorrect. The results suggested that explicitly stating the phase is important for avoiding misunderstandings and safety issues.

Mixed reality (MR) technologies have a high potential to enhance obstacle negotiation training beyond the capabilities of existing physical systems. Despite such potential, the feasibility of using MR for obstacle negotiation on typical training treadmill systems and its effects on obstacle negotiation performance remains largely unknown. This research bridges this gap by developing an MR obstacle negotiation training system deployed on a treadmill, and implementing two MR systems with a video see-through (VST) and an optical see-through (OST) Head Mounted Displays (HMDs). We investigated the obstacle negotiation performance with virtual and real obstacles. The main outcomes show that the VST MR system significantly changed the parameters of the leading foot in cases of Box obstacle (approximately 22 cm to 30 cm for stepping over 7cm-box), which we believe was mainly attributed to the latency difference between the HMDs. In the condition of OST MR HMD, users tended to not lift their trailing foot for virtual obstacles (approximately 30 cm to 25 cm for stepping over 7cm-box). Our findings indicate that the low-latency visual contact with the world and the user's body is a critical factor for visuo-motor integration to elicit obstacle negotiation.

To promote rapid recovery and quality of life after a musculoskeletal disorder, rehabilitation exercises that are suitable for each individual's physical condition are important. In cases of disuse muscle atrophy of the quadriceps, inappropriate training can cause injury. Although resistance-training robotic systems have been developed and could adjust resistance load, a systematic detection method with appropriate force strength for automatic adjustment for each individual has not yet been established. In the current study, we developed an electromyogram (EMG) based method that determines the minimum effective resistance load for muscle growth. Using an integrated EMG (IEMG) model of incremental resistance load focused, we constructed a method to determine the minimum effective resistance load with logarithmic functions. The feasibility of our method was tested with a slow training protocol using a wire-driven leg extension training robot to measure the relationship between IEMG and resistance load by applying the incremental resistance load. The proposed model was found to be suitable for six young and four elderly subjects with different levels of muscle mass, and the load derived for each person was shown to induce effectively acute thigh circumference expansion, which is a factor leading to future muscle hypertrophy.

A treadmill system has a large potential to provide humans with an augmented walking experience in real-life without a spatial limitation. However, a treadmill gait is different from walking on level ground. In previous studies, the adaptive belt speed control of a treadmill was developed to achieve a self-paced walking for making the users' treadmill gait similar to their level ground gait. Such studies have focused on steady-state walking and regulating the user's position on the treadmill. A normal gait can be divided into an acceleration state after gait initiation, a steady state, and a deceleration state for stopping. The objective of this study is to develop a treadmill system with a wire tension application enabling a human to experience a similar gait to a level ground gait during the transition phase from an acceleration state to a steady state. We developed a treadmill 4 m long × 1 m wide. To allow a user to move on the treadmill during the gait acceleration phase, an insensitive zone where a user can move without the treadmill belt drive was set. In addition, the treadmill was equipped with a wire traction system to apply a traction force canceling the effect of the belt floor acceleration of the treadmill when the belt speed of the treadmill changes. Through an experiment with six participants, the proposed treadmill system allowed the users to move in an acceleration state with the same head acceleration pattern as with level ground walking and cancel the inertial effect with the wire traction, which enabled the users to transition to a steady state from an acceleration state.

Vision plays an important role in motion planning for mobile robots which coexist with humans. Because a method predicting a pedestrian path with a camera has a trade-off relationship between the calculation speed and accuracy, such a path prediction method is not good at instantaneously detecting multiple people at a distance. In this study, we thus present a method with visual recognition and prediction of transition of human action states to assess the risk of collision for selecting the avoidance target. The proposed system calculates the risk assessment score based on recognition of human body direction, human walking patterns with an object, and face orientation as well as prediction of transition of human action states. First, we investigated the validation of each recognition model, and we confirmed that the proposed system can recognize and predict human actions with high accuracy ahead of 3 m. Then, we compared the risk assessment score with video interviews to ask a human whom a mobile robot should pay attention to, and we found that the proposed system could capture the features of human states that people pay attention to when avoiding collision with other people from vision.

Adaptive assistance of gait training robots has been determined to improve gait performance through motion assistance. An important control role during walking is to avoid tripping by controlling minimum toe clearance (MTC), which is an indicator of tripping risk, to avoid its decrease among gait cycles. No conventional gait training robots can adjust assistance timing based on MTC. In this paper, we propose a system that applies force intermittently based on the MTC prediction algorithm to encourage people to avoid lowering the MTC. This prediction algorithm is based on a radial basis function network, the input data of which include the angles, angular velocities, and angular accelerations of the hip, knee, and ankle joints in the sagittal and coronal planes at toe-off. The cable-driven system that can switch between assistance and non-assistance modes applies force when the predicted MTC is lower than the mean value. Nine participants were asked to walk on a treadmill, and we tested the effect of the system. The MTC data before, during, and after the assistance phase were analyzed for 120 s. The results showed that the minimum and first quartile values of MTC could be increased after the assistance phase.

Although conventional service providers are independent from each other when attending most of the population, demanded services are changing along with the social structure. Especially in the case of Taiwan, the number of co-working families has been increasing, and self-employed households occupy a large proportion of all working forms. Due to their diverse lifestyles and work styles, services that are suitable for personal objectives and that optimize the use of time are required. To meet this demand, it is important to connect people and city facilities to make it easier to provide suitable services. Based on those backgrounds, an innovated personal service platform in Taiwan is proposed, focusing on three factors, including time, place and personal information to connect people and city service facilities. Among various kinds of services, we targets services purchased in cities such as sales, mobility services, health services, government services and so on. It aims to link these services flexibly and dynamically to achieve personal objectives according to each situation. And, it can provide suitable services for a variety of every-day living situations. With this system, people can increase satisfaction and free time, improving life quality while making the economy more dynamic.

To ensure that humans and robots can safely coexist, the ability to recognize human behavior is a prerequisite for robots and a fundamental technical challenge for researchers. Current research can only recognize relatively simple cases of human behavior due to the lack of enough data and archetypally designed experiments. Our study elucidates human behavior in a systematic manner by observing the behavior of human subjects under more complex situations where they are surrounded by other people or objects to address the challenge. We focus on the following common situation that people pass each other through a narrow passage. We constructed a motion capture room with a narrow passage environment and measured the motion of human subjects performing different tasks. In addition, during the narrow passage experiment, we made subjects hold different daily necessaries (such as a backpack) to observe influences on human behavior. Our study found that passing and avoidance behavior exhibited by each of our subjects were significantly influenced by what kind of daily necessaries subjects carry. This research provides novel findings on human behavior in complex environments: in the case where subjects holding a handbag (a type of daily necessaries that stays next to one's body), they showed the tendency to be affected by the other subject and move more dynamically compared to the subject without anything or with other daily necessaries; in the case where subjects carrying a backpack (a type of daily necessaries on one's back) and looking at a smartphone, they also showed the tendency of being affected by others, but their motion is restricted compared to the subject without anything.

Since the deceleration of the markets of infrastructures in developed economies occurred, the growing demand for infrastructure development in semi-developed countries has become more outstanding. Semi-developed countries such as Thailand aims to become a developed country. However, it is stuck in 'semi-developed country trap'. To break through this, Thai people need local industries with high-added value. Therefore, we proposed an inversed innovative strategy focusing on railway industry. At first, how railway industry was established so far was investigated for finding keys to establish it. Next, we had field trips so that we can find real needs from Thai people. Actually, many countries compete in Thai railway markets. Thus, a comparison between our proposal and others were made. Our proposal for establishing railway industry includes some steps to get local industry focusing on Thai situation. Finally, it also presents ASEAN market as a future plan after they acquire their local industries.

Elderly people are at risk of falling because of their low toe clearance. Gait training to improve toe clearance could be instrumental in avoiding tripping. We propose using a gait-training robot that applies torque during the pre-swing phase to achieve this goal. It is still possible to revert to their original trajectory after the training, however, depending on the magnitude of the applied torque. We investigated the relation between the magnitude of the applied torque and the change in toe clearance before and after application of torque. We developed a robot and carried out an experiment in which a motor pulls a string embedded on the robotic frame worn by the participants, thereby applying torque during the pre-swing phase. The experimental task included walking on a treadmill for 50 s. We applied torque to the knee during the pre-swing phase for 20 s. The phases before and after applying torque were 15-s normal walking phases with no interference from the robot. We compared toe clearance during the phases before and after applying torque. We found that the toe clearance increased after applying a torque of 8 Nm. We were thus able to verify the influence of torque on toe clearance.