Fiducial markers enable reliable service robot control. In human-robot coexistence environments, efficient placement of square or circular markers can be challenging due to limited space. In this study, we developed a world-first, elongated fiducial marker, capable of high-accuracy 6-DoF measurements, designed to be installable in tight spaces. We introduced two types of lenticular angle gauges to enhance pose estimation and developed new marker patterns and measurement algorithms to maintain recognition distance and accuracy. The proposed marker achieved a measurement accuracy of 0.1% position error and 0.5° orientation error. This technology will enhance the practicality and applicability of fiducial markers, contributing to the creation of robot-friendly space for future service robots.

As people age, walking can become difficult. This difficulty in walking can lead to further physical decline and eventually result in the need for caregiving. To prevent this, wearable robots supporting walking rehabilitation have been developed. In this study, we aim to develop a modular robot wear that can be customized according to the user’s needs. The modular robot wear consists of motor modules, sensor modules, and a processor. Motor modules can be attached to any part of the body that requires assistance, enabling support not only in the sagittal plane but also in the frontal plane. By calculating the relative movement of the center of gravity based on the information from the acceleration sensors in the sensor modules, commands to the motor modules are generated. This allows for assistance tailored to the user’s walking pattern. Through verification experiments of the robot wear’s operation, we confirmed its ability to provide support according to changes in walking patterns. However, while the robot wear can induce changes in walking patterns, there are challenges regarding the output it provides to the wearer.

Musculoskeletal diseases such as low back pain may cause physical disabilities and make it difficult to continue working. From this point of view, logging of workers' conditions in the working environment is an important issue. So far, limited information acquisition such as smart watches has been realized, but comprehensive information has not been realized with a single device. Therefore, in this study, we estimated the wearer's state using a device called THINKLET, which can acquire image information and inertial information at the same time. By acquiring the pose of the hand from the image information, we were able to estimate the work. More detailed information can be obtained by calculating the position at the same time. By integrating image information with inertial information, it becomes possible to measure the position and posture of the wearer in the working environment. By complementing each other's sensor information, it is possible to obtain robust and accurate self-location. In this research, we developed these basic technologies and evaluated their performance.

To extend the coverage of people able to receive high-quality rehabilitation, remote rehabilitation is required in addition to traditional face-to-face rehabilitation. Although remote rehabilitation using video conferencing systems has been realized to date, communication through physical sensations such as detailed patient motoring information and manual instructions from the therapist has not yet been realized. Therefore, the ultimate goal of this study was to develop multimodal wearable sensor system to support remote rehabilitation with a somatosensory system. To this end, we conducted a basic study of sensing technology in multimodal wear. Multiple strain sensors were attached to the shoulder to digitize the detailed behavior of the shoulder girdle. It was confirmed that the movement of the scapula can be acquired by this strain sensor. Furthermore, it was confirmed that the combination of strain sensors and an inertia measurement unit can be applied for the motion estimation of the entire upper limb.

Technologies for estimating self-position and orientation are important for both humans and robots. These technologies allow robots to perform tasks such as carrying objects and allow people to reach their destinations. Although self-position estimation technologies using GPS and laser rangefinders have been developed, few methods can be used by both humans and robots. Therefore, we developed a method that can estimate three-dimensional position and orientation using visual markers and an inertia measurement unit (IMU). Self-position can be measured with high accuracy by using a visual marker and monocular camera, but such measurement data is discrete and sparse. In contrast, an IMU can continuously measure acceleration data, but data obtained from an acceleration sensor are double-integrated, which increases position error. By combining visual marker and IMU information, position error calculations based on the acceleration sensor can be corrected, and the movement path of the object can be estimated. In demonstration experiments, the proposed method accurately estimates the three-dimensional movement distance when a person walks about 13 m, with an average error of about 40.3mm.

Lenticular angle gauge (LEAG) is a planar pattern that visualizes relative attitude by the position of the black line, which moves according to the viewing angle. The authors developed a new LEAG in which the direction of movement of the black line is 90◦different from the previous LEAG, and used it to develop a non-square high-accuracy fiducial marker. The new marker realized accurate pose estimation with a position error of 0.15% of the distance and an attitude error of 0.5°. This research contributes to the development of high-accuracy markers with a more flexible design. This paper describes the principle and behavior of the new LEAG, the design of the new fiducial marker, and the pose estimation algorithm, followed by the results of performance verification experiments.

Globalization and the effect of recent infectious disease are changing the remote conversations as a new normal in business meetings, social provision and casual chatting. Previous research show that the remote conversations have a difficulty on showing the presence or attention to the other interlocutors, especially in the situation where the majority of interlocutor share the same place and one or a few interlocutor participate from difference place. This paper proposed "Parappa", a remote video conversation system, for those unbalanced condition by utilizing both physical and virtual approaches to share the nonverbal behaviors of remotely-participating interlocutor. The proposed system is constructed with a rotatable screen which projects the life-size avatar of remote interlocutor. The rotation of the screen represents the body posture and the projected avatar show the gesture. The results of preliminary analyses of eye gaze activities and frequency of the screen rotation during conversation suggests the possibilities that proposed system enables to show the presence of remote interlocutors.

Automatic control of mobile robots and robot arms requires techniques for estimating the position and orientation of objects with high accuracy and robustness. Although methods using markers or machine learning have been developed, general-purpose and highly accurate estimations have not been realized. Our team developed a high-accuracy visual marker “LentiMark” for high-accuracy estimations of position and posture, but data are lost when the camera cannot obtain marker images. We therefore developed the Marker-IMU system for integrating visual markers with an inertia measurement unit (IMU). When cameras cannot acquire the image of a visual marker, any missing data are restored from IMU data. However, when calculating positions from acceleration sensor values, sensor error increases estimation error. Therefore, we developed a method for error correction using measurements from before and after the missing data. Evaluation experiments confirm that missing data can be estimated using the proposed method.

The recent demographic trend across developed nations shows a dramatic increase in the aging population, fallen fertility rates and a shortage of caregivers. Robotic solutions to clothing assistance can significantly improve the Activity of Daily Living (ADL) for the elderly and disabled. We have developed a clothing assistance robot using dual arms and conducted many successful demonstrations with healthy people. It was, however, impossible to systematically evaluate its performance because human arms are not visible due to occlusion from a shirt and robot during dressing. To address this problem, we propose to use another robot, Whole-Body Robotic Simulator of the Elderly that can mimic the posture and movement of the elderly persons during the dressing task. The dressing task is accomplished by utilizing Dynamic Movement Primitives (DMP) wherein the control points of DMP are determined by applying forward kinematics on the robotic simulator. The experimental results show the plausibility of our approach.

This paper describes the development of a whole-body robotic simulator of an elderly person for evaluating robotics devices for nursing care. To improve the quality of life of the elderly persons, physical assistance such as transfer, movement, and bathroom assistance is important. It is also important to reduce the workload of caregivers in an aging society. In recent years, assistive robotic devices for nursing care have been developed and commercialized for such purposes. However, such devices have not become popular in the care facilities yet. One of the reasons is that it is still difficult to evaluate the effects of the devices on the care receivers and caregivers. In particular, it is necessary to quantitatively evaluate the effect of the devices on the human body from the viewpoint of safety and comfort. We have developed a whole-body robotic system to simulate the pose and motion of the elderly persons. The purpose of this system is to realize quantitative physical evaluation of robotics devices for nursing care of the human body. The experimental results of the preliminary evaluation of assistive robotic devices are also presented.

Some stroke patients are affected by hemiplegia, which is the complete paralysis of half of the body, and need to train in upper limb motions with high frequency to improve their quality of life. Therefore, the authors developed a portable robot with a main processor, actuators, motor drivers, a force sensor, and a touch display to assist such high-frequency training at home. To recover the motor function of the upper limbs by voluntary movement, this robot instructs the joint motion based on the force input. Therefore, this study proposes a force control algorithm for the omni-wheel mechanism and a method that estimates the odometry of the omni-wheel mechanism based on the circuit equation of the DC motor without rotary encoders. Moreover, a method that estimates integrated electromyograph patterns based on the force and movement of the robot using a multiple linear regression model is proposed for the evaluation of co-operative movement. The effectiveness of these proposed methods were confirmed by evaluation experiments.

Visual markers are useful tools that aid autonomous robots in object recognition and pose measurement; however, conventional markers have two fundamental problems in orientation estimation: (1) inaccurate estimation in the frontal direction and (2) pose ambiguity. We previously developed a visual marker “LentiMark” that solves problem (1). In this paper, we propose a solution to problem (2) by improving LentiMark. For this purpose, we use two wavelike slices that change two-tone patterns according to the angle of visual line. The new marker enables stable and robust pose estimation without pose ambiguity. We solve the two biggest problems of visual markers in the framework of small planar markers. We verify the efficiency of our method through experiments.

This paper introduces a method of force control for a jumping biped robot to correctly control the ground reaction force during continuous jumping. Assuming dynamic motion, such as jumping and running, the attitude of the robot depends on the dynamics in response to the ground reaction force. Therefore, controlling the ground reaction force is necessary for stabilizing the robot's motion. However, control of the ground reaction force involves a fundamental problem: feedback control does not work properly against the impact force owing to limitations in the control bandwidth. This study introduces a method for stiffness ellipse control that utilizes three antagonist pairs of six pneumatic artificial muscles. When the tip of the robot makes contact with the ground, an external force is induced in the direction of lower stiffness. By utilizing this property of the stiffness ellipse, it is possible to control the ground reaction force by feedforward control, which is not dependent on the control bandwidth. Based on this idea, impact force control at landing and jumping force control at takeoff were proposed to correctly control the ground reaction force during the continuous jumping of the robot. The results of several experiments conducted convince us that the relationship between the ground reaction force and the stiffness ellipse is almost linear, and that the ground reaction force can be controlled with high reproducibility by adjusting the stiffness ellipse.

Computer vision plays an important role in various applications. However, the accuracy of computer vision is often degraded by specular reflection. To solve this problem, a system utilizing a high-speed camera and a strobe for removing specular reflection has been shown to be effective. This method utilizes a principle of estimation using images with luminances varied by the flickering of a strobe. However, the method generates noise due to the difference in position of specular reflection among images in video imaging. In this paper, the authors propose a method combining optical flow with the above-mentioned method. The proposed method enables the removal of specular reflection for video imaging without generating noise. Experimental results and comparison with other methods of specular reflection removal show the effectiveness of the proposed method.

This paper introduces the control method for the impact force based on feed-forward control considering the stiffness ellipse of a biped robot. Assuming dynamic motion, such as jumping and running, the jumping direction depends on the dynamics in response to the ground reaction force. Therefore, it is necessary to control the ground reaction force in order to stabilize the system. However, control of the ground reaction force involves a fundamental problem: feedback control does not work properly against the impact force owing to the control bandwidth limitation. This study introduces a method for the stiffness ellipse control utilizing three antagonist pairs of six pneumatic artificial muscles. When the tip of the biped robot makes contact with the ground, an external force is induced in the direction that has lower stiffness. By utilizing this property of the stiffness ellipse, it is possible to control the ground reaction force by feed-forward control, which is not dependent on the control bandwidth. The results of several experiments convince us that the relationship between the ground reaction force and the stiffness ellipse is almost linear, and the ground reaction force can be controlled with high reproducibility by adjusting the stiffness ellipse. It is possible to stabilize the system during dynamic movements because this method can control the impact force.

Several exoskeleton robots have been developed for use in rehabilitation. These robots help users to execute transitive motion rehabilitation with power assist. We propose a new robotics system that executes automatic movement training using target motion instruction. The proposed robot is wearable with a flexible body like clothes, and is actuated using a wire-driven mechanism. The robotics system determines the desired posture based on the operator's posture. The robot executes a wire-driven force display based on the operator's pose and the target pose. We developed a prototype of this wearable robot, and verified the force display using wire-driven mechanism. Moreover, we proposed the multi-joint coordinated motion method, and evaluated the mechanical contribution of the proposed method quantitatively using computer simulations. The simulation results confirm that this robot can instruct the coordinated motion.

To help with care work and rescue operations, it is necessary for humanoid robots to have the ability to transport humans steadily and gently. In this research we consider "piggyback" motions for transporting humans. Most people can perform this motion, allowing us to measure and analyze piggyback motions of human subjects using tactile sensing and whole body movements to design whole body contact control. One interesting result of this investigation is that frictional forces are skillfully controlled by the carrier. In the first experiment, we study a "knack" that allows the carrier to reposition the rider. In the second experiment we verify the effectiveness of the knack in achieving the repositioning result. We also studied the principle of the repositioning motion, and found that it is similar in many ways to a jumping motion. Then we confirmed the validity of our modeling assumptions using a dynamical simulator.