Facial expression detection (FED) holds significant application value in affective computing, yet existing methods predominantly employ cascaded “detection + recognition” schemes that suffer from the absence of end-to-end optimization, poor adaptability of generic detectors, and neglect of facial geometric structures. This paper proposes a dual-stream end-to-end FED framework that simultaneously achieves face localization, expression classification, and 37-point keypoint regression within a unified network through deep coupling of the face branch and keypoint branch. Three cross-branch fusion modules are designed: Keypoint-Guided Multi-Scale Deformable Attention (KG-MSDA) enables high-level features to focus on expression-salient regions, Semantic-Guided Structure Fusion (SGSF) enhances keypoint prediction stability, and Keypoint-to-Face Local Structure Guided Modulation (K2F-LS) leverages global structural configurations to improve bounding box localization. Experiments on the RAF-DB [1] dataset demonstrate that our method achieves 80.65% mAP, a 2.01 percentage point improvement over the baseline, with particularly outstanding performance on expression categories such as Anger and Surprise. Ablation studies validate the effectiveness of each module, while visualization analysis reveals that the network learns expression-related differentiated attention patterns. Our method significantly improves detection accuracy and robustness while maintaining a lightweight design, providing new insights for end-to-end modeling of FED.

The effective fusion of infrared and visible images could enhance environment perception during robot rescue mission by combining complementary information from both sensors. However, most existing fusion methods are developed for images captured under normal conditions, which limits their performance in real-world rescue scenarios where images often suffer from diverse degradation such as haze, low-light, noise, low-contrast, and so on. To address this challenge, we propose MT-Fusion, a novel deep learning workflow based on multi-task learning (MTL) that targets multiple specific degradation scenarios. MT-Fusion incorporates specific encoder modules for processing various degraded images, a degradation attention mechanism for fusion, and a shared decoder for image reconstruction. Extensive experiments demonstrate that our proposed specific scenario-guided image fusion strategy has obvious advantages in robot perception in the image fusion performance and degradation treatment. For the inference stage, we propose a selector that could automatically categorize the degradation of input and activate the specific encoder. The MT-Fusion also provides a more practical solution for enhancing robot rescue operations in challenging environments.

In this paper, we consider the case of carrying an object with a quadruped-wheeled robot and examine and verify the angle planning method for the robot’s base link (load section) that suppresses the relative tilt of the carried object. It is important to suppress the tilt of the carried object relative to the base link. We formulated the angle of the base link at which the carried object does not begin to tilt relative to the base link. We examined an angle planning method for the base link that would suppress the relative tilting of the carried object. Then, we verified the method using a simulation and a quadruped-wheeled robot: MELEW-3 (Meiji Leg-Wheeled Robot - No.3). As a result, both in the simulation and on the actual robot, we succeeded in carrying the object without turning over, and the base link was tilted based on the planning method.

Retinex theory, which treats an image as a composition of illuminance and reflectance, has made significant progress in low-light image enhancement. Previous methods attempt to refine the impractical Retinex theory by introducing deviations in estimated illumination and reflectance to develop more practical and robust enhancement techniques. However, the fact that state-of-the-art approaches still produce inferior results suggests that some form of corruption may be over-looked. In this paper, we propose a novel Complete Corruption-Aware Retinex Framework (CCRF), which not only considers corruption in low-light imaging—such as high ISO or long exposure settings—but, more importantly, also accounts for corruption induced by the enhancement method itself. Guided by this framework, we propose a Robust Corruption-Aware Loss (RCL) that enables the model to be robust under extreme darkness and complex light-object interactions. Additionally, we propose a Light-Up Map Denoising (LMD) module, which further eliminates model-induced perturbations. With these two plug-and-play modules, downstream tasks (e.g., low-light object detection) can benefit significantly. Extensive experiments demonstrate that our methods can be seamlessly integrated into state-of-the-art approaches, resulting in significant performance improvements over these methods. Code will be available at github.com/eafi/ccrf.

Weighted context-free grammar (WCFG) is a quantitative extension of context-free grammar (CFG). It is known that unambiguous weighted automata (WA), finitely-ambiguous WA, polynomially-ambiguous WA and general WA over the tropical semiring have different expressive powers. We prove that there exists a similar ambiguity hierarchy of WCFG over the tropical semiring, using an extended Ogden's lemma. In addition, we prove that each of the classes of finitely-ambiguous, polynomially-ambiguous, and exponentially-ambiguous WCFG can be subdivided into a finer strict hierarchy. We further show that the hierarchy we proved is different from the known ambiguity hierarchy of unweighted CFG.

Biped robots require massive power on each leg while walking, hopping, and running. We have developed a flow-based control system-called hydraulic direct drive system- that can achieve high output while avoiding spatial limitations. To implement the proposed system with simple equipment configuration, a pump and single-rod cylinder are connected in a closed loop. However, because compensation for flow rate is impossible in a completely closed loop, owing to the difference in the pressure receiving area caused by the rod, a passive flow compensation valve is employed. This valve has a simple structure and is easy to implement. Further, an additional sensor is required to detect the open/close state because the valve state will cause an error in flow control. Therefore, we implemented a model in the controller to predict the state of the flow compensation valve and formulated a method of switching from flow control to pressure control according to the predicted state. Experimental results indicate that the error of the joint angle is reduced to less than 1.6 degrees for walking patterns, and stable walking is realized when the system is installed in biped humanoid robots.

Biped robots require substantial amounts of power alternately on each leg while walking, hopping, and running. However, it is difficult to mount high-power large electrical motors in conventional mechanical transmission systems owing to spatial limitations. A hydraulic direct-drive system is proposed in which the size of the motor in each leg can be reduced by sharing the motor outputs between the legs. In this paper, the hydraulic direct-drive system is evaluated in an actual hydraulic system. Velocity followability, excellent energy saving, and virtually perfect position tracking are achieved with the proposed system. The results of performance comparison with a valve-based control system show that energy consumption is controlled and good position-following capability is achieved using the proposed system.

In this paper, we describe approaches to enable a four-limbed robot to get over a step higher than its leg with torso contact. The higher the step becomes, the more difficult it is for legged robots to get over from the viewpoint of stability and kinematic reachability. Torso landing contributes to improving stability and robustness of motion for moving onto high step because of lower center of mass (CoM) and larger support polygon, which is seldomly utilized by previous human-sized legged robots. The approaches in this paper consist of the following two components. As for hardware, spikes are arranged on the bottom of robot’ s body for stable torso landing on a high step. As for motion generation, Sequential Quadratic Programming (SQP) is utilized to generate motion with torso landing to guarantee stability of robots during getting over the high step. From experiments, it is confirmed that the fourlimbed robot WAREC-I succeeded in moving onto a step with the height of 865mm.

For building a disaster response robot, WAREC-l's fully-automated system for manipulating a valve, this paper proposes a method for (1) detecting a valve which is far away from the robot, (2) estimating the position and orientation for grasping the valve by the robot at a closer position. Our methods do not need any prior information about a valve for the above-mentioned detection and estimation for grasping. In addition, our estimation for grasping provides useful information, by which WAREC-I can rotate a valve autonomously. The method (1) uses the RGB image and the point cloud data captured by Multisense SL as the input, and estimate the position and orientation of a valve far away from the robot. The method (2) uses both the RGB and depth images captured by KinectV2 as input and estimate information for grasping the valve. Our experiments are conducted using a real disaster response robot. Our experimental results show the error of the estimation by the (a) two methods are small enough to achieve a fully-automated system for detecting and rotating the valve by WAREC-I.

Biped robots require substantial amounts of power on each leg alternately while walking, hopping, and running. However, it is difficult to adopt large high-power electrical motors in conventional mechanical transmission systems owing to spatial limitations. To address this problem, a hydraulic interlocking drive system that incorporates two hydraulic direct-drive systems is proposed for biped humanoid robots. The hydraulic interlocking drive system connects the two hydraulic direct-drive systems and concentrates the pump output on one side cylinder. The other side cylinder meter-in flow rate is controlled by the meter-out flow rate from the cylinder on which the pump is concentrated. Good position tracking and excellent energy saving are achieved with the proposed system. A performance comparison with a single hydraulic direct-drive system shows that the motor power of the hip pitch joint is reduced by 27.3% for walking patterns. This result shows that the rated output of the motor can be reduced, and smaller and lighter motors can be installed in biped robots.

In this paper, we propose an end-effector for realizing various locomotion modes and simple work of a legged robot. The locomotion modes include climbing a vertical ladder, crawling, and walking. The simple work includes grasping and switching motions required at a disaster site. The developed end-effector has a two-pronged hook shape and two fingers for grasping and working and can be used to perform the locomotion and manipulation tasks described above. The experimental results confirmed that the four-limb robot WAREC-1 (WAseda REsCuer-No. 1) equipped with our proposed end-effector was able to climb a vertical ladder and perform the crawling motion. We also confirmed that the end-effector could grasp and switch five types of objects: a cylinder, cylinder with trigger, T-shaped, disk, and thin plate.

To improve the movement ability of humanoid robots, instead of traditional methods dependent on only power of actuators, there is possibility that utilizing elasticity inspired from collaboration of muscle and tendon of human is effective to achieve high-power movement. In this study, we aimed to realize a jumping motion that accumulates energy more appropriately in spring by combining active joint driving with spring behavior like human tendons and muscles. We proposed a countermovement jump method using the resonance with the leg's active pushing-off movement and leg stiffness. To achieve active pushing-off and joint stiffness, we developed a new joint mechanism using leaf springs and an actuator unit with a worm gear. We then performed experiments to evaluate the effectiveness of the proposed mechanism and methods. Finally, the robot achieved a countermovement jump using active kicking and leg's elasticity.

Humans utilize their torsos and arms while running to compensate for the angular momentum generated by the lower-body movement during the flight phase. To enable this capability in a humanoid robot, the robot should have human-like mass, a center of mass position, and inertial moment of each link. To mimic this characteristic, we developed an angular momentum control method using a humanoid upper body based on human motion. In this method, the angular momentum generated by the movement of the humanoid lower body is calculated, and the torso and arm motions are calculated to compensate for the angular momentum of the lower body. We additionally developed the humanoid upper-body mechanism that mimics the human link length and mass property by using carbon fiber reinforced plastic and a symmetric structure. As a result, the developed humanoid robot could generate almost the same angular momentum as that of human through human-like running motion. Furthermore, when suspended in midair, the humanoid robot produced the angular momentum compensation in the yaw direction.

This paper describes a novel four-limbed robot having high mobility capability in extreme environments. At disaster sites, there are various types of environments where a robot must move such as rough terrain with possibility of collapse, narrow places, stairs, vertical ladders and so forth. In this paper, first we categorized extreme environments based on three indexes: unevenness, narrowness, and inclination. To move in such extreme environments, we proposed a four-limbed robot having various locomotion styles such as bipedal/quadrupedal walking, crawling and ladder climbing. The main contribution of this paper is the concept and hardware design of the four-limbed robot. We developed a prototype of the four-limbed robot having commonly structured limbs. The number of DoF for the whole body is 29, with 7-DoFs in each limb and 1-DoF in the trunk. The robot weight is 110 kg, and the height is 1,290 mm when standing on two legs. The end-effector has hook-like shape. Verification of the prototype robot is conducted through simulations and experiments.

This paper describes an approach that enables the four-limbed robot “WAREC-1” to climb up and down vertical ladders stably. First, the four-limbed robot “WAREC-1” is introduced and dynamic stability conditions in multi-mass model for ladder climbing are proposed as the basis of judging whether a four-limbed robot is stable or not while climbing a vertical ladder. According to the proposed stability conditions, 3 different types of moment will directly affect the stability of the robot on a ladder: gravitational moment, inertial moment and reaction force moment. With the analysis of these 3 kinds of moments and the relationship among them, stability control methods are proposed to maintain stability of the robot on a ladder to the greatest degree and avoid their mutual interference. Combining with the stability conditions and stability control proposed, stable motion planning of climbing up and down a vertical ladder, a motion planning method proposed by the authors that allows independent path and time planning in trajectory planning is also applied to reinforce the efficiency of the stability control. Eventually, results from the simulation and physical robot verify the validity of the proposed control methods.

We developed a four-arm four-crawler advanced disaster response robot called OCTOPUS. Disaster response robots are expected to be capable of both mobility, e.g., entering narrow spaces over very rough unstable ground, and workability, e.g., conducting complex debris-demolition work. However, conventional disaster response robots are specialized in either mobility or workability. Moreover, strategies to independently enhance the capability of crawlers for mobility and arms for workability will increase the robot size and weight. To balance environmental applicability with the mobility and workability, OCTOPUS is equipped with a mutual complementary strategy between its arms and crawlers. The four arms conduct complex tasks while ensuring stabilization when climbing steps. The four crawlers translate rough terrain while avoiding toppling over when conducting demolition work. OCTOPUS is hydraulic driven and teleoperated by two operators. To evaluate the performance of OCTOPUS, we conducted preliminary experiments involving climbing high steps and removing attached objects by using the four arms. The results showed that OCTOPUS completed the two tasks by adequately coordinating its four arms and four crawlers and improvement in operability needs.

Cats have protractible claws to fold their tips to keep them sharp. They protract claws while hunting and pawing on slippery surfaces. Protracted claws by tendons and muscles of toes can help cats anchoring themselves steady while their locomotion trends to slip and releasing the hold while they retract claws intentionally. This research proposes a kind of modularized self-adaptive toe mechanism inspired by cat claws to improve the extremities' contact performance for legged robot. The mechanism is constructed with four-bar linkage actuated by contact reaction force and retracted by applied spring tension. A feasible mechanical design based on several essential parameters is introduced and an integrated Sole-Toe prototype is built for experimental evaluation. Mechanical self-adaption and actual contact performance on specific surface have been evaluated respectively on a biped walking platform and a bench-top mechanical testing.

This paper introduces a motion planning method to generate ladder climbing motion for a four-limbed robot. This method contains the following points: (1) independent planning of path and time in 3 dimensional space for trajectory planning; (2) path length minimization according to given midpoints. In trajectory planning, arc-length parameterization is used to separate path planning and time planning so that they can be done independently. After path is planned, time planning along the planned path can be given freely to meet our requirement, such as speed and acceleration adjustment for the protection of motors, optimization for dynamics analysis, dynamic obstacle avoidance and so on. Results from simulations and experiments authenticate the validity of our motion generation method.

Stereo confidence measures are important functions for global reconstruction methods and some applications of stereo. In this article we evaluate and compare several models of confidence which are defined at the whole disparity range. We propose a new stereo confidence measure to which we call the Histogram Sensor Model (HSM), and show how it is one of the best performing functions overall. We also introduce, for parametric models, a systematic method for estimating their parameters which is shown to lead to better performance when compared to parameters as computed in previous literature. All models were evaluated when applied to two different cost functions at different window sizes and model parameters. Contrary to previous stereo confidence measure benchmark literature, we evaluate the models with criteria important not only to winner-take-all stereo, but also to global applications. To this end, we evaluate the models on a real-world application using a recent formulation of 3D reconstruction through occupancy grids which integrates stereo confidence at all disparities. We obtain and discuss our results on both indoors' and outdoors' publicly available datasets.

Energy efficiency and robustness of locomotion to different terrain conditions are important problems for humanoid robots deployed in the real world. In this paper, we propose a footstep-planning algorithm for humanoids that is applicable to flat, slanted, and slippery terrain, which uses simple principles and representations gathered from human gait literature. The planner optimizes a center-of-mass (COM) mechanical work model subject to motion feasibility and ground friction constraints using a hybrid A* search and optimization approach. Footstep placements and orientations are discrete states searched with an A* algorithm, while other relevant parameters are computed through continuous optimization on state transitions. These parameters are also inspired by human gait literature and include footstep timing (double-support and swing time) and parameterized COM motion using knee flexion angle keypoints. The planner relies on work, the required coefficient of friction (RCOF), and feasibility models that we estimate in a physics simulation. We show through simulation experiments that the proposed planner leads to both low electrical energy consumption and human-like motion on a variety of scenarios. Using the planner, the robot automatically opts between avoiding or (slowly) traversing slippery patches depending on their size and friction, and it chooses energy-optimal stairs and climbing angles in slopes. The obtained motion is also consistent with observations found in human gait literature, such as human-like changes in RCOF, step length and double-support time on slippery terrain, and human-like curved walking on steep slopes. Finally, we compare COM work minimization with other choices of the objective function.

Analysis of human running has revealed that the motion of the human leg can be modeled by a compression spring because the leg's joints behave like a torsion spring in the stance phase. Moreover, the knee bends rapidly to avoid contact of the foot with the ground in the swing phase. In this paper, we describe the development of a knee joint mechanism that mimics the elastic characteristics of the stance leg and rapid bending knee of the idling leg of a running human. The knee was equipped with a mechanism comprising two leaf springs and a worm gear for adjusting the joint stiffness and high-speed bending knee. Using this mechanism, we were able to achieve joint stiffness within the range of human knee joints that could be adjusted by varying the effective length of one of the leaf springs. In addition, the mechanism was able to bend rapidly by changing the angle between the two leaf springs. The equation proposed for calculating the joint stiffness considers the difference between the position of the fixed point of the leaf spring and the position of the rotational center of the joint. We evaluated the performance of the adjustable joint stiffness and the effectiveness of the proposed equation for joint stiffness and high-speed knee bending. We were able to make a bipedal robot hop using pelvic oscillation for storing energy produced by the resonance to leg elasticity and confirmed the mechanism could produce large torque 210 Nm.

Human running motion can be modeled using a spring-loaded inverted pendulum (SLIP), where the linear-spring-like motion of the standing leg is produced by the joint stiffness of the knee and ankle. To use running speed control in the SLIP model, we should only decide the landing placement of the leg. However, for using running speed control with a multi-joint leg, we should also decide the joint angle and joint stiffness of the standing leg because these affect the direction of the ground reaction force. In this study, we develop a running control method for a human-like multi-joint leg. To achieve a running motion, we developed a running control method including pelvis oscillation control for attaining jumping power with the joint stiffness of the leg and running speed control by changing the landing placement of the leg. For using running speed control, we estimated the ground reaction force using the equation of motion and detected the joint angles of the leg for directing the ground reaction force toward the center of mass. To evaluate the proposed control methods, we compared the estimated ground reaction force with the force measured by the real robot. Moreover, we performed a running experiment with the developed running robot. By using ground reaction force estimation, this robot could accomplish the running motion with pelvic oscillation for attaining jumping power and running speed control.

This paper describes the bipedal humanoid robot that makes human laugh with its whole body expression and affect human's psychological state. In order to realize "Social interaction" between human and robot, the robot has to affect human's psychological state actively. We focused on "laugh" because it can be thought as a typical example for researching "Social interaction". Looking through a Japanese comedy style called "manzai" or the art of conversation, we picked out several methods for making human laugh. Then we made several skits with the advice of comedians, and made the whole body humanoid robot perform them. Results of experimental evaluation with these skits shows that the robot's behavior made subjects laugh and change their psychological state seen as a decrease of "Depression" and "Anger".

Human running motion can be modeled by a spring loaded inverted pendulum (SLIP). However, this model, despite being widely used in robotics, does not include human-like pelvic motion. In this study, we show that the pelvis actually contributes to the increase in jumping force and absorption of landing impact, both of which findings can be used to improve running robots. On the basis of the analysis of human running motion, we propose a new model named SLIP2(spring loaded inverted pendulum using pelvis). This model is composed of a body mass, a pelvis, and leg springs; the model can control its springs during running by use of pelvic movement in the frontal plane. To achieve hopping and running motions, we developed pelvis oscillation control, running velocity control, and stabilization control using an upper body, as control methods. We also developed a new hopping robot using the SLIP2model. To evaluate the proposed model and control methods, we performed hopping and running simulations. The simulation results showed that the SLIP2model successfully achieves hopping and running motions. The hopping robot was also able to accomplish hopping motion. The simulation results also showed that the difference between the pelvic rotational phase and the phase of oscillation of the mass vertical displacement affects the jumping force. In particular, the results revealed that the human-like pelvic rotation contributes to the absorption of landing impact and to the increase in takeoff forces, which validates our observations in human motion analysis.

We present a grid-based 3D reconstruction method which integrates all costs given by stereo vision into what we call a Cost-Curve Occupancy Grid (CCOG). Occupancy probabilities of grid cells are estimated in a Bayesian formulation, from the likelihood of stereo cost measurements taken at all distance hypotheses. This is accomplished with only a small set of probabilistic assumptions which we discuss in the paper. We quantitatively characterize the method's performance under different conditions of both image noise and number of used stereo pairs, compared also to traditional algorithms. We complement the study by giving insights on design choices of CCOGs such as likelihood model, window size of the cost function and use of a hole filling method. Experiments were made on a real-world outdoors dataset with ground-truth data.

Humanoid robots have this formidable advantage to possess a body quite similar in shape to humans. This body grants them, obviously, locomotion but also a medium to express emotions without even needing a face. In this paper we propose to study the effects of emotional gaits from our biped humanoid robot on the subjects' perception of the robot (recognition rate of the emotions, reaction time, anthropomorphism, safety, likeness, etc.). We made the robot walk towards the subjects with different emotional gait patterns. We assessed positive (Happy) and negative (Sad) emotional gait patterns on 26 subjects divided in two groups (whether they were familiar with robots or not). We found that even though the recognition of the different types of patterns does not differ between groups, the reaction time does. We found that emotional gait patterns affect the perception of the robot. The implications of the current results for Human Robot Interaction (HRI) are discussed.

In this study we describe a shoes-wearable foot mechanism which mimics human's foot arch structure and the characteristics of the foot skin to examine the effect of the human foot on biped walking. We approximate the arch structure by a rotational spring and model the human's foot skin with compression springs. The mimesis of the foot arch elasticity is realized by using compression springs. Regarding the characteristics of human's foot skin, first we develop a measuring equipment to measure the human's plantar transverse elasticity. We set the target value of the modulus of transverse elasticity of the foot skin to 11.2 N/mm, and we set the modulus of compression elasticity to 0.136 MPa, referring to a relevant study. The characteristics of human's foot skin are mimicked by using a super-soft urethane resin called Hitohada-gel with 8 mm thick. Through evaluation tests we confirmed that the shoes-wearable foot mechanism could mimic well the characteristics of human's foot arch and skin.

This paper describes the implementation in a walking humanoid robot of a mental model, allowing the dynamical change of the emotional state of the robot based on external stimuli; the emotional state affects the robot decisions and behavior, and it is expressed with both facial and whole-body patterns. The mental model is applied to KOBIAN-R, a 65-DoFs whole body humanoid robot designed for human-robot interaction and emotion expression. To evaluate the importance of the proposed system in the framework of human-robot interaction and communication, we conducted a survey by showing videos of the robot behaviors to a group of 30 subjects. The results show that the integration of dynamical emotion expression and locomotion makes the humanoid robot more appealing to humans, as it is perceived as more “favorable” and “useful”, and less “robot-like".

This paper describes the development of a new shank mechanism and mimicking the human-like walking in the horizontal and frontal plane. One of human walking characteristics is that the COM (Center Of Mass) motion in the lateral direction is as small as 30 mm. We assume that it is thanks to the human walking characteristics in the horizontal plane that the step width is as narrow as 90 mm and the foot rotation angle is 12 deg. To mimic these characteristics, we developed a new shank and implemented it in a humanoid robot WABIAN-2RIII. It has a parallel mechanism which mimics the shank's size of human. Thanks to its size almost the same as human's the robot is capable of realizing gait with the narrow step width of 90 mm and the foot rotation angle of 12 deg. We evaluated the performance of the shank using WABIAN-2RIII. The robot could realize stepping in place with lateral displacement of CoM within 34 mm, which is almost as small as that of human.

Extensive literature has been written on occupancy grid mapping for different sensors. When stereo vision is applied to the occupancy grid framework it is common, however, to use sensor models that were originally conceived for other sensors such as sonar. Although sonar provides a distance to the nearest obstacle for several directions, stereo has confidence measures available for each distance along each direction. The common approach is to take the highest-confidence distance as the correct one, but such an approach disregards mismatch errors inherent to stereo. In this work, stereo confidence measures of the whole sensed space are explicitly integrated into 3D grids using a new occupancy grid formulation. Confidence measures themselves are used to model uncertainty and their parameters are computed automatically in a maximum likelihood approach. The proposed methodology was evaluated in both simulation and a real-world outdoor dataset which is publicly available. Mapping performance of our approach was compared with a traditional approach and shown to achieve less errors in the reconstruction.

The understanding of emotions in humans is really important in the Human-Robot Interaction field. The affective state of a person can be expressed in several ways, one of them being through the way we walk and our gait can convey emotional clues in social context. Those clues can be used to improve the personal interactions with our peers or add meaning to any message we want to express. However, only a few studies in humanoid robotics were done on the effects of the emotions on the walking. In this paper, we propose to assess the emotional walking patterns created from motion capture data with a survey. Those patterns represent different emotions (sadness, happiness) with different intensities (middle, high and exaggerated). Those emotional walking patterns achieved a high recognition rate of the emotions and the subjects (N=13) could recognize whole body emotions without facial expression on our humanoid robot. Additionally, we found out that at first people might perform poorly at recognizing emotions and their intensities but can get better, even without correction or feedback on their performances.

A complex relationship exists between national cultural background and interaction with robots, and many earlier studies have investigated how people from different cultures perceive the inclusion of robots into society. Conversely, very few studies have investigated how robots, speaking and using gestures that belong to a certain national culture, are perceived by humans of different cultural background. The purpose of this work is to prove that humans may better accept a robot that can adapt to their specific national culture. This experiment of Human-Robot Interaction was performed in Egypt. Participants (native Egyptians versus Japanese living in Egypt) were shown two robots greeting them and speaking respectively in Arabic and Japanese, through a simulated video conference. Spontaneous reactions of the human subjects were measured in different ways, and participants completed a questionnaire assessing their preferences and their emotional state. Results suggested that Egyptians prefer the Arabic version of the robot, while they report discomfort when interacting with the Japanese version. These findings confirm the importance of a culture-specific customisation of robots in the context of Human-Robot Interaction.

This paper describes a walking stabilization control based on gait analysis for a biped humanoid robot. We have developed a human-like foot mechanism mimicking the medial longitudinal arch to clarify the function of the foot arch structure. To evaluate the arch function through walking experiments using a robot, a walking stabilization control should also be designed based on gait analysis. Physiologists suggest the ankle, hip and stepping strategies, but these strategies are proposed by measuring human beings who are not “walking” but “standing” against force disturbances. Therefore, first we conducted gait analysis in this study, and we modeled human walking strategy enough to be implemented on humanoid robots. We obtained following two findings from gait analysis: i) a foot-landing point exists on the line joining the stance leg and the projected point of CoM on the ground, and ii) the distance between steps is modified to keep mechanical energy at the landing within a certain value. A walking stabilization control is designed based on the gait analysis. Verification of the proposed control is conducted through experiments with a human-sized humanoid robot WABIAN-2R. The experimental videos are supplemented.

This paper describes a walking stabilization control on a soft ground based on gait analysis for a humanoid robot. There are many researches on gait analysis on a hard ground, but few scientists analyze the walking ability of human beings on a soft ground. Therefore, we conducted anthropometric measurement using a motion capture system on a soft ground. By analyzing experimental data, we obtained two findings. The first finding is that although there are no significant differences in step width and step length, step height tends to increase to avoid the collision between the feet and a soft ground. The second finding is that there are no significant differences in the lateral CoM trajectories but the vertical CoM amplitude increases when walking on a soft ground. Based on these findings, we developed a walking stabilization control to stabilize the CoM motion in the lateral direction on a soft ground. Verification of the proposed control is conducted through experiments with a human-sized humanoid robot WABIAN-2R. The experimental videos are supplemented.

This paper describes the development of a new expressive robotic head for the bipedal humanoid robot. Facial expressions of our old robotic head have low facial expression recognition rate and in order to improve it we asked amateur cartoonists to create computer graphics (CG) images. To realize such expressions found in the CGs, the new head was provided with 24-DoFs and facial color. We designed compact mechanisms that fit into the head, which dimensions are based on average adult Japanese female size. We conducted a survey with pictures and videos to evaluate the expression ability. Results showed that facial expression recognition rates for the 6 basic emotions are increased compared to the old KOBIAN's head.

The results of the neuroscientific research show that humans tend to stabilize the head orientation during locomotion. In this paper we describe the implementation of inverse kinematics based head stabilization controller on the humanoid platform. The controller uses the IMU feedback and controls neck joints in order to align the head orientation with the global orientation reference. Thanks to the method, we can decouple the orientational motion of the head from the rest of the body. This way stabilized head becomes better platform for proprioceptive sensory apparatus, such as cameras or IMU. In the paper we present three experiments which prove that the method has good performance in damping both, high and low frequency motion of the head. We also prove that the proposed controller improves the stability of the tracked goal point on the image of in-built camera.

In this work we propose an adaptive model for the head stabilization based on a feedback error learning (FEL). This model is capable to overcome the delays caused by the head motor system and adapts itself to the dynamics of the head motion. It has been designed to track an arbitrary reference orientation for the head in space and reject the disturbance caused by trunk motion. For efficient error learning we use the recursive least square algorithm (RLS), a Newton-like method which guarantees very fast convergence. Moreover, we implement a neural network to compute the rotational part of the head inverse kinematics. Verification of the proposed control is conducted through experiments with Matlab SIMULINK and a humanoid robot SABIAN.

This paper describes a fast turning method for a humanoid robot by using slipping motion with both feet. The humanoid robot, WABIAN-2R, has achieved human-like walking with heel contact and toe off motions by using a human-like foot mechanism with a passive toe joint. The human-like foot enables a robot to turn by using slipping motion between the feet and the ground because it can switch ground contact conditions such as heel contact, sole contact and toe contact. To realize a slipping turn, we develop an attitude control. Verification of the proposed method is conducted through experiments with a biped humanoid robot WABIAN-2R. WABIAN-2R realized a quick turn by using slipping motion with both feet. We also confirmed that the energy consumption of a slipping turn is less than that of a stepping turn.

A four degrees of freedom (DoF) waist and trunk mechanism, as well as human-like foot, enable the humanoid robot WABIAN-2R to perform human-like walk with stretched knees, and heel-contact and toe-off gait phases. The inverse kinematics (IK) method, used in the present system, requires specification of not only task space reference trajectories, but also reference trajectories for all redundant DoFs. In this paper, we propose a novel, unified inverse kinematics method significantly simplifying the pattern generation. The method enables generation of the above described gait by specifying only the task space trajectories. We divide the forward locomotion task into subtasks with different priorities and combine them in the single IK equation. We also perform experiments in simulation environment as well as on WABIAN-2R, which prove that the method can be used to calculate IK for human-like gait. The equation evaluated in this paper is applied to the forward locomotion task, however it can be easily modified to perform other tasks on humanoid robots with different kinematic structures.

This paper describes an avoidance behavior from unknown external forces for a biped walking vehicle. To distinguish between external forces from passenger and those from environments, we use the data of a force sensor mounted on foot, and external forces are estimated from ZMP errors. To guarantee a walking stability, the waist position is adjusted to match the measured ZMP to the reference ZMP, and the position of landing foot is adjusted so that the waist trajectory does not diverge. By implementing the developed method on the human-carrying biped robot, the robot realized a stable walk under unknown external forces from environments. When pushing the robot stepping, the robot moved backward and moved away from the generation source of external forces about 400 mm. When pushing the robot walking forward, the robot stopped going forward and prevented from coming closer to the generation source of external forces. We confirmed the effectiveness of the proposed control through these experiments.

We have developed a new biped foot mechanism capable of detecting ground surface to realize stable walking on uneven terrain. The size of the foot mechanism is 160 mm × 277 mm and its weight is 1.5 kg. The foot system consists of four spikes each of which has an optical sensor to detect ground height. The foot makes a support polygon on uneven terrain by using three or four spikes. We have conducted several experiments on the outdoor ground surface that has a slope of 7.0 degrees and a maximum height of 15 mm bumps, and the effectiveness of the foot mechanism is confirmed.

The humanoid robot, WABIAN-2R, has achieved human-like walking with knee-stretched, heel contact and toe off motions by using a foot mechanism with a passive toe joint. However, the foot structure is different from a human's. In this paper, we describe a new foot mechanism capable of mimicking the human's foot arch structure to figure out the function of the arch structure. Especially, the developed foot mimics the elastic properties of the arch of the human's foot and the change of the arch height during walking. The foot mechanism consists of a passive joint in the internal toe, a passive joint in the external toe, and a joint in the foot arch. We conducted several walking experiments by using WABIAN-2R, and the function of the arch structure is clarified quantitatively. As a result, we confirmed that the arch elasticity could absorb a foot-landing force at the plantar contact phase and the change of the arch height contributed to a strong thrust at the push-off phase.

This paper reports real-time virtual haptization of, for example, a remote object surface based on high-speed 3D shape measurement. With this system, the operator feels sense of touch as if tracing the surface of an object. We use a high-speed projector-camera system to achieve high-speed measurement of the object surface so that high-frequency haptic feedback is maintained at a haptic device PHANToM. An actively directed structured light is projected so that the projected slit ray intersects with the haptic interaction point, which is determined by the stylus position of the haptic device. A camera image of this scene gives the positional relationship between the object surface and the haptic interaction point. We use this relationship to generate force feedback such that the haptic interaction point is pushed back to a point on the intersection of the slit ray and the object surface. Experimental results show that the proposed system displays an object shape correctly and that the high-speed measurement contributes to displaying smooth motion of an moving object.

This paper describes our novel work of using GPUs to improve the performance of a homography-based visual servo system. We present our novel implementations of a GPU based Efficient Second-order Minimization (GPU-ESM) algorithm. By utilizing the tremendous parallel processing capability of a GPU, we have obtained significant acceleration over its CPU counterpart. Currently our GPU-ESM algorithm can process a 360×360 pixels tracking area at 145 fps on a NVIDIA GTX295 board and Intel Core i7 920, approximately 30 times faster than a CPU implementation. This speedup substantially improves the realtime performance of our system. System reliability and stability are also greatly enhanced by a GPU based Scale Invariant Feature Transform (SIFT) algorithm, which is used to deal with such cases where ESM tracking failure happens, such as due to large image difference, occlusion and so on. In this paper, translation details of the ESM algorithm from CPU to GPU implementation and novel optimizations are presented. The co-processing model of multiple GPUs and multiple CPU threads is described in this paper. The performance of our GPU accelerated system is evaluated with experimental data.

Personal robots anticipated to become popular in the future are required to be active in joint work and community life with humans. These personal robots must recognize changing environment and must conduct adequate actions like human. Visual tracking can be said as a fundamental function from the view point of environmental sensing and reflex reaction against it. The authors developed a visual tracking motion algorithm by using upper body. Then, we integrated it with an online walking pattern generator and developed a visual tracking biped locomotion. Finally, we conducted an experimental evaluation with emotion expression.

Many researchers have studied on walking stability controls for biped robots. Most of them are highly accurate acceleration controls based on the mechanics model of the robot. However, the control algorithms are difficult to be applied to human-carrying biped robots due to modeling errors. In the previous report, we proposed the landing pattern modification method, but it had a problem that a foot landing impact increased when a walking speed became fast. So, we propose a new terrain-adaptive control that can reduce a landing- impact force. To increase a concave terrain adaptation, we set a target landing position beneath a reference level. To reduce the landing-impact force, we change the position gain control value to a small value at a swing phase. Moreover, we set landing-foot speed at zero after detecting a foot-landing by the force sensor mounted on a foot. To follow uneven terrain, a virtual spring is installed to the vertical direction after detecting a foot-landing on a ground, and a virtual compliance control is applied to the roll and pitch axes. In a stable walk while carrying a 65 kg human on uneven terrain, the new control method decreased the landing-impact force than the previous terrain-adaptive control.

Many control methods have been studied on the assumption that the feet of biped robots contact the ground with four points. However, it is difficult for almost all of such biped robots to maintain four-point contact on uneven terrains because they have rigid and flat soles. In order to solve the problem, foot mechanisms should be studied. In 2003, we developed WS-1R (Waseda Shoes - No.1 Refined) which is able to maintain four-point contact. However, it is difficult to deal with the concave or convex ground because of the problems of the contact detection and sideways slip. So, WS-5 (Waseda Shoes - No.5) has been developed. To avoid the slip of the foot, a cam-slider locking system consisting of a solenoid and a cam is constructed and installed at the foot. Also, linear encoders are employed to measure the position of the foot sliders. Through walking experiments on uneven terrains, the effectiveness of WS-5 is confirmed.

Previously, the realization of ascending and descending stairs and landing pattern modification control by WL-16RII were reported. However, it is difficult to use the landing pattern modification control when ascending stairs, because of the insufficient stroke of linear actuators. In this report, the design of a double stage linear actuator with a larger expansion and contraction ratio is described. The new linear actuators developed have been installed in WL-16RII, and several experiments were conducted. Through experiments involving walking with a wide step length and ascending a stair with landing pattern modification control, the effectiveness of the actuator developed is confirmed.

This paper describes how to compensate unknown external forces caused by a rider's motion of a biped walking vehicle. When external forces act on a robot's waist, the waist is accelerated so that a measured ZMP may be equal to a reference ZMP. To inhibit the divergence of the waist motion, the reference ZMP is varied inside a support polygon. However, if a large external force acts on a robot, the waist trajectory does not converge by only controlling a reference ZMP. So, ZMP trajectory is varied by changing a foot-landing point. Using the proposed control method, WL-16RIV (Waseda Leg-No. 16 Refined IV) achieved a stable human-carrying walking under unknown external forces which exert forward and sideways on the robot's waist. Through various walking experiments, the effectiveness of the proposed method was confirmed.

Many researchers have been studied on acceleration control algorithms for biped robots to deal with uneven terrain. However, the control algorithms are difficult to be used for human-carrying biped walking robots because of modeling errors. In this paper, a landing pattern modification method is proposed which based on nonlinear compliance control. Theoretical compliance displacements calculated from a walking pattern are compared with the actual compliance displacements, while a robot's foot touches slightly uneven terrain. In result, the height of landing terrain is detected, and the preset walking pattern is modified. Using this method, a human-carrying biped robot would be able to walk stably on uneven terrain. This pattern modification method does not need any special sensors except force sensors. Through various walking experiments, the effectiveness of the method is confirmed

This paper describes one method to stop suddenly and safely avoid falling using only hardware and without a computer system. This method consists of attaching a braking function to leg actuators and the development of a fall avoidance mechanism, attached to the robot's foot. Using the human-carrying biped locomotor WL-16RII developed by Sugahara et al., the effectiveness of the method proposed in this paper was confirmed through an emergency stopping experiment during walking, while carrying a heavy load

Many researchers have been studying on walking control methods for biped robots. However, the effectiveness of these control methods was not verified in outdoor environments such as pedestrian roads and gravel roads. In this paper, a landing pattern modification method adaptable to uneven terrain in a real environment is proposed which is based on a predictive attitude compensation control and a nonlinear compliance control. This method does not require any other sensors except force sensors. Also, a new biped foot system is described which can form larger support polygons on uneven terrain than conventional biped foot systems. Using the modification method and the foot system, WL-16RII (Waseda Leg-No.16 Refined II) achieved a stable walk on bumpy terrain with 20 mm height and 10 degrees inclination. Furthermore, a stable dynamic walk was realized on a paved road, when a human rode it. Through various walking experiments, the effectiveness of the method was confirmed

This paper describes a passive dynamic model of passenger for a biped walking vehicle. The walking pattern generation that enables stable walking even if passenger sits naturally is also described. The model consists of lower-limbs part assumed to be fixed to the robot, and the upper body assumed to be 1 particle with 2 DOF mounted on the seat via 2 springs and dampers. The parameters are identified through waist shaking experiments by using a force-torque sensor under the seat. The walking pattern generation method involves the proposed model being built onto a strict model of the robot, and through iteration computation, a stable walking pattern is generated. The effectiveness of the proposed method is confirmed through experiments

Biped walking is easily adaptable to rough terrain such as stairs and stony paths, but the walk speed and energy efficiency on the flat ground is not so effective compared with wheeled locomotion. In this paper, we propose a biped robot to be able to walk or wheel according to the ground conditions. For wheeled locomotion, WS-2 (Waseda Shoes -No. 2) is developed which is composed of a DC motor, a spherical caster and two rubber pads on each foot. WS-2 is attached to the feet of WL-16 (Waseda Leg -No. 16) that is the world's first biped-walking robot capable of carrying a human. Also, a path planning for wheeled locomotion is presented. Through hardware experiments, the effectiveness of this foot module is confirmed.

This paper describes a walking control method on inclined planes for a biped locomotor. The walking control consists of a position control, virtual compliance control and posture control. Parameters of the compliance control are changed continuously in support and swing phases. The orientation of robot’s waist is kept level by posture control. Several walking experiments on inclined planes are conducted using WL-16R (Waseda Leg - No. 16 Refined). WL-16R has achieved stable walking on unknown inclined planes using the walking control method, and the effectiveness of the walking control is confirmed.

To date, many control methods have been studied, assuming that the soles of a biped walking robot contact the ground as four points. It is difficult for a biped robot with rigid and flat soles to maintain four-point contact on uneven terrain. It means that the biped robot can lose its balance. To solve this kind of problem, we should study not only stability control methods but also foot mechanisms. In this paper, a new foot system, WS-1 (Waseda shoes $no.1), is proposed to maintain four-point contact. This foot system consists of cam-type locking mechanism. WS-1 is attached to the feet of WL-16 (Waseda leg - no.16) that is the world's first biped-walking robot capable of carrying a human. Through hardware experiments, the effectiveness of the foot system is confirmed.

This paper describes the means of tuning-up method of the walking parameters to go up and down stairs for a biped robot with leg mechanisms using Stewart platforms. It has been confirmed that the stroke range of use could be reduced by tuning up the waist yaw trajectory and preset ZMP trajectories for motion pattern generation. By using the developed method, a walking experiment involving movement up and down a stair with the rise of 250 mm and certain walking experiments ascending and descending stairs carrying a human were successfully completed. Through these experiments, the effectiveness of the proposed method was confirmed.

This paper describes the development of the support torque reduction mechanism for a biped locomotor with leg mechanisms using Stewart platforms, designed for heavy payload and low power consumption during walking. The basic design of this mechanism was decided through preliminary experiments. Each leg of the robot is equipped with this mechanism consisting of 2 gas springs with different reaction forces, which are switched according to the swing/support phase. The experiment measuring motor current during walking with payload and the walking experiment with maximum payload confirmed the effectiveness of this mechanism.

This paper reports our work to improve a bigram language model for Japanese TV broadcast news speech recognition. First, frequent word strings were grouped into phrases in order that the phrases were added to the lexicon as new units of recognition. The test set perplexity was improved when frequent function word strings were used as additional recognition units. The speech recognition performance was improved both by grouping function word strings and by grouping compound nouns that were selected by word association ratio. Secondly, in order to alleviate the OOV problem related with nouns, we built and tested a language model that allows switching its noun lexicon according to the domain of the article to be recognized next. 1 INTRODUCTION The DARPA Hub-4 launched in 1995 is evaluating LVCSR systems to transcribe audio recordings of broadcast news [1]. The evaluation results have been updated annually [2]. The Japanese counterpart of HUB-4 broadcast news materials has been being deve...