This paper presents a hierarchical framework for planning and control of in-hand manipulation of a rigid object involving grasp changes using fully-actuated multifin-gered robotic hands. While the framework can be applied to the general dexterous manipulation, we focus on a more complex definition of in-hand manipulation, where at the goal pose the hand has to reach a grasp suitable for using the object as a tool. The high level planner determines the object trajectory as well as the grasp changes, i.e. adding, removing, or sliding fingers, to be executed by the low-level controller. While the grasp sequence is planned online by a learning-based policy to adapt to variations, the trajectory planner and the low-level controller for object tracking and contact force control are exclusively model-based to robustly realize the plan. By infusing the knowledge about the physics of the problem and the low-level controller into the grasp planner, it learns to successfully generate grasps similar to those generated by model-based optimization approaches, obviating the high computation cost of online running of such methods to account for variations. By performing experiments in physics simulation for realistic tool use scenarios, we show the success of our method on different tool-use tasks and dexterous hand models. Additionally, we show that this hybrid method offers more robustness to trajectory and task variations compared to a model-based method.

While haptics research has traditionally focused on the fingertips and hands, other locations on the body provide large areas of skin that could be utilized to relay large-area haptic sensations. Researchers have thus developed wearable devices that use distributed vibrotactile actuators and distributed pneumatic force displays, but these methods have limitations. In prior work, we presented a novel actuation technique involving stacking pneumatic pouches and evaluated the actuator output. In this work, we developed a wearable haptic device using this actuation technique and evaluated how the actuator output is perceived. We conducted a user study with 20 participants to evaluate users' perception thresholds, ability to localize, and ability to detect differences in contact area and compare their perception using the stacked pneumatic pouch actuation to traditional single-layer pouch actuation. We also used our device with stacked pneumatic actuation in a demonstration of a haptic hug that replicates the dynamics, pressure profile, and mapping to the human back, showcasing how this actuation technique can be used to create novel haptic stimuli.

Pneumatic wearable haptic devices can provide distributed pressure feedback to human operators during robot teleoperation and in virtual and augmented reality. However, these devices have an inherent trade-off between the spatial coverage of their pressure output and their resolution and dynamic response. To achieve specified spatial resolution and dynamic response, we propose a macro-mini actuation approach that stacks a number of smaller inflatable pouches atop a larger inflatable pouch. We develop models for the static and dynamic responses of single and stacked pouches and compare these with experimental results, providing guidelines for the design of wearable stacked pneumatic displays. Finally, we demonstrate this pneumatic macro-mini approach by replicating the time series pressure profiles of data collected from a huggable robot embedded with distributed force sensors.

Robotic fabric manipulation is challenging due to the infinite dimensional configuration space, self-occlusion, and complex dynamics of fabrics. There has been significant prior work on learning policies for specific fabric manipulation tasks, but comparatively less focus on algorithms which can perform many different tasks. We take a step towards this goal by learning point-pair correspondences across different fabric configurations in simulation. Then, given a single demonstration of a new task from an initial fabric configuration, these correspondences can be used to compute geometrically equivalent actions in a new fabric configuration. This makes it possible to define policies to robustly imitate a broad set of multi-step fabric smoothing and folding tasks. The resulting policies achieve 80.3% average task success rate across 10 fabric manipulation tasks on two different physical robotic systems. Results also suggest robustness to fabrics of various colors, sizes, and shapes. See https://tinyurl.com/fabric-descriptors for supplementary material and videos.

In this paper, we present a capacitance-based sensor array for physical human-robot interaction (pHRI) applications that can measure the proximity, near-zero-force (NZF) contacts, and pressure between a robot and human body. The top segment including the electrodes is made of soft, stretchable materials, while the bottom segment consists of electrodes patterned from a thin copper film. The resulting device is soft and conformable to smooth curved surfaces of robot links while ensuring high signal integrity. It can be fabricated in different sizes from fingertips to torso because the fabrication process employs conventional, scalable methods. Using this sensor, we investigate the problem of recognizing gentle contact gestures often seen in affectionate physical interactions. The output of this multi-modal sensor is a 2D array compatible with machine learning algorithms used for pressure and image-based recognition problems. We utilize the spatio-temporal information of the 2D capacitance data by applying two existing deep neural network architectures. The highest accuracy achieved is over 99% in 7-class recognition of contact gestures involving proximity, NZF contacts, and medium pressure.

This paper presents a learning-from-demonstration (LfD) framework for teaching human-robot social interactions that involve whole-body haptic interaction, i.e. direct human-robot contact over the full robot body. The performance of existing LfD frameworks suffers in such interactions due to the high dimensionality and spatiotemporal sparsity of the demonstration data. We show that by leveraging this sparsity, we can reduce the data dimensionality without incurring a significant accuracy penalty, and introduce three strategies for doing so. By combining these techniques with an LfD framework for learning multimodal human-robot interactions, we can model the spatiotemporal relationship between the tactile and kinesthetic information during whole-body haptic interactions. Using a teleoperated bimanual robot equipped with 61 force sensors, we experimentally demonstrate that a model trained with 121 sample hugs from 4 participants generalizes well to unseen inputs and human partners.

Sequential pulling policies to flatten and smooth fabrics have applications from surgery to manufacturing to home tasks such as bed making and folding clothes. Due to the complexity of fabric states and dynamics, we apply deep imitation learning to learn policies that, given color (RGB), depth (D), or combined color-depth (RGBD) images of a rectangular fabric sample, estimate pick points and pull vectors to spread the fabric to maximize coverage. To generate data, we develop a fabric simulator and an algorithmic supervisor that has access to complete state information. We train policies in simulation using domain randomization and dataset aggregation (DAgger) on three tiers of difficulty in the initial randomized configuration. We present results comparing five baseline policies to learned policies and report systematic comparisons of RGB vs D vs RGBD images as inputs. In simulation, learned policies achieve comparable or superior performance to analytic baselines. In 180 physical experiments with the da Vinci Research Kit (dVRK) surgical robot, RGBD policies trained in simulation attain coverage of 83% to 95% depending on difficulty tier, suggesting that effective fabric smoothing policies can be learned from an algorithmic supervisor and that depth sensing is a valuable addition to color alone. Supplementary material is available at https://sites.google.com/view/fabric-smoothing.

Mobile robots moving in crowded environments have to navigate among pedestrians safely. Ideally, the way the robot avoids the pedestrians should not only be physically safe but also perceived safe and comfortable. Despite the rich literature in collision-free crowd navigation, limited research has been conducted on how humans perceive robot behaviors in the navigation context. In this paper, we implement three local pedestrian avoidance strategies inspired by human avoidance behaviors on a self-balancing mobile robot and evaluate their perception in a human-robot crossing scenario through a large-scale user study with 98 participants. The study reveals that the avoidance strategies positively affect the participants' perception of the robot's safety, comfort, and awareness to different degrees. Furthermore, the participants perceive the robot as more intelligent, friendly and reliable in the last trial than in the first even with the same strategy.

This paper presents a 6 DOF leg mechanism for a humanoid robot. The proposed Hybrid Leg is designed to combine serial and parallel mechanisms and consists of a pair of twin 3 DOF serial chains in parallel. A 5-bar-linkage mechanism is implemented to the serial mechanism to generate 2 DOF motion regarding hip and knee pitch rotation. The hardware prototype is designed by matching the kinematic specification of a commercial robot's leg to compare the proposed mechanism with a conventional serial leg. We derive the analytical expressions of its forward and inverse kinematics. End-effector workspaces are shown with plots and inverse dynamics analysis of Hybrid Leg and serial leg with a given walking gait trajectory is presented. Hardware experiment is conducted with a prototype to verify the simulated workspace and trajectory tracking performance.

Balance control to recover from a wide range of disturbances is an important skill for humanoid robots. Traditionally, researchers have often designed a balance controller by applying optimal control theory on a simplified model that abstracts the full-body dynamics. However, the resulting controller may not be able to recover from unexpected scenarios such as non-planar pushes, or fail to exploit full-body actions such as balancing with arm movements. This paper presents a learning framework for enhancing the performance of a model-based optimal controller by expanding the region of attraction (RoA). We train a control policy that generates additional control signals on top of the model-based controller using deep reinforcement learning techniques. Instead of relying on standard reinforcement learning formulations, we explicitly model the region of attraction and continuously adjust it during the training. By drawing the training disturbances at the boundary of the RoA, we can effectively expand the RoA while avoiding local minima. We test our learning framework for in-place balancing as well as balancing with stepping on a humanoid model in simulation.

The hardware compatibility of legged locomotion is often illustrated by Zero Moment Point (ZMP) that has been extensively studied for decades. One of the most popular models for computing the ZMP is the linear inverted pendulum (LIP) model that expresses ZMP as a linear function of the center of mass(COM) and its acceleration. In the real world, however, it may not accurately predict the true ZMP of hardware due to various reasons such as unmodeled dynamics and differences between simulation model and hardware. In this paper, we aim to improve the theoretical ZMP model by learning the real hardware dynamics from experimental data. We first optimize the motion plan using the theoretical ZMP model and collect COP data by executing the motion on a force plate. We then train a new ZMP model that maps the motion plan variable to the actual ZMP and use the learned model for finding a new hardware-compatible motion plan. Through various locomotion tasks of a quadruped, we demonstrate that motions planned for the learned ZMP model are compatible on hardware when those for the theoretical ZMP model are not. Furthermore, experiments using ZMP models with different complexities reveal that overly complex models may suffer from over-fitting even though they can potentially represent more complex, unmodeled dynamics.

In this paper, we present the design and performance evaluation of a bipedal robot that utilizes the Hybrid Leg mechanism. It is a leg mechanism that achieves 6 DOF with a combined structure of serial and parallel mechanism. It is designed to have a light structural inertia and large workspace for agile bipedal locomotion. A new version of Hybrid Leg is fabricated with carbon fiber tubes and bearings to improve its structural rigidity and accuracy while supporting its weight. A pair of Hybrid Legs is assembled together for bipedal locomotion. In the assembly, we adopt a pelvis structure with an yaw angle offset to enlarge the feet workspace, inspired by the toe-out angle of the human feet. The workspace and range of velocity are presented in simulation and verified with hardware experiments. We also demonstrate a simple forward walking motion with the developed robot.

We present a novel computational approach to design the robotic devices from high-level motion specifications. Our computational system uses a library of modular components-actuators, mounting brackets, and connectors-to define the space of possible robot designs. The process of creating a new robot begins with a set of input trajectories that specify how its end effectors and/or body should move. By searching through the combinatorial set of possible arrangements of modular components, our method generates a functional, as-simple-as-possible robotic device that is capable of tracking the input motion trajectories. To significantly improve the efficiency of this discrete optimization process, we propose a novel heuristic that guides the search for appropriate designs. Briefly, our heuristic function estimates how much an intermediate robot design needs to change before it becomes able to execute the target motion trajectories. We demonstrate the effectiveness of our computational design method by automatically creating a variety of robotic manipulators and legged robots. To generate these results, we define our own robotic kit that includes off-the-shelf actuators and 3-D printable connectors. We validate our results by fabricating two robotic devices designed with our method.

We develop a reconfigurable legged robot, named Snapbot, to emulate configuration changes and various styles of legged locomotion. The body of Snapbot houses a microcontroller and a battery for untethered operation. The body also contains connections for communication and power to the modular legs. The legs can be attached to and detached from the body using magnetic mechanical couplings. In the center of this coupling, there is a multi-pin spring-loaded electrical connector that distributes power and transmits data between the controller and leg actuators. The locomotion algorithm is implemented on the microcontroller. The algorithm enables Snapbot to locomote in various configurations with one to six legs by recognizing configuration changes and selecting the locomotion method according to the current configuration. Snapbot will be utilized for further research on legged locomotion.

Deformation of skin and muscle is essential for bringing an animated character to life. This deformation is difficult to animate in a realistic fashion using traditional techniques because of the subtlety of the skin deformations that must move appropriately for the character design. In this paper, we present an algorithm that generates natural, dynamic, and detailed skin deformation (movement and jiggle) from joint angle data sequences. The algorithm has two steps: identification of parameters for a quasi-static muscle deformation model, and simulation of skin deformation. In the identification step, we identify the model parameters using a musculoskeletal model and a short sequence of skin deformation data captured via a dense marker set. The simulation step first uses the quasi-static muscle deformation model to obtain the quasi-static muscle shape at each frame of the given motion sequence (slow jump). Dynamic skin deformation is then computed by simulating the passive muscle and soft tissue dynamics modeled as a mass–spring–damper system. Having obtained the model parameters, we can simulate dynamic skin deformations for subjects with similar body types from new motion data. We demonstrate our method by creating skin deformations for muscle co-contraction and external impacts from four different behaviors captured as skeletal motion capture data. Experimental results show that the simulated skin deformations are quantitatively and qualitatively similar to measured actual skin deformations.

Among legged robots, hopping and running robots are useful because they can traverse terrain at high speeds and are a benchmark platform for locomotion actuators; if an actuator can power a hopping robot, it can power a walking robot. We aim to create a hopping mechanism for a small-scale, one-legged, untethered hopping robot. A parallel-elastic actuator is an efficient way to do this, and enables the actuator to directly inject energy into the spring, but requires a high-speed, low-inertia actuator. Voice coil actuators are electrically-powered direct-drive translational motors that have very low moving inertia, low friction, can produce force at high speeds, and have a linear force output. These qualities make them ideal candidate motors for a linear elastic actuator in parallel (“LEAP”). Here, we derive an electromechanical model of the LEAP mechanism, develop a simple bang-bang hopping controller, and simulate hopping with a range of spring parameters to find an optimal spring stiffness that maximizes hopping height. We detail our implemented design, and characterize its performance through a series of experiments. We test our robot with different spring stiffnesses, and demonstrate hopping at a maximum steady-state of 3.5 cm ground-clearance (approx. 20% leg length). Our results suggest that the LEAP mechanism may serve the weight-bearing functions of a robot leg.

The design of legged robots is often inspired by animals evolved to excel at different tasks. However, while mimicking morphological features seen in nature can be very powerful, robots may need to perform motor tasks that their living counterparts do not. In the absence of designs that can be mimicked, an alternative is to resort to mathematical models that allow the relationship between a robot's form and function to be explored. In this paper, we propose such a model to co-design the motion and leg configurations of a robot such that a measure of performance is optimized. The framework begins by planning trajectories for a simplified model consisting of the center of mass and feet. The framework then optimizes the length of each leg link while solving for associated full-body motions. Our model was successfully used to find optimized designs for legged robots performing tasks that include jumping, walking, and climbing up a step. Although our results are preliminary and our analysis makes a number of simplifying assumptions, our findings indicate that the cost function, the sum of squared joint torques over the duration of a task, varies substantially as the design parameters change.

Effective teleoperation requires real-time control of a remote robotic system. In this work, we develop a controller for realizing smooth and accurate motion of a robotic head with application to a teleoperation system for the Furhat robot head [1], which we call TeleFurhat. The controller uses the head motion of an operator measured by a Microsoft Kinect 2 sensor as reference and applies a processing framework to condition and render the motion on the robot head. The processing framework includes a pre-filter based on a moving average filter, a neural network-based model for improving the accuracy of the raw pose measurements of Kinect, and a constrained-state Kalman filter that uses a minimum jerk model to smooth motion trajectories and limit the magnitude of changes in position, velocity, and acceleration. Our results demonstrate that the robot can reproduce the human head motion in real time with a latency of approximately 100 to 170 ms while operating within its physical limits. Furthermore, viewers prefer our new method over rendering the raw pose data from Kinect.

We have developed a toy sized humanoid robot with soft air-filled modules on its links which sense contact and protect the robot and any interacting humans from damaging collisions. This robot, meant for robust physical interaction, is required to endure contact with children in the form of hugs and other playful interactions. It is therefore necessary to quantify the forces exerted during these interactions so that robots can be designed to both withstand these forces, as well as interact safely and intuitively in these situations. To quantify the range of forces exerted by children when performing both soft and strong hugs, we conducted a study in which 28 children (11 boys, 17 girls) between 4 and 10 years old hugged a pressure sensing doll while the pressure was recorded. We found a child's maximum expected hugging force (2.623 psi for our setup) during free play. The data gathered in this study will guide the further development of our physically interactive robot.

The thirteen papers in this special section focus on the topic of movement science for humans and humanoids. The papers include the collection and organization of human movement data for enabling robotics research; the use of human movement as inspiration for humanoid planning, control, and motion generation; the development of algorithms for improved estimation of human and humanoid system parameters; and the use of human movement understanding in robotics applications including human-robot interaction and rehabilitation.

The anthropomorphic body form is a complex articulated system of links/limbs and joints, simultaneously redundant and underactuated, and capable of a wide range of sophisticated movement. The human body and its movement have long been a topic of study in physiology, anatomy, biomechanics, and neuroscience and have served as inspiration for humanoid robot design and control. This survey paper reviews the literature on robotics research using anthropomorphic design principles as an inspiration, at both the design and control levels. Next, anthropomorphic body modeling, motion analysis, and synthesis techniques are overviewed. Finally, key applications arising at the intersection of robotics and human movement science are introduced. The survey ends with a discussion of open research questions and directions for future work.

Conducting hardware experiment is often expensive in various aspects such as potential damage to the robot and the number of people required to operate the robot safely. Computer simulation is used in place of hardware in such cases, but it suffers from so-called simulation bias in which policies tuned in simulation do not work on hardware due to differences in the two systems. Model-free methods such as Q-Learning, on the other hand, do not require a model and therefore can avoid this issue. However, these methods typically require a large number of experiments, which may not be realistic for some tasks such as humanoid robot balancing and locomotion. This paper presents an iterative approach for learning hardware models and optimizing policies with as few hardware experiments as possible. Instead of learning the model from scratch, our method learns the difference between a simulation model and hardware. We then optimize the policy based on the learned model in simulation. The iterative approach allows us to collect wider range of data for model refinement while improving the policy.

Our goal is to bring animation characters to life in the real world. We present a bipedal robot that looks like and walks like an animation character. We start from animation data of a character walking. We develop a bipedal robot which corresponds to lower part of the character following its kinematic structure. The links are 3D printed and the joints are actuated by servo motors. Using trajectory optimization, we generate an open-loop walking trajectory that mimics the character's walking motion by modifying the motion such that the Zero Moment Point stays in the contact convex hull. The walking is tested on the developed hardware system.

Having human-like motions will make humanoid robots more predictable and safer for the people around them. An effective way to realize this would be to use human motions as reference. Due to different kinematic and dynamic properties between humans and humanoid robots, however, a human motion could be physically infeasible for a robot and cause the robot to fall over. Therefore, it is necessary to modify and adapt an infeasible human motion to the robot. This paper presents a method for adapting human motions to humanoid robots based on a technique called time warping, which modifies the time line of a reference motion to speed up or slow down the motion. By doing this, the velocity and acceleration profiles of the motion are changed and it is possible to turn an infeasible motion into a feasible one. The optimal time warping is obtained through a generalized motion feasibility index that quantifies the feasibility of a motion considering the friction and center-of-pressure constraints. Thanks to the generality of the index, the proposed motion adaptation method can be applied to motions on arbitrary terrains or number of links in contact with the environment. Through dynamics simulation, we demonstrate that the method facilitates the reproduction of human motions on a humanoid robot.

The purpose of this research is the development of a soft skin module with a built-in airtight cavity in which air pressure can be sensed. A pressure feedback controller is implemented on a robotic system using this module for contact sensing and gentle grasping. The soft skin module is designed to meet size and safety criteria appropriate for a toy-sized interactive robot. All module prototypes are produced using a muti-material 3D printer. Experimental results from collision tests show that this module significantly reduces the impact forces due to collision. Also, using the measured pressure information from the module, the robotic system to which these modules are attached is capable of very gentle physical interaction with soft objects.

This paper presents algorithms to compute the generalized distance between two separated or penetrating compact convex sets, which is defined as the minimum or maximum scale factor of a given gauge set such that the scaled gauge set intersects or is contained in the Minkowski difference of the two sets. The traditional Euclidean distance is a special case where the origin-centered unit ball is used as the gauge set. While the generalized distance was proposed almost a decade ago, the only practical method for its computation has been general-purpose numerical optimization, which is computationally expensive. In contrast, our geometry-based algorithms are efficient and guarantee globally optimal solutions. Important applications of the algorithms in robotics include collision detection and grasp planning. The algorithm for computing the penetration distance also provides an accurate and efficient approach to flatness error evaluation, which is a fundamental problem in manufacturing. We demonstrate that our algorithms possess superior efficiency and accuracy in these applications.

This paper presents a novel universal balancing controller that successfully stabilizes a planar bipedal robot in dynamic, unstable environments like seesaw and bongoboards, and also in static environments like curved and flat floors. These different dynamic systems have state spaces with different dimensions, and hence instead of using full state feedback, the universal controller is derived as a single output feedback controller that stabilizes them. This paper analyzes the robustness of the derived universal controller to disturbances and parameter uncertainties, and demonstrates its universality and superiority to similarly derived LQR and H∞controllers. This paper also presents nonlinear simulation results of the universal controller successfully stabilizing a family of bongoboard, curved floor, seesaw, tilting and rocking floor models.

This paper presents an offline, centralized motion planning algorithm for displaying stick figure animations by a group of mobile robots equipped with a light source. The algorithm plans collision-free trajectories for the robots such that the figure appears visually consistent across frames including overlaps between body parts. We use 3D motion capture data as input to obtain clean stick figure images. The algorithm consists of three steps: segment generation, robot assignment, and trajectory optimization. In the segment generation step, the input 3D animation is converted to a set of segments of visible robot trajectories in the 2D image plane. The robot assignment step then assigns a robot to each segment using dynamic programming. Finally, the trajectory optimization step computes the complete collision-free trajectory for every robot, including when a robot is not assigned to any segment. We demonstrate the algorithm in simulation using up to 75 robots.

This paper studies the balancing of simple planar bipedal robot models in dynamic, unstable environments such as seesaw, bongoboard, and board on a curved floor. This paper derives output feedback controllers that successfully stabilize seesaw, bongoboard, and curved floor models using only global robot information and with no direct feedback of the dynamic environment and, hence, demonstrates that direct feedback of environment information is not essential for successfully stabilizing the models considered in this paper. This paper presents an optimization to derive stabilizing output feedback controllers that are robust to disturbances on the board. It analyzes the robustness of the derived output feedback controllers to disturbances and parameter uncertainties and compares their performance with similarly derived robust linear quadratic regulator controllers. This paper also presents nonlinear simulation results of the output feedback controllers' successful stabilization of bongoboard, seesaw, and curved floor models.

This paper presents a method for synthesizing motions of a humanoid robot that receives an object from a human, with focus on a natural object passing scenario where the human initiates the passing motion by moving an object towards the robot, which continuously adapts its motion to the observed human motion in real time. In this scenario, the robot not only has to recognize and adapt to the human action but also has to synthesize its motion quickly so that the human does not have to wait holding an object. We solve these issues by using a human motion database obtained from two persons performing the object passing task. The rationale behind this approach is that human performance of such a simple task is repeatable, and therefore the receiver (robot) motion can be synthesized by looking up the passer motion in a database. We demonstrate in simulation that the robot can start extending the arm at an appropriate timing and take hand configurations suitable for the object being passed. We also perform hardware experiments of object handing from a human to a robot.

This paper proposes a new grasp force efficiency (GFE) measure that considers not only contact point locations but also the hand configuration and mechanism. GFE evaluates the largest wrench applied to the object that the grasp can resist with unit contact forces. Traditional GFE measures depend solely on the contact point locations without considering how the unit contact forces are generated. Intuitively, however, the actuators' effort required to generate unit contact forces depends on the hand configuration and mechanism and therefore should affect the grasp efficiency. For example, generating a unit contact force with an under-actuated finger would be more difficult than with a fully actuated finger. Our new GFE measure addresses this issue and is potentially useful for hand mechanism design as well as grasp planning. We also present a novel geometry-based iterative algorithm for computing the generalized penetration distance of a point in an arbitrary convex set. The algorithm allows unified and accurate computation of the new and traditional GFE measures with various criteria without linear approximation. Other applications of the algorithm include penetration depth computation of two convex objects for physics simulation.

This paper presents an efficient algorithm for computing a distance measure between two compact convex sets Q and A, defined as the minimum scale factor such that the scaled Q is not disjoint from A. An important application of this algorithm in robotics is the computation of the minimum distance between two objects, which can be performed by taking A as the Minkowski difference of the objects and Q as a set containing the origin in its interior. In this generalized definition, the traditional Euclidean distance is a special case where Q is the unit ball. While this distance measure was proposed almost a decade ago, there has been no efficient algorithm to compute it in general cases. Our algorithm fills this void and we demonstrate its superior efficiency compared to approaches based on general-purpose optimization.

Simple inverted pendulum models and their variants are often used to control humanoid robots in order to simplify the control design process. These simple models have significantly fewer degrees of freedom than the full robot model. The design and choice of these simple models are based on the designer's intuition, and the reduced state mapping and the control input mapping are manually chosen. This paper presents an automatic model reduction procedure for humanoid robots, which is task-specific. It also presents an optimization framework that uses the auto-generated task-specific reduced models to control humanoid robots. Successful simulation results of balancing, fast arm swing, and hip rock and roll motion tasks are demonstrated.

Ray shooting is a well-studied problem in computer graphics and also has applications in robotics such as collision detection and contact force optimization. Unfortunately, most ray-shooting algorithms developed for graphics applications only allow 3-dimensional (3-D) objects represented as triangle meshes, and therefore are not suited for objects with parametric surfaces or general convex sets in high-dimensional space which robotics applications often require. In contact force optimization, for example, the problem is in the 6-dimensional (6-D) wrench space and it is desirable to consider the nonlinear friction cone without approximating it by a pyramid. This paper discusses existing and novel geometry-based ray-shooting algorithms applicable to general convex sets, and compares their performances in two robotics applications: computing the distance between two convex objects and optimizing contact forces in grasping.

In this paper, we demonstrate that a neuromuscular controller built based on the human anatomical structure and motion data can realize human-like responses to unexpected disturbances during locomotion. This particular work concerns the response to trips due to obstacles and shows that the two strategies identified in biomechanics emerge from a single controller. We first identify the parameters of a neuromuscular network model using the muscle tension data during a human walking motion. The anatomically-correct network models the somatosensory reflex of the human neuromuscular system. We use this network as the controller for a musculoskeletal human model to simulate its response to disturbances. Simulation results show that our neuromuscular controller automatically results in the appropriate trip recovery strategy with a single set of parameters, although we do not explicitly model the trip response or the condition to invoke each strategy. This result implies that an appropriately designed locomotion controller can also provide rapid responses to trips without deliberate controller selection or planning.

This paper presents a control framework for a biped robot to maintain balance and walk on a rolling ball. The control framework consists of two primary components: a balance controller and a footstep planner. The balance controller is responsible for the balance of the whole system and combines a state-feedback controller designed by pole assignment with an observer to estimate the system's current state. A wheeled linear inverted pendulum is used as a simplified model of the robot in the controller design. Taking the output of the balance controller, namely the ideal center of pressure of the biped robot on the ball, as the input, the footstep planner computes the foot placements for the robot to track the ideal center of pressure and avoid a fall from the rolling ball. Simulation results show that the proposed controller can enable a biped robot to stably walk on balls of different sizes and rotate a ball to desired positions at desired speeds.

Human motion coordination is a long-standing research issue in biomechanics, and it should also have some implications for humanoid robot control.We have built a whole-body somatosensory reflex model based on our neuromusculoskeletal model and identified its parameters through non-invasive measurements and statistical analysis. Such models are crucial for analyzing and estimating signals in the nervous system. In this paper, we focus on signal transmission delay of the somatosensory reflex loop and investigate its relationship with the generalization capability of the reflex model. We obtain some sets of model parameters assuming different time delays using the data obtained from a stepping motion, and perform cross validations against stepping motions with different cycles as well as entirely different behaviors such as squat and jump. Interestingly, time delays close to the value expected from physiological properties show better cross-validation results than others. This result suggests that relatively simple reflex control can be generalized to multiple behaviors if the parameters are appropriate, and that robust control is possible even with large feedback delay.

This paper presents a method for mapping captured human motion with stepping to a humanoid model, considering the current state and the controller behavior. The mapping algorithm modifies the joint angle, trunk and center of mass (COM) trajectories so that the motion can be tracked and desired contact states can be achieved. The mapping is performed in two steps. The first step modifies the joint angle and trunk trajectories to adapt to the robot kinematics and actual contact foot positions. The second step uses a predicted center of pressure (COP) to determine if the balance controller can successfully maintain the robot's balance, and if not, modifies the COM trajectory. Unlike most humanoid control work that handles motion synthesis and control separately, our COM trajectory modification is performed based on the behavior of the robot controller. We verify the approach in simulation using a captured Tai-chi motion that involves unstructured contact state changes.

Contact-free estimation of the human somatosensory information is an essential skill for robots working in daily environments. The main objective of this paper is to develop a method for estimating muscle tensions without any sensors attached to the body. Muscle tension is an important information for evaluating physical load during motions. Existing approaches utilizing optimization techniques and/or electromyography (EMG) signals are not appropriate due to lack of physiological validity or usage of electrodes. In this paper, we propose to use a database of muscle tension distribution for obtaining physiologically realistic muscle tensions only from motion data. Using such database instead of direct EMG measurement is justified by the fact that muscle tension distribution is relatively highly correlated even among different subjects. For each new motion frame, we search for a similar entry in the database and use the corresponding muscle tension distribution to estimate the current muscle tensions. We demonstrate that the muscle tensions obtained by this approach is much closer to the result using the EMG data than that using pure numerical optimization, even when the database is constructed from other person's data.

This paper presents a control framework for humanoid robots that uses all joints simultaneously to track motion capture data and maintain balance. The controller comprises two main components: a balance controller and a tracking controller. The balance controller uses a regulator designed for a simplified humanoid model to obtain the desired input to keep balance based on the current state of the robot. The simplified model is chosen so that a regulator can be designed systematically using, for example, optimal control. An example of such controller is a linear quadratic regulator designed for an inverted pendulum model. The desired inputs are typically the center of pressure and/or torques of some representative joints. The tracking controller then computes the joint torques that minimize the difference from desired inputs as well as the error from desired joint accelerations to track the motion capture data, considering exact full-body dynamics. We demonstrate that the proposed controller effectively reproduces different styles of storytelling motion using dynamics simulation considering limitations in hardware.

This paper presents a dynamics simulator that can handle complex robotic systems including position-controlled joints and closed kinematic chains. We first extend our prevous algorithm for linear-time forward dynamics algorithm to handle closed kinematic chains. The extended algorithm is formally presented for the first time. We then present another extension that allows position-controlled joints, whose angles exactly follow the reference by perfect servo controllers. This feature is often useful for simple trial simulations only using joint angle commands because the user does not have to design low-level servo controllers for simulation. The simulation can also be performed without precise friction parameters. The algorithm is tested on a humanoid robot having toe joints with four-bar linkage structure.

This paper presents image stabilization for microscopy using horizontal visual feedback control of the objective lens through a five-bar linkage and piezoelectric actuators, and its application to in vivo imaging. Even very small in vivo motion due to heartbeat and breathing makes microscopic observation difficult by blurring the microscope image or impossible by sending a region of interest out of view. In order to remove those unwanted effects of the motion, we have introduced motion-canceling robotic technologies into microscopy. Our image stabilization system through motion-canceling provides users with stabilized image sequences with respect to trembling of in vivo subjects. The developed image stabilization system, in term of robotics, corresponds to a visual feedback control system that consists of a robotic mechanism and a high-speed vision. A high-speed camera installed in the microscope detects the motion of the in vivo subject having topically applied fiducials. To virtually cancel this motion, we move the objective lens, synchronizing the motions of the subject and the lens to remove the relative motion between the two. As a result, we observe motion-free images to m. This technology is one of the very demanding technologies in biological research for in vivo observation with high resolution. In this paper, we verify the effectiveness of the developed system through in vivo experiments.

Motion capture systems are used to obtain motion data such that humanoid robots or computer graphics (CG) characters can behave naturally. However, it has proven to be hard not only to modify the capture data without losing its reality but also to search for the required capture data in a lot of capture data. In this paper, we provide a solution to these problems based on our previous work on symbolization of motion patterns for developing humanoid intelligence. Similar motion sequences in the database are abstracted as a symbol, which will be applied to searching motion patterns in the database similar to a given motion. This paper also introduces a method for building a stochastic symbol-word mapping model utilizing the word labels provided by the operator during motion capture sessions. This model converts a input sequence of words into a sequence of symbols, and then allows the capture database both to be searched for capture data corresponding to the input (a sequence of words) and to provide the users with new motion data generated by the symbols. Finally, we apply analogy of symbols to establishing the database in order to provide an appropriate motion data in response to an unsupervised sequence of words and then demonstrate the validity of analogy theory.

We present a solution to estimate in-vivo the joint dynamics of the human limbs during passive movements. The method is based on well-known modelling and approach used in Robotics that allow simultaneous multi-joint estimation. The modelling of the human body and the human joint as well as the method are described. The experimental set-up based on the use of an optical motion capture system is detailed. Three types of movements are recorded and used to perform the identification. We concluded that designed movements and movements from clinical diagnosis of neuromuscular diseases are good to perform the identification; however swing of the arms during normal walk does not provide enough excitation to obtain consistent results.

Communication skill is essential for social robots in various environments such as homes, offices, and hospitals, where the robots are expected to interact with humans. In this paper, we model the primitive nonverbal communication between two persons by mimetic communication model. The model consists of three groups of hidden Markov models (HMMs) hierarchically combined to recognize motions of the human and to generate the interactive motions of the robot. HMMs in the lower layer abstract the motion patterns and HMMs in the upper layer represent the interaction patterns. We demonstrate the validity of this model through kick boxing match between a motion-captured human and humanoid robot, where the robot can autonomously generate its motion in response to attacks by the human

Human upper limb joints dynamics is very important in the fields of humanoid robotics, medical robotics as well as medical research. To make human-like passive movements of the arms when walking humanoid robot arms must have similar dynamics to the human arms, even more if this arm is to be used as a prosthesis. Moreover medical diagnosis of muscle or neuro-motor diseases are based on a visual qualitative estimation of joint passive stiffness. There is a pressing need in human body dynamics characterization and especially in subject specific characterization. In this paper a solution to estimate in-vivo the passive dynamic of the arm joint is proposed. It is based on the use of the musculo-skeletal description of the human body and its kinematics computation. The linear passive joint dynamics: stiffness, viscosity and friction, is then estimated with least squares method. Acquisition of movements both designed for estimation or from medical diagnosis check-up, are achieved with motion capture studio only (no pain, no distress on subject). Experimental results for three valid subject are given

This paper presents a stable penalty-based model for simulating frictional contacts between many complex objects. The major advantage of our model is that it solves the problems in implementing Coulomb's friction model for computer simulation: iterative computation and slip velocity threshold. We also introduce a robust method for computing the normal vector and penetration depth at each contact point of a pair of interpenetrating polygonal objects. We demonstrate the validity and usability of the model by comparing the simulation results with closed-form solutions of Coulomb's friction model and conventional friction model, as well as performing dynamics simulation of highly complex scenes with tens of objects composed of thousands of polygons

This paper presents a method for capturing detailed human motion by using a suit covered with retroreflective mesh. We can attach huge number of markers on the subject without replacing the hardware of current passive optical motion capture systems. Compared to normal motion capture using spherical markers, the connectivity information of the mesh can be used to improve the efficiency and accuracy of the reconstruction process. As a result, the system can achieve faster and more precise measurement of hundreds of markers on the subject than other approaches such as using natural image or 3D scanning. The total computation time required to reconstruct the 3D mesh information including 408 markers (intersections) is 65.5 ms, allowing realtime motion capture at 15 fps.

The goal of this research is to enable precise estimation of human muscle forces in whole-body motions based not only on physiological muscle model but also on the equation of motion. The potential application areas include human-machine interface, medicine, biomechanics, and computer animation. Towards this goal, in this paper we discuss the inverse dynamics of musculoskeletal human model using the data from electromyogram (EMG) and force sensor. The inverse dynamics of musculoskeletal human models is formulated as an optimization problem subject to equality and inequality conditions taking into account of the equation of motion and muscle model from physiology literature. We evaluate the developed algorithm on a complex musculoskeletal model with 366 muscles driving a skeleton with 155 degrees of freedom.

Today optical motion capture system is becoming an essential tool for motion analysis, synthesis, and character animation. This paper focuses on improving the ability of current passive optical motion capture systems, which provides the highest precision and flexibility with the lowest interference among current motion capture technologies but has two major drawbacks. Firstly, it usually requires expensive post-processing computation including reconstruction and labeling. The second problem is that it is difficult to achieve both high precision and high frame rate at the same time due to the limitation of data transmission rate, that is, high-resolution cameras have low frame rate and vice versa. In this paper, we try to solve these problems by combining cameras of different types that complement the limitations of each others. The marker positions measured by the high-resolution cameras correct the low-precision data from high-speed cameras, which in turn helps real-time tracking of markers by inserting new data at higher frame rate. The proposed method is implemented on a PC cluster and experimental results show that we can obtain high-precision data even for high-speed motions. We also demonstrate the real-time joint angle computation using the real-time tracking capability.

Even such simple tasks as placing a box on a shelf are difficult to animate, because the animator must carefully position the character to satisfy geometric and balance constraints while creating motion to perform the task with a natural-looking style. In this paper, we explore an approach for animating characters manipulating objects that combines the power of path planning with the domain knowledge inherent in data-driven, constraint-based inverse kinematics. A path planner is used to find a motion for the object such that the corresponding poses of the character satisfy geometric, kinematic, and posture constraints. The inverse kinematics computation of the character's pose resolves redundancy by biasing the solution toward natural-looking poses extracted from a database of captured motions. Having this database greatly helps to increase the quality of the output motion. The computed path is converted to a motion trajectory using a model of the velocity profile. We demonstrate the effectiveness of the algorithm by generating animations across a wide range of scenarios that cover variations in the geometric, kinematic, and dynamic models of the character, the manipulated object, and obstacles in the scene.

In this paper, we present a method for controlling a motorized, string-driven marionette using motion capture data from human actors. The motion data must be adapted for the marionette because its kinematic and dynamic properties differ from those of the human actor in degrees of freedom, limb length, workspace, mass distribution, sensors, and actuators. This adaptation is accomplished via an inverse kinematics algorithm that takes into account marker positions, joint motion ranges, string constraints, and potential energy. We also apply a feedforward controller to prevent extraneous swings of the hands. Experimental results show that our approach enables the marionette to perform motions that are qualitatively similar to the original human motion capture data.

In this paper, we describe the concept and implementation of a dynamics filter, an online, full-body motion generator that converts a physically infeasible reference motion into a feasible one for the given human figure. Our implementation of the dynamics filter only uses time-local information, that is, does not require the whole motion sequence in advance. Therefore, the reference motion may be changed online in response to the interaction with a human or the environment. The dynamics filter is implemented based on an efficient rigid-body collision/contact model. This model itself provides an efficient algorithm for dynamics simulation of collisions and contacts. We demonstrate the power of the dynamics filter by several example motions that use motion capture data as a reference.

This paper presents a computational technique for creating whole-body motions of human and animal characters without reference motion. Our work enables animators to generate a natural motion by dragging a link to an arbitrary position with any number of links pinned in the global frame, as well as other constraints such as desired joint angles and joint motion ranges. The method leads to an intuitive pin-and-drag interface where the user can generate whole-body motions by simply switching on or off or strengthening or weakening the constraints. This work is based on a new interactive inverse kinematics technique that allows more flexible attachment of pins and various types of constraints. Editing or retargeting captured motion requires only a small modification to the original method, although it can also create natural motions from scratch. We demonstrate the usefulness and advantage of our method with a number of example motion clips.

This paper introduces an open architecture humanoid robotics platform (OpenHRP) on which various building blocks of humanoid robotics can be investigated. OpenHRP is a virtual humanoid robot platform with a compatible humanoid robot, and consists of a simulator of humanoid robots and motion control library for them which can also be applied to a compatible humanoid robot as it is. OpenHRP is expected to initiate the exploration of humanoid robotics on an open architecture software and hardware, due to the unification of the controllers and the examined consistency between the simulator and a real humanoid robot.

This paper presents the real time processing of optical motion capture with pan-tilt camera tracking. Pan-tilt camera tracking expands the range of capturing field dynamically. The asymmetrical marker distribution and polyhedra search algorithm realize robust labeling against missing markers. The algorithm is developed for parallel cluster computation and enables real time data processing. Experimental results demonstrate the effectiveness of the system.

An efficient parallel algorithm for forward dynamics computation of human figures is proposed. The algorithm is capable of handling any kinematic chains including structure-varying ones. The asymptotic complexity of the algorithm is O(N) in serial computation and O(log N) in parallel computation on O(N) processors for most practical kinematic chains. The idea is to assemble a kinematic chain by adding the joints one by one and compute the constraint forces at the new joints using the principle of virtual work. The parallelism of the algorithm can be adapted for parallel processing systems with any number of processors by simply changing the assembly order. Simulation examples on an 8-node cluster demonstrate the effectiveness of the algorithm.

Presents an interface for creating whole-body motions of human and animal characters without reference motion. Its basic function is to enable animators to generate a natural motion by dragging a link to an arbitrary position with any number of links pinned in the global frame, as well as other constraints such as desired joint angles and joint motion ranges. Each constraint can be switched on or off, strengthened or weakened for each joint at a user's will. The interface is based on an online inverse kinematics technique that allows more flexible attachment of pins and various types of constraints. Editing or retargeting captured motion requires only a small modification to the original method, although the method can create natural motions from scratch. We also demonstrate the power and usability of the proposed method by a number of example motion clips.

Presents a practical motion planner for humanoids and animated human figures. Modeling human motions as a sum of rigid body and cyclic motions, we identify body postures that represent the rigid-body part of typical motion patterns. This leads to a model of the configuration space that consists of a multi-layered grid, each layer corresponding to a single posture. A global search through this reduced configuration space yields a feasible path and the corresponding postures along the path. A velocity profile is calculated along the optimal path, subject to the speed and acceleration limits assumed for each posture. Cyclic motions, generated from "primitive" cyclic motion patterns for each posture, are then added to the trajectory produced by the path planner. This "kinematic" motion is then modified by a dynamics filter to result in dynamically consistent behavior. Examples are presented which demonstrate the use of this planner in an office environment.

This paper describes an efficient algorithm for the forward dynamics of open kinematic chains with O(N) complexity, where N is the number of links in the chain. The method is based on the principle of virtual work and does not use any theory in linear algebra or the concept of articulated body inertia. The idea of this method is to add a link one by one from the leaflinks to the root evaluating the constraint force at each new joint. The algorithm consists of two iterative procedures: from the leaflinks to the root to compute the constraint forces, and from the root to the leaf to compute the joint accelerations. Some numerical examples show the efficiency of the proposed algorithm. Similarity and differences with other O(N) algorithms are also discussed.

This paper presents a virtual humanoid robot platform (V-HRP for short) on which we can develop the identical controller for a virtual humanoid robot and its real counterpart. The unification of the controllers for the virtual and real robot has been realized by introducing software adapters for two robots respectively and employing ART-Linux on which real-time processing is available at the user level. Thanks to the unification, the controllers can share softwares with the dynamics simulator of V-HRP, including the parameter parser, kinematics and dynamics computations and the collision detector. This feature can make the development of the controllers more efficient and the developed controllers more reliable.

Humanoid robots are required to make a variety of dynamics and even expressive motions in changing environments. However, the conventional methods for generating humanoid motions fail do achieve this requirement since they can only generate quite artificial and predefined motions through rather complicated optimization processes. In this paper, we propose the concept of "dynamics filter" which transforms a physically inconsistent motion into a consistent one, and provide an example of its implementation using feedback control and local optimization. The optimization is based on the equation of motion of constrained kinematic chains, which is derived from our previously proposed method for computing the dynamics of structure-varying kinematic chains. The proposed method can be applied to online motion generator of humanoid robots.

This paper discusses the dynamics computation of structure-varying kinematic chains which imply mechanical link systems whose structure may change from open kinematic chain to closed one and vice versa. The proposed algorithm can handle and compute the dynamics and motions of any rigid link systems in a seamless manner without switching among algorithms. The computation is developed on the foundation of the dynamics computation algorithms established in robotics, which is superior in efficiency due to explicit use of the generalized coordinates to those used in the general-purpose motion analysis softwares. Although the structure-varying kinematic chains are commonly found in computing human and animal motions, the computation of their dynamics has not been discussed in literature. The developed computation will provide a general algorithm for the computation of motion and control of humanoid robots and computer graphics human figures.

Discusses the dynamics computation of structure-varying kinematic chains which imply mechanical link systems whose structure may change from open kinematic chain to closed one and vice versa. The proposed algorithm can handle structure changes in a seamless manner without switching among algorithms for different kinematic chains. The structure-varying kinematic chains are commonly found in computing human motions. The developed computation will provide the general algorithm for the computation of motion and control of humanoid robots and computer graphic human figures.

This paper discusses the dynamics computation of closed kinematic chains, especially those found in motions of human figures. A number of efficient dynamics computation algorithms have been established in robotics for open kinematic chains and particular types of closed kinematic chains such as parallel five-bar link mechanisms and the Stewart platform. The dynamics computation of closed kinematic chains, however is still challenging and among open research issues. In this paper, we describe the mobility of closed kinematic chains by the minimal set of independent variables, which we call the generalized coordinates of a closed kinematic chain. We then develop a systematic procedure to find them out, and establish the computational algorithms for the inverse and forward dynamics of any closed kinematic chains. The numerical examples show the effectiveness of the algorithms in particular for computing high-degrees-of-freedom human/animal motions.

We propose a control scheme of parallel manipulators focusing on the accuracy of acceleration on the endplate, which is an important factor when parallel manipulators are used as acceleration displays. We use two controllers-a dynamic controller to achieve accuracy of position and to stabilize the system, and an H/sub /spl infin// controller to feedback the acceleration measured on the endplate. The main problem of dynamic control is computational complexity. In order to reduce computation time for inverse dynamics, a parallel processing method called multi-thread programming is applied. The H/sub /spl infin// controller is added outside the closed loop of the dynamic control to remove the vibration of the structure and the influence of modeling errors in the dynamic controller.