This paper revisits the numerical inverse kinematics (IK) problem, leveraging modern computational resources and refining the seed selection process to develop a solver that is competitive with analytical-based methods. The proposed method discretizes the robot configuration space via Centroidal Voronoi Tessellation (CVT) indexed in a KDTree, ranks candidate joint configurations by minimal joint-space adjustment, and iteratively re-attempt with the next-nearest seeds in pose space according to CVT Voronoi distance. The joint space adjustment-based seed selection increases the likelihood of rapid convergence, while the re-attempt strategy effectively helps circumvent local minima and joint limit constraints. Comparison results with both traditional numerical solvers and learning-based methods demonstrate the strengths of the proposed approach in terms of success rate, time efficiency, and accuracy. Additionally, we conduct detailed ablation studies to analyze the effects of various parameters and solver settings, providing practical insights for customization and optimization. The proposed method consistently exhibits high success rates and computational efficiency. It is suitable for time-sensitive applications.

Robotic grasping is facing a variety of real-world uncertainties caused by non-static object states, unknown object properties, and cluttered object arrangements. The difficulty of grasping increases with the presence of more uncertainties, where commonly used learning-based approaches struggle to perform consistently across varying conditions. In this study, we integrate the idea of similarity matching to tackle the challenge of grasping novel objects that are simultaneously in motion and densely cluttered using a single RGBD camera, where multiple uncertainties coexist. We achieve this by shifting visual detection from global to local states and operating grasp planning from static to dynamic scenes. Notably, we introduce optimization methods to enhance planning efficiency for this time-sensitive task. Our proposed system can adapt to various object types, arrangements and movement speeds without the need for extensive training, as demonstrated by real-world experiments.

In assembly tasks, multiple operations, such as positioning, snap-fitting, and screw fastening, are often required for a single workpiece. The multiple operations add complexity to the planning process. To address this challenge, we propose an assembly sequence planning method that considers the combination of multiple operations associated with each workpiece. We define the sequence of these operations as a "workflow" and search for an optimal assembly sequence while respecting the workflow constraints of the workpieces. Beyond handling multi-operation constraints, our method optimizes the robot’s motion costs by assigning weights to the search tree and minimizing these costs accordingly. To evaluate the effectiveness of the proposed approach, we compare assembly planning results for multi-operation tasks with and without workflow decomposition. Additionally, we analyze the influence of motion cost minimization on planning performance and computational efficiency. Experimental results verified the effectiveness of the proposed method in improving assembly planning efficiency.

Diffusion models demonstrate superior performance in capturing complex distributions from large-scale datasets, providing a promising solution for quadrupedal locomotion control. However, the robustness of the diffusion planner is inherently dependent on the diversity of the pre-collected datasets. To mitigate this issue, we propose a two-stage learning framework to enhance the capability of the diffusion planner under limited dataset (reward-agnostic). Through the offline stage, the diffusion planner learns the joint distribution of state-action sequences from expert datasets without using reward labels. Subsequently, we perform the online interaction in the simulation environment based on the trained offline planner, which significantly diversified the original behavior and thus improves the robustness. Specifically, we propose a novel weak preference labeling method without the ground-truth reward or human preferences. The proposed method exhibits superior stability and velocity tracking accuracy in pacing, trotting, and bounding gait under different speeds and can perform a zero-shot transfer to the real Unitree Go1 robots. The project website for this paper is at https://shangjaven.github.io/preference-aligned-diffusion-legged/.

Grasping unknown objects from a single view has remained a challenging topic in robotics due to the uncertainty of partial observation. Recent advances in large-scale models have led to benchmark solutions such as GraspNet-1Billion. However, such learning-based approaches still face a critical limitation in performance robustness for their sensitivity to sensing noise and environmental changes. To address this bottleneck in achieving highly generalized grasping, we abandon the traditional learning framework and introduce a new perspective: similarity matching, where similar known objects are utilized to guide the grasping of unknown target objects. We newly propose a method that robustly achieves unknown-object grasping from a single viewpoint through three key steps: 1) Leverage the visual features of the observed object to perform similarity matching with an existing database containing various object models, identifying potential candidates with high similarity; 2) Use the candidate models with pre-existing grasping knowledge to plan imitative grasps for the unknown target object; 3) Optimize the grasp quality through a local fine-tuning process. To address the uncertainty caused by partial and noisy observation, we propose a multi-level similarity matching framework that integrates semantic, geometric, and dimensional features for comprehensive evaluation. Especially, we introduce a novel point cloud geometric descriptor, the C-FPFH descriptor, which facilitates accurate similarity assessment between partial point clouds of observed objects and complete point clouds of database models. In addition, we incorporate the use of large language models, introduce the semi-oriented bounding box, and develop a novel point cloud registration approach based on plane detection to enhance matching accuracy under single-view conditions. Real-world experiments demonstrate that our proposed method significantly outperforms existing benchmarks in grasping a wide variety of unknown objects in both isolated and cluttered scenarios, showcasing exceptional robustness across varying object types and operating environments.

Deep learning methods have recently exhibited impressive performance in object detection. However, such methods needed much training data to achieve high recognition accuracy, which was time-consuming and required considerable manual work like labeling images. In this paper, we automatically prepare training data using robots. Considering the low efficiency and high energy consumption in robot motion, we proposed combining robotic in-hand observation and data synthesis to enlarge the limited data set collected by the robot. We first used a robot with a depth sensor to collect images of objects held in the robot’s hands and segment the object pictures. Then, we used a copy-paste method to synthesize the segmented objects with rack backgrounds. The collected and synthetic images are combined to train a deep detection neural network. We conducted experiments to compare YOLOv5x detectors trained with images collected using the proposed method and several other methods. The results showed that combined observation and synthetic images led to comparable performance to manual data preparation. They provided a good guide on optimizing data configurations and parameter settings for training detectors. The proposed method required only a single process and was a low-cost way to produce the combined data. Interested readers may find the data sets and trained models from the following GitHub repository: github.com/wrslab/tubedetNote to Practitioners—The background of this study is a requirement in lab automation – Using robots to arrange randomly placed tubes automatically. Before sending test tubes to an examination machine for gradient tests, humans need to categorize and organize the tubes into specific patterns to fit the machine’s internal design. Employing humans is difficult as the tube arrangement requirements are time-varying. A preferred solution is using robots to replace humans. The robots should have a vision system to detect the tubes and a manipulation system to perform physical arranging actions. They will be used in busy seasons while deployed for other tasks in leisure time. Deep neural networks like YOLO are effective for the tube detection task. However, preparing the training data is challenging and unsuitable for lab end users. Pre-trained neural networks are options but have limited tube detection ability and cannot deal with newly included tube types. The method developed in this work helps solve the training data preparation problem. With its support, the robot can automatically prepare training data that has comparable quality to manually labeled ones in a single-process and low-cost way.

This paper presented a regrasp planning method to eliminate grasp uncertainty while considering the geometric constraints of a fixture. The method automatically finds the Stable Placement Poses (SPPs) of an object on a Triangular Corner Fixture (TCF), elevates the object from its SPPs to dropping poses and finds the Deterministic Dropping Poses (DDPs), builds regrasp graphs by using the SPP-DDP pairs and their associated grasp configurations, and searches the graph to find regrasp motion sequences for precise assembly. Since the SPPs and their associated regrasps are constrained by the TCF’s geometry and have high precision, the final object poses regrasped via it has low uncertainty and can be directly used for assembly by position control. In the experimental section, we study the performance of analytical and learning-based methods for estimating the DDPs of different objects and quantitatively examine the proposed method’s ability to suppress uncertainty using assembly tasks like peg-in-hole insertion and sheathing tubes, aligning holes, mounting bearing housings, etc. The results demonstrate the method’s robustness and efficacy. Note to Practitioners—In production lines, robots interact with peripheral devices to improve efficiency and reduce uncertainty. In this work, we focus on a particular peripheral device – a Triangular Corner Fixture (TCF) made by three inclined and mutually perpendicular plates. We study using the TCF to improve manipulation precision. The inclined plates of the TCF form a gravity bucket that holds dropped objects in stable states under gravity. In a real scenario, a robot picks up an object and releases it above the TCF. The released object will reach a stable state on the TCF. Then, the robot regrasps and moves the stabilized object to the target pose with reduced uncertainty. Using the method proposed in this paper, a robot can automatically finish the above procedure by finding all the object’s stable states in the TCF, planning grasp configurations, invalidating infeasible states and grasps, building regrasp graphs and searching the graph to find a regrasp motion sequence that moves the object to a goal pose with high precision for assembly. In industrial applications, the proposed method has the potential to improve the flexibility of robotic systems for high-precision tasks. In the research fields, it may promote the research on sensorless manipulation and extrinsic manipulation, and push forward the studies in robotic regrasp.

Elasto-plastic metal wire curving task is commonly seen in manufacturing and medical fields. This paper presents a combined task and motion planner (TAMP) for a robot arm to work aside a bending machine and carry out 3D metal wire curving tasks. We assume a collaborative robot that is safe for humans but has a weak payload and develop the combined planner for the robot to use the bending machine. The contributions of the study are three-fold. First, we propose a coarse-to-fine optimization-based method to convert a 3D curve to a structured bending set. Second, we build a planner to generate the feasible bending sequence, machine operation, robotic grasp poses, and arm motion while considering constraints from the bending machine and the robot. Third, we use visual feedback to build and dynamically update the springback model of a metal wire and use the model to predict and compensate for bending errors caused by springback. Compared with previous work, the proposed planner does not require the robot arm to have a large payload, making it suitable for lightweight collaborative robots. We evaluate the system using both simulated and real-world 3D curving tasks. The results show that the proposed planner can solve robotic 3D curving problems with satisfying time efficiency and precision. It is flexible and applicable to different robots and metal wire materials without a significant change. The method is expected to accelerate the high-variation low-volume manufacture of 3D metal wire curves.Note to Practitioners—Using robots to bend metal wires has been an old topic in robotics and automation. In previous robotic metal wire bending systems, the robot motion was usually pre-programmed to feed parts to bending machines. It was not easy to be extended to multiple goal shapes. Also, the bending was limited to a few discrete action points instead of an arbitrary curve. The method developed in this paper solved the problem by adding up optimized goal shape parameterization, combined task and motion planning, and springback compensation. It helps to auto-program robot actions and ensures satisfying precision by correcting bending results online with visual feedback. Practitioners are encouraged to use the planner for either offline programming or online motion generation. The springback estimation and compensation are independent of motion generation and can be connected to both motions pre-programmed offline or generated online. However, it is advisable to employ robots with higher DoFs (Degree of Freedom) and avoid the online planning of curves with many bending actions.

This paper presents a system integration approach for a 6-DoF (Degree of Freedom) collaborative robot to operate a pipette for liquid dispensing. Its technical development is three-fold. First, we designed an end-effector for holding and triggering manual pipettes. Second, we took advantage of direct teaching to specify global labware poses and planned robotic motion based on them. Third, we leveraged hand-mounted cameras and visual classifiers to predict and correct positioning errors, which allowed precisely attaching pipettes and tips without calibration. Through experiments and analysis, we confirmed that the developed system, especially the planning and visual recognition methods, could help secure high-precision and flexible liquid dispensing. The developed system is suitable for low-frequency, high-repetition biochemical liquid dispensing tasks. We expect it to promote the deployment of collaborative robots for laboratory automation and thus improve the experimental efficiency without significantly customizing a laboratory environment.Note to Practitioners—The proposed system helps to automate low-frequency, high-repetition biochemical experiments using a vertical articulated robot with uncalibrated hand-mounted cameras and TCP (Tool Center Point). It can be quickly deployed in tight lab spaces or beside existing automation instruments for experiments. Implementing the system does not require high-quality cameras or precise manufacturing. The system relies on software, particularly vision compensation, and the flexibility provided by unfixed racks to successfully perform pipetting tasks. We particularly recommend using collaborative robots to implement the proposed system. Collaborative robots typically meet safety requirements and do not need to be enclosed in a cage. This allows them to be deployed in the same workspace as human researchers, enabling efficient lab experiments. Human researchers can prepare tips and microwell plates and place them in front of the robot with a certain degree of freedom, which is less burdensome than traditional automatic devices that require precise placement of tips and plates in specified positions. Practitioners can view an example of the robot working with a plant phenotyping system for screening chemicals in the supplementary video.

This paper presents a stiffness-changeable soft finger using chain mail jamming. This finger can achieve adaptive grasping and in-hand manipulation by reshaping and exerting changeable gripping force. The jamming phenomenon happens when particles in a chamber get interlocked where confining pressure is exerted at their boundaries, which is widely used to construct mechanisms with changeable stiffness. Compared with the traditional granular media, chain mail has a lower packing fraction and provides a stronger tensile force. In this paper, we proposed to apply chain mail jamming to the field of robotic finger design. Especially, we propose the design of the finger, the fabrication process, the method of predicting gripping force, and the grasping strategies. The experiments quantitatively verify the model of gripping force prediction. The demonstrations validate the advantages of adaptive grasp by picking a variety of items including foods, goods, and industrial components, and show the application of in-hand manipulation.

To safely and efficiently extract an object from the clutter, this paper presents a bimanual manipulation planner in which one hand of the robot is used to slide the target object out of the clutter while the other hand is used to support the surrounding objects to prevent the clutter from collapsing. Our method uses a neural network to predict the physical phenomena of the clutter when the target object is moved. We generate the most efficient action based on the Monte Carlo tree search. The grasping and sliding actions are planned to minimize the number of motion sequences to pick the target object. In addition, the object to be supported is determined to minimize the position change of surrounding objects. Experiments with a real bimanual robot confirmed that the robot could retrieve the target object, reducing the total number of motion sequences and improving safety.

A vacuum lifter is widely used to hold and pick up large, heavy, and flat objects. Conventionally, when using a vacuum lifter, a human worker watches the state of a running vacuum lifter and adjusts the object’s pose to maintain balance. In this work, we propose using a dual-arm robot to replace the human workers and develop planning and control methods for a dual-arm robot to raise a heavy plate with the help of a vacuum lifter. The methods help the robot determine its actions by considering the vacuum lifer’s suction position and suction force limits. The essence of the methods is two-fold. First, we build a Manipulation State Graph (MSG) to store the weighted logical relations of various plate contact states and robot/vacuum lifter configurations, and search the graph to plan efficient and low-cost robot manipulation sequences. Second, we develop a velocity-based impedance controller to coordinate the robot and the vacuum lifter when lifting an object. With its help, a robot can follow the vacuum lifter’s motion and realize compliant robot-vacuum lifter collaboration. Real-world experiments are carried out to investigate the proposed planning and control methods. The results show that a robot can effectively and flexibly work together with a vacuum lifter to manipulate large and heavy plate-like objects with the methods’ support. Note to Practitioners—This paper is motivated by the vacuum lifters used for transporting heavy plates in a factory that produces building materials. In the factory, a human worker attaches the suction cup of a vacuum lifter to a plate and controls the vacuum lifter to pull the plate up. Meanwhile, another human worker moves and lifts the plate to a goal pose, following the vacuum lifter while maintaining balance. The job is dangerous as the plate is heavy, and the vacuum lifter is not always strong enough to hold the plate firmly. Inspired by the usage and safety problem, we in this paper develop a planning and control method for a dual-arm robot to replace humans. The robot coordinates its motion to work with the vacuum lifter and performs lifting tasks. The vacuum lifter could remain operated by a human worker or be actuated by signals from the dual-arm robot or other third-party machines. The work is complementary to our previous study that developed planners for robots to use pulley blocks. They together provide extensive knowledge for using low-payload collaboratively robots to manipulate heavy plates.

When humans want to understand an object’s 3D shape, they watch the object from different viewpoints. Changing the viewpoint is either performed actively, i.e., moving eye sights or the human head, or passively, i.e., holding and reposing the object. Inspired by the humans’ passive policy, we propose a method to plan the motion for a dual-arm robot to hold and repose an object, capture multiple views using a stationary depth sensor mounted on the robot head, and obtain the object’s 3D shape from the multiple views. Primarily, we develop algorithms to determine the Next Best Configuration (NBC) for observation and Next Best Regrasp/Grasp (NBR/G) poses while considering elements like the confidence of captured partial point clouds, robotic manipulability, robotic motion distances, and sensing ranges. We study the necessity and influence of these elements on the time costs and surface coverage quality in the experimental section using several representative objects. The results show that the elements play essential roles in supporting specific actions or suppressing certain costs. They help to secure efficient robot motion and satisfactory 3D shape recovery quality.Note to Practitioners—This paper is motivated by the difficulties in using commercial 3D scanners. A commercial 3D scanner set usually comprises a scanning sensor, a rotating table, and editing software. To scan the 3D shape of an object, a human needs to place the object on the rotating table with different poses, let the scanner obtain several partial point clouds, and use the editing software to merge them into a final model. The human must carefully design the different poses by considering both the object’s self obstructions and stable placements, which is tiring and difficult to be applied to large-scale tasks like building 3D shape databases containing many objects. On the other hand, although several robotic solutions exist for automatic scanning, they either use an eye-in-hand scanner to scan a stationary object or an arm to hold and move an object for scanning. In the former case, the bottom or downward faces of the object cannot be covered. In the latter case, the surface blocked by the fingers of the holding hand will be lost. The method proposed by this paper plans dual-arm robot motion to grasp and move objects for scanning. It automatically determines pick-up, rotation, and handover to maximize scanning coverage. Compared with commercial scanners and existing robotic solutions, the method performs automatic scanning with high coverage and is more advantageous for scanning many objects without human intervention.

This paper presents a combined task and motion planner for a robot arm to carry out 3D metal wire curving tasks by collaborating with a bending machine. We assume a collaborative robot that is safe to work in a human environment but has a weak payload to bend objects with large stiffness, and developed a combined planner for the robot to use a bending machine. Our method converts a 3D curve to a bending set and generates the feasible bending sequence, machine usage, robotic grasp poses, and pick-and-place arm motion considering the combined task and motion level constraints. Compared with previous deformable linear object shaping work that relied on forces provided by robotic arms, the proposed method is suitable for the material with high stiffness. We evaluate the system using different tasks. The results show that the proposed system is flexible and robust to generate robotic motion to corporate with the designed bending machine.

When generating robot motions based on instructions such as cooking recipes, ambiguity of the instructions and lack of necessary information are problematic for the robot. To solve this problem, we propose an efficient motion planning approach for a dual-arm robot by constructing a graph repre-senting a motion sequence based on a recipe consisting of verbal instructions and cooking images. A functional unit is generated based on the linguistic instructions in the recipe. Since most recipes lack the necessary information for executing the motion, we first consider extracting the information about the cooking motion like cutting from the food images of the recipe and supplementing it. In addition, to supplement the actions that humans perform unconsciously, we generate functional units for actions not explicitly mentioned in the recipe based on the current situation of the cooking process, and then connect them to the functional units generated from the recipe. Moreover, during the connection we consider the motion of the robot's arms in parallel for an efficient execution of the recipe, similar to those of a human. Through experiments, we demonstrate that for a given recipe, the proposed method can be used to generate a cooking sequence with the supplementary information needed, and executed by a dual-arm robot. The results show that the proposed method is effective and can simplify robot teaching in cooking tasks.

The work is inspired by the assembly of Soma block puzzles. Soma block puzzles usually include unstable intermediate states that require additional support to maintain stability temporarily. In the puzzles’ solution manual, we can observe that designers consider the characteristics that humans have two hands and can avoid an unstable intermediate state by using one hand to support the finished component and using the other hand to assemble an upcoming workpiece. Motivated by human behavior, this paper develops a planner that automatically finds an optimal assembly sequence for a dual-arm robot to build a woodblock structure while considering various constraints and supporting grasps from a second hand. It uses the mesh model of wood blocks and the final assembly state to generate possible assembly sequences and evaluate the optimal assembly sequence by considering the stability, graspability, assemblability, and the need for a second hand. Especially, the need for a second hand is resolved when supports from worktables and other workpieces are not enough to produce a stable assembly. A second hand can hold and support the unstable components so that the robot can further assemble new workpieces until the structure state becomes stable again. The output of the planner includes the optimal assembly orders, candidate grasps, assembly directions, and the supporting grasps (if needed). The output can help guide a dual-arm robot to perform motion planning and thus generate assembly motion. Experiments using various blocks and structures show the effectiveness of the proposed planner. Note to Practitioners—The presented planner can generate an optimal assembly order for a large variety of structures like decoration accessories, furniture, home interiors, frames, etc., in the practices. They can also be used for scenarios that need stacking or piling up multiple objects. The generated optimal assembly order is more friendly to dual-arm robot systems than previous assembly planners that ignored the merits of robotic collaboration. Also, the proposed assembly planner generates the necessary information for the motion planner, such as grasp poses and optimal assembly directions. A motion planner can directly use the generated results to plan robotic assembly motion.. The proposed assembly planner is expected to significantly reduce human effort and increase the efficiency of robotic assembly lines.

This paper proposes a combined planning and optimization method that enables a dual-arm robot to lift up and flip heavy plates using crane pulley blocks. The problem is motivated by the low payload of modern collaborative robots. Instead of directly manipulating heavy plates that collaborative robots cannot afford, the paper develops a planner for collaborative robots to operate crane pulley blocks. The planner assumes a target plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot’s dual-arm motion that pulls the rope of the crane pulley blocks to lift up the plate. The crane pulley blocks reduce the payload that each robotic arm needs to bear. When the plate is lifted up to a satisfying pose, the planner plans a sliding-pushing motion for one of the robot arms to tumble over the plate while considering force and moment constraints. The article presents the technical details of the planner and several experiments and analysis carried out using a dual-arm robot made by two Universal Robots UR3 arms. The influence of various parameters and optimization goals are investigated and compared in depth. The results show that the proposed planner is flexible and efficient. This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets. Note to Practitioners—This paper is motivated by a cleaning process in a factory that produces sewage press machines. The pressboard of sewage press machines could be as heavy as 1000 kg. Human workers need to flip and clean both sides of the board before installing them to the main axis of a sewage machine. Their solution is using a gantry crane. They attach the board to the crane hook using bearing belts, activate the crane to lift up the board. When the board is raised to a satisfying pose, the workers turn the board over by pushing it. Motivated by human workers’ actions, we developed the planner presented in this paper. We assumed crane pulley blocks in the experiments and analysis, but in practice, they may be replaced with electronic ones to improve effort and efficiency. Using the electronic ones will be a sub-problem since there is no need for pulling ropes. The proposed method is expected to help a company’s technicians better judge if they need a heavy payload manipulator or keep their current crane equipment while employing several intelligent collaborative robots to operate them. As a result, it may help to accelerate the upgrade of manufacturing sites while reducing reforming budgets.

This article develops a mechanical screwing tool and its manipulation policies for two-finger parallel robotic grippers. The tool is based on a combined scissor-like element (SLE) and double-ratchet mechanism that converts the gripping motion of two-finger parallel grippers into a continuous rotation to realize tasks like fastening screws. The tool is entirely mechanical. There is no need for external cable connections. The manuscript includes two parts. For one thing, it shows the details of the tool design, optimizes the tool’s dimensions and effective stroke lengths, and studies the contacts and forces to achieve stable grasping and screwing. For another, it presents the related manipulation and control policies, including recognizing the tool, changing tool poses, and completing screw fastening tasks. The designed tool, together with the related manipulation and control policies, are analyzed and verified in several real-world applications. The results show that the tool has satisfying mechanical properties. Robots with parallel grippers can robustly and flexibly use the tool to fasten screws. The tool can also be used collaboratively with other tools to finish difficult tasks. In the future, similar tools are expected to replace special-purpose end-effectors or tool changers for more flexible robot integration.

This article develops a flexible and robust robotic system for autonomously drawing on 3-D surfaces. The system takes 2-D drawing strokes and a 3-D target surface (mesh or point clouds) as input. It maps the 2-D strokes onto the 3-D surface and generates a robot motion to draw the mapped strokes using visual recognition, grasp pose reasoning, and motion planning. The system is flexible compared to conventional robotic drawing systems as we do not fix drawing tools to the end of a robot arm. Instead, a robot recognizes and picks up pens online and holds the pens to draw 3-D strokes. Meanwhile, the system has high robustness thanks to the following crafts: First, a high-quality mapping method is developed to minimize deformation in the strokes. Second, visual detection is used to reestimate the drawing tool’s pose before executing each drawing motion. Third, force control is employed to compensate for noisy visual detection and calibration and ensure a firm touch between the pen tip and the surface. Fourth, error detection and recovery are implemented to deal with slippage and other anomalies. The planning and executions are performed in a closed-loop manner until the strokes are successfully drawn. We evaluate the system and analyze the necessity of the various crafts using different real-world tasks. The results show that the proposed system is flexible and robust to generate robotic motion that picks up the pens and successfully draws 3-D strokes on given surfaces.

Robotic assembly tasks involve complex and low-clearance insertion trajectories with varying contact forces at different stages. While the nominal motion trajectory can be easily obtained from human demonstrations through kinesthetic teaching, teleoperation, simulation, among other methods, the force profile is harder to obtain especially when a real robot is unavailable. It is difficult to obtain a realistic force profile in simulation even with physics engines. Such simulated force profiles tend to be unsuitable for the actual robotic assembly due to the reality gap and uncertainty in the assembly process. To address this problem, we present a combined learning-based framework to imitate human assembly skills through hybrid trajectory learning and force learning. The main contribution of this work is the development of a framework that combines hierarchical imitation learning, to learn the nominal motion trajectory, with a reinforcement learning-based force control scheme to learn an optimal force control policy. To further improve the imitation learning part, we develop a hierarchical architecture, following the idea of goal-conditioned imitation learning, to generate the trajectory learning policy on the skill level offline. Through experimental validations, we corroborate that the proposed learning-based framework is robust to uncertainty in the assembly task, can generate high-quality trajectories, and can find suitable force control policies, which adapt to the task’s force requirements more efficiently.

This article presents a novel software architecture enabling the analysis of assembly actions from fine-grained hand motions. Unlike previous works that compel humans to wear ad-hoc devices or visual markers in the human body, our approach enables users to move without additional burdens. Modules developed are able to: (i) reconstruct the 3D motions of body and hands keypoints using multi-camera systems; (ii) recognize objects manipulated by humans, and (iii) analyze the relationship between the human motions and the manipulated objects. We implement different solutions based on OpenPose and Mediapipe for body and hand keypoint detection. Additionally, we discuss the suitability of these solutions for enabling real-time data processing. We also propose a novel method using Long Short-Term Memory (LSTM) deep neural networks to analyze the relationship between the detected human motions and manipulated objects. Experimental validations show the superiority of the proposed approach against previous works based on Hidden Markov Models (HMMs).

This paper presents a motion planning algorithm that enables robots to efficiently pick up objects by considering simultaneous multi-object grasping. At the center of the algorithm is a cost function that helps to determine one of the following three grasping policies considering distance and friction constraints – Grasping a single object; Grasping two objects simultaneously; Grasping two object simultaneously after pushing one of the objects close to the other. After recognizing the object distributions on a table by using a depth camera and Mask R-CNN, our algorithm will select grasp policies from the three candidates considering the cost function, and plan a policy sequence that can most quickly finish picking all the objects using dynamic programming. Both simulation and real-world experiments are carried out to examine the performance of the proposed planner. Results show that the proposed method significantly improves the efficiency of robotic picking compared to conventional single-object-based methods.

This paper proposes a robot assembly planning method by automatically reading the graphical instruction manuals designed for humans. Essentially, the method generates an Assembly Task Sequence Graph (ATSG) by recognizing a graphical instruction manual. An ATSG is a graph describing the assembly task procedure by detecting types of parts included in the instruction images, completing the missing information automatically, and correcting the detection errors automatically. To build an ATSG, the proposed method first extracts the information of the parts contained in each image of the graphical instruction manual. Then, by using the extracted part information, it estimates the proper work motions and tools for the assembly task. After that, the method builds an ATSG by considering the relationship between the previous and following images, which makes it possible to estimate the undetected parts caused by occlusion using the information of the entire image series. Finally, by collating the total number of each part with the generated ATSG, the excess or deficiency of parts are investigated, and task procedures are removed or added according to those parts. In the experiment section, we build an ATSG using the proposed method to a graphical instruction manual for a chair and demonstrate the action sequences found in the ATSG can be performed by a dual-arm robot execution. The results show the proposed method is effective and simplifies robot teaching in automatic assembly.

This article develops model-based grasp planning algorithms. It focuses on industrial end-effectors like grippers and suction cups, and plans grasp configurations considering computer aided design (CAD) models of target objects. The developed algorithms can stably find many high-quality grasps, with satisfying precision and little dependency on the quality of CAD models. The undergoing core technique is superimposed segmentation, which preprocesses a mesh model by peeling it into superimposed facets. The algorithms use the facets to locate contacts and synthesize grasp poses for popular industrial end-effectors. Several tunable parameters are prepared to adapt the algorithms to meet various requirements. The experimental section studies the influence of the tunable parameters and analyzes the cost, precision, and robustness of the proposed algorithms and their planned grasps, with both simulations and real-world systems. Besides, the proposed algorithms are applicable to mesh models reconstructed from point clouds obtained by depth sensors. Some experiments and analysis are also carried out to study and demonstrate the ability.

In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a sequence of base positions that reduce the number of necessary base movements for collecting the target objects, the base sequence is robust in that the mobile manipulator is able to complete the part-supply task even there is certain deviation from the planned base positions. Our experiments demonstrate both the efficiency compared to regular base sequence and the feasibility in real world applications.

Synergy provides a practical approach for expressing various postures of a multi-fingered hand. However, a conventional synergy defined for reproducing grasping postures cannot perform in-hand manipulation, e.g., tasks that involve simultaneously grasping and manipulating an object. Locking the position of particular fingers of a multi-fingered hand is essential for in-hand manipulation tasks either to hold an object or to fix unnecessary fingers. When using conventional synergy based control to manipulate an object, which requires locking some fingers, the coordination of joints is heavily restricted, decreasing the dexterity of the hand. We propose a functionally divided manipulation synergy (FDMS) method, which provides a synergy-based control to achieves both dimensionality reduction and in-hand manipulation. In FDMS, first, we define the function of each finger of the hand as either "manipulation" or "fixed." Then, we apply synergy control only to the fingers having the manipulation function, so that dexterous manipulations can be realized with a few control inputs. Furthermore, we propose the Synergy Switching Framework as a method for applying a finely defined FDMS to sequential task changes. The effectiveness of our method is experimentally verified.

This article develops a robotic manipulation planner for human-robot collaborative assembly. Unlike previous methods that study an independent and fully artificial intelligent (AI)-equipped autonomous system, this article explores the subtask distribution between a robot and a human and studies a human-in-the-loop robotic system for collaborative assembly. The system distributes the subtasks of an assembly to robots and humans by exploiting their advantages and avoiding their disadvantages. The robot in the system will work on pick-and-place tasks and provide workpieces to humans. The human collaborator will work on fine operations, such as aligning, fixing, and screwing. A constraint-based incremental manipulation planning method is proposed to generate the motion for the robots. The performance of the proposed system is demonstrated by asking a human and the dual-arm robot to collaboratively assemble a cabinet. The results show that the proposed system and planner are effective and efficient and can assist humans in finishing the assembly task comfortably.

This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed algorithm could plan single- and dual-arm manipulation sequences to solve the problems. The mechanism under the planner is a regrasp graph, which encodes grasp configurations and object poses. The algorithms search the graph to find a sequence of robot motion to reorient objects. The planner is able to plan both single- and dual-arm manipulation. It could also automatically determine whether to use a single arm, dual arms, or their combinations to finish given tasks. The planner is examined by various humanoid robots like Nextage, HRP2Kai, HRP5P, etc., using both simulation and real-world experiments.

In this research, we tackle the challenge of picking only one object from a randomly stacked pile where the objects can potentially be tangled. No solution has been proposed to solve this challenge due to the complexity of picking one and only one object from the bin of tangled objects. Therefore, we propose a method for avoiding the situation where a robot picks multiple objects. In our proposed method, first, grasping pose candidates are computed by using the graspability index. Then, a Convolutional Neural Network (CNN) is trained to predict whether or not the robot can pick one and only one object from the bin. Additionally, since a physics simulator is used to collect data to train the CNN, an automatic picking system can be built. The effectiveness of the proposed method is confirmed through experiments on robot Nextage and compare with previous bin-picking methods.

Planning dual-arm assembly of more than three objects is a challenging Task and Motion Planning (TAMP) problem. The assembly planner shall consider not only the pose constraints of objects and robots, but also the gravitational constraints that may break the finished part. This paper proposes a planner to plan the dual-arm assembly of more than three objects. It automatically generates the grasp configurations and assembly poses, and simultaneously searches and backtracks the grasp space and assembly space to accelerate the motion planning of robot arms. Meanwhile, the proposed method considers gravitational constraints during robot motion planning to avoid breaking the finished part. In the experiments and analysis section, the time cost of each process and the influence of different parameters used in the proposed planner are compared and analyzed. The optimal values are used to perform real-world executions of various robotic assembly tasks. The planner is proved to be robust and efficient through the experiments.

This paper explores the use of robots to autonomously assemble parts with variations in colors and textures. Specifically, we focus on peg-in-hole assembly with some initial position uncertainty and holes located on surfaces of different colors and textures. Two in-hand cameras and a force-torque sensor are used to account for the position uncertainty. A program sequence comprising learning-based visual servoing, spiral search, and impedance control is implemented to perform the peg-in-hole task with feedback from the above sensors. Contributions are mainly made in the learning-based visual servoing component of the sequence, where a deep neural network is trained with various sets of synthetic data generated using the concept of domain randomization to predict where a hole is. In the experiments and analysis section, the network is analyzed and compared, and a real-world robotic system to insert pegs to holes using the proposed method is implemented. The results show that the implemented peg-in-hole assembly system can perform successful peg-in-hole insertions on surfaces with various colors and textures. It can generally speed up the entire peg-in-hole process, especially when the initial position uncertainty is large.

Modern manufacturing strategy requires the robotic infrastructure to be able to adapt to new products or to accomplish new tasks quickly. In order to respond to this demand, teaching a robot to realize a task by demonstration has regained popularity in recent years, especially for dual-arm or humanoid robots. One of the main issues using this method is to adapt the captured motion from the human demonstration to the robot's specific kinematics and control. In this paper we present a method where the motion and grasping adaptation is tackled during the capture. We demonstrate the validity of this method with an experiment where a humanoid robot realizes an assembly previously demonstrated by a user wearing a Head Mounted Display (HMD) performing an assembly task in a virtual environment.

The goal of this research is developing a system that a rich facial animation can be used in communication is generated from only speech. Generally, a source of the generating facial animation is a camera. Using cameras as an input source, it causes limitations of the angle of view of the camera or problems that cannot be aware of the human face, depending on the orientation of the face. Therefore, it is reasonable for developing a system for generating a facial animation using only voice. In this study, we generate facial expressions from only speech using LSTM-RNN. Comparing 3 patterns of speech analysis data, we showed that the proposed method using A-weighting is effective for facial expression estimation.

This paper develops an adaptive gripper for peg-in-hole tasks. Conventional grippers require complicated compliant mechanisms or complicated control strategy and force sensing to successfully insert pegs into holes. Different from them, this paper proposes a simple gripper with an L-shape finger as a low-cost peg-in-hole solution. The basic idea is to divide a peg-in-hole process into a preparation phase and an execution phase, and eliminate uncertainty step-by-step by pushing using the L-shape finger in the preparation phase. The robustness of the gripper for peg-in-hole tasks is examined by repeated executions for different pegs in the International Robotic Exhibition 2017 (IREX) in Tokyo. The experimental section presents details of the executions, and qualitatively shows the high performance of the proposed gripper.

Typical Head-Mounted Displays (HMDs) that provide a highly immersive Virtual Reality (VR) experience make any interaction between a user and real space difficult by occluding the user's entire field of view. Video see-through type HMDs can solve this problem by superimposing real-space information on the VR environment. The existing method of supporting interactions with the real space is superimposition of boundary lines of the real space on the virtual space in the HMD. However, overlaying the boundary lines on the entire field of view may reduce the user's immersive feeling. In this paper, we propose two methods to support interactions with the real world while playing immersive VR games without reducing the user's immersive feeling as much as possible, even when the user wanders. The first method is to superimpose a 3D point cloud of real space around the user on the virtual space in the HMD. The second method is to deploy familiar objects (e.g., furniture in his/her room) in the virtual space in the HMD. The user traces the familiar objects as subgoals to reach the goal. We implement the two methods and conduct a user study to compare interaction performance. As a result of the user study, we find that the second method provides better spatial information about the real space without reducing the user's immersive feeling, compared to the existing method.

When people anonymously use social network sites (SNSs) like Twitter, they may interact with not only real world friends but also strangers, who are not acquaintances in the real world. Therefore, both of real friendship (RF) and virtual friendship (VF) coexist in these SNSs. In this research, we investigated the differences in similarity of user pairs in Japan by their types of relationship, i.e. RF or VF. The primary results indicated that RF user pairs have more common follow users on SNSs than VF user pairs, and that contents posted by VF user pairs include more similar words than RF user pairs. It is implied that two users with RF have a similar interest in neighborhood users, and two users with VF are interested in similar topics. After that, we built the models to classify user pairs into RF or VF using the similarity measures. These models showed high performance to distinguish between RF and VF (their F-measures are larger than 0.80).

Robotic failure is all too common in unstructured robot tasks. Despite well-designed controllers, robots often fail due to unexpected events. Robots under a sense-plan-act paradigm do not have an additional loop to check their actions. In this work, we present a principled methodology to bootstrap online robot introspection for contact tasks. In effect, we seek to enable the robot to recognize and expect its behavior, else detect anomalies. We postulated that noisy wrench data inherently contains patterns that can be effectively represented by a vocabulary. The vocabulary is obtained by segmenting and encoding data. And when wrench information represents a sequence of sub-tasks, the vocabulary represents a set of words or sentence and provides a unique identifier. The grammar, which can also include unexpected events, was classified both offline and online for simulated and real robot experiments. Multi-class Support Vector Machines (SVMs) were used offline, while online probabilistic SVMs were used to give temporal confidence to the introspection result. Our work's contribution is the presentation of a generalizable online semantic scheme that enables a robot to understand its high-level state whether nominal or anomalous. It is shown to work in offline and online scenarios for a particularly challenging contact task: snap assemblies. We perform the snap assembly in one-arm simulated and real one-arm experiments and a simulated two-arm experiment. The data set itself is also fully available online and provides a valuable resource by itself for this type of contact task. Our verification mechanism can be used by high-level planners or reasoning systems to enable intelligent failure recovery or determine the next most optimal manipulation skill to be used. Supplemental information, code, data, and other supporting documentation can be found at [1].

This paper develops intelligent algorithms for robots to reorient objects. Given the initial and goal poses of an object, the proposed algorithms plan a sequence of robot poses and grasp configurations that reorient the object from its initial pose to the goal. While the topic has been studied extensively in previous work, this paper makes important improvements in grasp planning by using over-segmented meshes, in data storage by using relational database, and in regrasp planning by mixing real-world roadmaps. The improvements enable robots to do robust regrasp planning using 10,000s of grasps and their relationships in interactive time. The proposed algorithms are validated using various objects and robots.

Pick-and-place regrasp is an important manipulation skill for a robot. It helps a robot accomplish tasks that cannot be achieved within a single grasp, due to constraints such as kinematics or collisions between the robot and the environment. Previous work on pick-and-place regrasp only leveraged flat surfaces for intermediate placements, and thus is limited in the capability to reorient an object. In this paper, we extend the reorientation capability of a pick-and-place regrasp by adding a vertical pin on the working surface and using it as the intermediate location for regrasping. In particular, our method automatically computes the stable placements of an object leaning against a vertical pin, finds several force-closure grasps, generates a graph of regrasp actions, and searches for the regrasp sequence. To compare the regrasping performance with and without using pins, we evaluate the success rate and the length of regrasp sequences while performing tasks on various models. Experiments on reorientation and assembly tasks validate the benefit of using support pins for regrasping.

We present a novel method for caging grasps in this paper by stretching ropes on the surface of a 3D object. Both topology and shape of a model to be grasped has been considered in our approach. Our algorithm can guarantee generating local minimal rings on every topological branches of a given model with the help of a Reeb graph. Cages and grasps can then be computed from these rings, and physical experimental tests have been conducted to verify the robustness of our approach.

Pick-and-place regrasp extends the manipulation capability of a robot by using a sequence of regrasps to accomplish tasks that are not possible using a single grasp due to constraints such as kinematics or collisions between the robot and the environment. Previous work on pick-and-place only leveraged static passive devices for intermediate placements, and thus is limited in the flexibility and robustness to reorient an object. In this paper, we extend the reorientation capability of a pick-and-place regrasp by adding an actively actuated gripper fixed in the working cell, and using it as the intermediate location for regrasping. In particular, our method automatically computes the stable placements of an object being hold in the gripper support, finds a rich set of force-closure grasps, performs k-means based grasp clustering, generates a graph of regrasp actions, and searches for the optimal regrasp sequence. To compare the regrasping performance with typical passive supports, we evaluate the success rate while performing tasks on various models. Experiments on reorientation tasks validate the benefit of using an actively actuated gripper for regrasp placement.

In this study, we experimentally investigated the effect of robot fingertip stiffness on the maximum resistible force. The maximum resistible force is defined as the maximum tangential force at which the fingertip can maintain contact when applying and increasing tangential/shearing force. We include in the definition of this term the effect of fingertip deformation. In contrast to our previous study [11], cylindrical fingertips with flat surfaces were used in this study so that the contact area would remain the same when there was no tangential/shearing force. This made it possible to see the effect of fingertip stiffness more clearly. We also investigated the effect of curvature of the contact surface, which was not investigated in depth in [11]. The main findings are as follows. 1) Harder fingertips produce larger resistible forces, irrespective of the shape of the contact surface (flat or curved). 2) For harder fingertips, the maximum resistible force depends largely on the shape of the contact surface, while for softer fingertips, the shape has little effect. 3) For softer fingertips, the magnitude of the resistible force changes little even when the normal force increases.

We propose an evaluation method of grasp stability which takes into account the elastic deformation of fingertips from the viewpoint of energy. An evaluation value of grasp stability is derived as the minimum energy which causes slippage of a fingertip on its contact surface. To formulate the evaluation value, the elastic potential energy of fingertips and the gravitational potential energy of a grasped object are considered. It is ensured that fingertips do not slip on grasped object surfaces if the external energy applied to the object is less than the evaluation value. Since our evaluation value explicitly considers the deformation values of fingertips, grasp stability is evaluated by taking into consideration the contact forces generated by the deformation. The effectiveness of our method is verified through numerical examples.

In this paper, we propose a general manipulation planner for dual-arm industrial manipulators. According to the context, the planner automatically determines whether both arms have to be used simultaneously or not. The approach is based on (i) the extension of an object placement algorithm previously developed in [23], and (ii) the introduction of several types of re-grasping motions dedicated to dual-arm manipulators. Such motions induce a special topological structure in the manipulation space that can be captured into a manipulation graph. The graph is then used to solve the manipulation problem by a simple graph search algorithm. After searching for a solution path, we further consider optimizing the path by minimizing the number of re-grasps. The effectiveness of the approach is demonstrated on the dual-arm manipulator HiroNX working in a realistic factory environment.

We discuss grasp stability under gravity where each finger makes soft-finger contact with an object. By clustering polygon models of a finger and an object, the contact area between a finger and an object is obtained as the common area between an object cluster and a finger cluster. Then, by assuming the Winkler elastic foundation, the pressure distribution within the contact area is obtained. By using this pressure distribution, we show that we can judge grasp stability under soft-finger contact. We further consider defining a quality measure of a soft-finger grasp by assuming that although the gravitational force is applied to an object, the direction of gravity is unknown. To demonstrate the effectiveness of the proposed approach, we show several numerical examples.

Failure detection and correction is essential in robust systems. In robotics, failure detection has focused on traditional parts assembly, tool breakage, and threaded fastener assembly. However, not much work has focused on sub-mode failure classification. This is an important step in order to provide accurate failure recovery. Our work implemented a novel failure characterization scheme for cantilever snap assemblies. The approach identified exemplars that characterized salient features for specific deviations from a nominal trajectory. Then, a rule based approach with statistical measures was used to identify failure and classify failure sub-modes. Failure sub-mode classification was evaluated by using a reliability measure. Our work classified failure deviations with 88% accuracy. Varying success was experienced in correlating failure deviation modes. Cases with only 1-deviation had 86% accuracy, cases with 2-deviations had 67% accuracy, and cases with 3 deviations had 55% accuracy. Our work is an important step in failure characterization of complex geometrical parts and serves as a stepping stone to enact failure recovery.

This paper presents a grasp planner which allows a robot to grasp the constricted parts of objects in our daily life. Even though constricted parts can be grasped more firmly than convex parts, previous planners have not sufficiently focused on grasping this part. We develop techniques for quadric surface approximation, grasp posture generation, and stability evaluation for grasping constricted parts. By modeling an object into multiple quadric surfaces, the planner generates a grasping posture by selecting one-sheet hyperbolic surfaces or two adjacent ellipsoids as constricted parts. When a grasping posture being generated, the grasp stability is evaluated based on the distribution of the stress applied to an object by the fingers. We perform several simulations and experiments to verify the effectiveness of our proposed method.

This paper presents a knowledge model of everyday objects for semantic grasp. This model is intended for extracting the grasp areas of everyday objects and approach directions for grasping when the 3D point cloud data and the intended purpose are given. Parts that make up everyday objects have functions related to their manipulation. We therefore represent everyday objects in terms of connected parts of functional units. This knowledge model describes the structure of everyday objects and information on their manipulation. The structure of an everyday object describes component parts of the object in terms of simple shape primitives to provide geometrical information and describes connections between parts with kinematic attributes. The information on the structure is used to map the manipulation knowledge onto the 3D point cloud data. The manipulation knowledge of the object includes the grasp areas and approach directions for the intended purpose. Fine grasps suitable for the intended task can be generated by performing a grasp planning with consideration for stable grasp and the kinematics of the robot in the grasp areas and approach directions.

This paper proposes a method for bin-picking for objects without assuming the precise geometrical model of objects. We consider the case where the shape of objects are not uniform but are similarly approximated by cylinders. By using the point cloud of a single object, we extract the probabilistic properties with respect to the difference between an object and a cylinder and consider applying the probabilistic properties to the pick-and-place motion planner of an object stacked on a table. By using the probabilistic properties, we can also realize the contact state where a finger maintain contact with the target object while avoiding contact with other objects. We further consider approximating the region occupied by fingers by a rectangular parallelepiped. The pick-and-place motion is planned by using a set of regions in combination with the probabilistic properties. Finally, the effectiveness of the proposed method is confirmed by some numerical examples and experimental result.

Dexterous manipulation is an important function for working robots. Manipulator tasks such as grasping, assembly and disassembly can generally be divided into several motion primitives. We call such motion primitives “skills” and explain how most manipulator tasks can be composed of sequences of these skills. We will address the issues involved with various types of robots such as maintenance robots and service robots. We have considered hierarchizing the manipulation tasks of these robots since their tasks have become more complex than ever before. Additionally, as errors are seen likely to increase in complex tasks, it is important to implement effective error recovery technology. This paper presents our proposal for a new type of error recovery that uses the concepts of task stratification and error classification which can be expressed specifically using flow charts.

This paper proposes a method for planning the pick-and-place motion of an object by dual-arm manipulators. Our planner is composed of the offline and the online phases. The offline phase generates a set of regions on the object and the environment surfaces and calculates several parameters needed in the online phase. In the online phase, the planner selects a grasping pose of the robot and a putting posture of the object by searching for the regions calculated in the offline phase. By using the proposed method, we can also plan the trajectory of the robot, and the regrasping strategy of the dual-arm. Here, the putting posture of the object can be planned by considering stability of the object placed on the environment. The effectiveness of the proposed method is confirmed by simulation and experimental results by using the dual-arm robot NX-HIRO.

This paper proposes an object placement planner for a grasped object during pick-and-place tasks. The proposed planner automatically determines the pose of an object stably placed near a user assigned point on an environment surface. The proposed method first constructs a polygon model of the surrounding environment, and then clusters the polygon model of both the environment and the object where each cluster is approximated by a planar region. The placement of the object can be determined by selecting a pair of clusters between the object and the environment. We further impose several conditions to determine the pose of the object placed on the environment. We show that we can determine the position/orientation of the object placed on the environment for several cases such as hanging a mug cup on a bar. The effectiveness of the proposed research is confirmed through several numerical examples.

Snap assembly automation remains a challenging task. While progress is being made in localization of parts, force controllers, and control strategies, little work has been done to help the robot reason about its current state, such that if necessary, the robot can assume corrective actions to accomplish the task. Error prone situations caused by the unexpected motion of parts, localization errors, jamming or wedging, cannot be solved by force controllers alone. For this reason we propose a snap assemblies verification system for cantilever-snap fasteners. The verification works in concert with a control strategy that makes use of constraint designs embedded in the snap parts' physical design. The constrained assembly motion generates similar sensory-signal patterns across trials that facilitates force signal discrimination into higher level abstractions of intuitive behavior. This work's contribution is the design of a hierarchical taxonomy for cantilever-snap verification based on increasingly abstract layers that encode relative-change in the task's force signatures. A five-layered taxonomy is built on the concept that relative-change patterns can be classified through a small category set and aided by contextual information. The verification system yielded human apropos intuitive categorizations of task behavior for every state and effectively determined the assembly result. This simple yet effective approach will be expanded to perform probabilistic online system verification to aid in fault tolerance and the automation of cantilever-based snap assemblies.

This paper presents preliminary results and analysis on generating turning motion of a humanoid robot by slipping the feet on the ground. Humans unconsciously exploit the fact that our feet slip on the ground; such slip motion is necessary for humanoids so as to realize sophisticated human-like motions. In order to generate the slip motion, we need to predict the amount of slip. We propose the hypothesis that the turning motion is caused by the effect of minimizing the power generated by floor friction. A model of rotation by friction force is described on the basis of our hypothesis. The case that a robot applies the same force on both feet is discussed; then, we extend the discussion to the case of different force distribution. The hypothesis is verified through experiments with a humanoid robot HRP-2.

A novel framework of biped walking stabilization control is introduced. The target robot is a 42 DOF humanoid robot HRP-4C which has a body dimensions close to the average Japanese female. We develop a body posture controller and foot force controllers on the joint position servo of the robot. By applying this posture/force control, we can regard the robot system as a simple linear inverted pendulum with ZMP delay. After a preliminary experiment to confirm the linear dynamics, we design a tracking controller for walking stabilization. It is evaluated in the experiments of HRP-4C walking and turning on a lab floor. The robot can also perform an outdoor walk on an uneven pavement.

In this paper, we generate the biped gait of a humanoid robot that looks like a human's one. To generate the human like motion, we first capture walking motion of a human. Then, we analyze the captured data and obtain several information such as the relationship between the step length and waist height etc. We consider applying these informations to the real humanoid robot. Also, when the human walks, the sway of the waist is smaller than most of the humanoid robot's one. By compensating the angular momentum of the robot and by modifying the ZMP trajectory, we show that sway of the robot's waist can be smaller. We show the effectiveness of the proposed method through simulation and experimental results.

This paper presents an easy and fast method of testing force closure when a multi-fingered hand grasps an object. Different from previous methods, we consider approximating the friction cone by using a few ellipsoids. By using this method, the total force/moment set applied to the grasped object can be obtained by a common set of multiple ellipsoids. This method is effective since we can test the force closure by simply calculating the inequalities of quadratic form. Moreover, by using the ellipsoidal approximation, we propose an easy method of evaluating the grasp stability. We show that the grasp stability can be calculated by using simple equations. The effectiveness of the proposed method is verified by several numerical examples where we show that the proposed method is fairly accurate and can evaluate the grasp stability faster than conventional methods.

This paper discusses the grasp planning of a multifingered hand attached at the tip of a robotic arm. By using the convex models and the new approximation method of the friction cone, our proposed algorithm can calculate the grasping motion within the reasonable time. For each grasping style used in this research, we define the grasping rectangular convex (GRC). We also define the object convex polygon (OCP) for the grasped object. By considering the geometrical relashionship among these convex models, we determine several parameters needed to define the final grasping configuration. To determine the contact point position satisfying the force closure, we use two approximation models of the friction cone. To save the calculation time, the rough approximation by using the ellipsoid is mainly used to check the force closure. Additionally, approximation by using the convex polyhedral cone is used at the final stage of the planning. The effectiveness of the proposed method is confirmed by some numerical examples.

This paper proposes the kinodynamic gait planning for humanoid robots where both kinematics and dynamics of the system are considered. We can simultaneously plan both the foot-place and the whole-body motion taking the dynamical balance of the robot into consideration. As a dynamic constraint, we consider the differential equation of the robotpsilas CoG. To solve this constraint, we assume two walking pattern generators; the offline and the online ones. We randomly sample the configuration space to search for the path connecting the start and the goal configurations. When sampling the configuration space, three milestones are sequentially connected to the parent milestone. To show the effectiveness of the proposed methods, we show simulation and experimental results where the humanoid robot HRP-2 walks on several environments.

In this paper, the development of humanoid robot HRP-3 is presented. HRP-3, which stands for Humanoid Robotics Platform-3, is a human-size humanoid robot developed as the succeeding model of HRP-2. One of features of HRP-3 is that its main mechanical and structural components are designed to prevent the penetration of dust or spray. Another is that its wrist and hand are newly designed to improve manipulation. Software for a humanoid robot in a real environment is also improved. We also include information on mechanical features of HRP-3 and together with the newly developed hand. Also included are the technologies implemented in HRP-3 prototype. Electrical features and some experimental results using HRP-3 are also presented.

This paper presents a biped humanoid robot that is able to walk on a rough terrain while touching a handrail. The contact wrench sum (CWS for short) is used as the criterion to judge if the contact between the robot and the environment is strongly stable under the sufficient friction assumption, where the contact points are not coplanar and the normal vectors at the points are not identical. It is confirmed that the proposed pattern generator can make the robot walk as desired in dynamics simulations and experiments, and the motions can be improved by a hand position control and using waist joints.

This paper presents a motion pattern generator of humanoid robots that walks on a flat plane, steps and a rough terrain. It is guaranteed rigorously that the desired contact between a humanoid robot and terrain should be maintained by keeping the contact wrench sum between them inside the contact wrench cone under the sufficient friction assumption. A walking pattern is generated by solving the contact wrench equations and by applying the resolved momentum control.

This paper presents a development of multi-fingered hand, which is modularized and can be attached to life-size humanoid robots. The developed hand has four fingers with 17 joints, which consist of 13 active joints and 4 linked joints. A miniaturized 6-axes force sensor is newly developed and is mounted on each fingertip for improving the manipulability. A main node controller with I/O, motor drivers, and amplifiers for 6-axes force sensors are also newly developed. These components are equipped in the hand for modularization. The developed hand is designed so as to realize about 8 [N] forces on the pad point of stretched finger, supposing transmission efficiency of drive system is 55 [%]. In this paper, the mechanisms of hand module, its specifications, and electrical system are also introduced.

This paper proposes a method of a real-time gait planning for humanoid robots which can change stride immediately at every step. Based on an analytical solution of an inverted pendulum model, the trajectories of the COG (center of gravity) and the ZMP (zero-moment point) are parameterized by polynomials. Since their coefficients can be efficiently computed with given boundary conditions, this framework can provide a real-time walking pattern generator for humanoid robots. To handle the unexpected result caused by immediate changes of foot placement, we made single support periods as an additional trajectory parameter and the ZMP fluctuation was suppressed by mixing the opposite phase of the ZMP error. The effectiveness of our method is shown by experiments of the humanoid robot HRP-2.

This paper studies an optimal planning of falling motions of a human-sized humanoid robot to reduce the damage of the robot. We developed a human-sized robot HRP-2FX which has a simplified humanoid robot shape with seven d.o.f. and can emulate motions in the sagittal plane of a humanoid robot. An optimal control is applied to generate the falling motion of HRP-2FX to minimize a performance index, and the optimality has been verified by the experiments on HRP-2FX.

In this paper, we plan the collision free motion for walking pattern generation of a humanoid robot. Our motion planner can take into account several features of the walking pattern generator. We first run the walking pattern generator by considering the contact wrench applied to the robot and monitor the collision among the links and the environments. Then, we plan the collision free motion for the period of time causing the collision. In our motion planner, we can consider the constraint condition which are the functions of time. Also, for keeping balance of the robot, we plan the motion with keeping the horizontal position of the COG as well as the position/orientation of the feet/hand. The effectiveness of the proposed method is confirmed by simulation and experiment.

This paper proposes a universal stability criterion of the foot contact of legged robots. The proposed method checks if the sum of the gravity and the inertia wrench applied to the COG of the robot, which is proposed to be the stability criterion, is inside the polyhedral convex cone of the contact wrench between the feet of a robot and its environment. The criterion can be used to determine the strong stability of the foot contact when a robot walks on an arbitrary terrain and/or when the hands of the robot are in contact with it under the sufficient friction assumption. The determination is equivalent to check if the ZMP is inside the support polygon of the feet when the robot walks on a horizontal plane with sufficient friction. The criterion can also be used to determine if the foot contact is sufficiently weakly stable when the friction follows a physical law. Therefore, the proposed criterion can be used to judge what the ZMP can, and it can be used in more universal cases

This paper presents a method of generating natural-looking motion primitives for humanoid robots. An optimization-based approach is used to generate these primitives, but the objective function is tailored to each one and complexity is reduced by identifying relevant degrees of freedom. Several examples are shown in simulation: for an arm movement to reach an object, it is better to minimize the acceleration of key parts of the robot over its entire trajectory; for a single step on flat ground, it is better to minimize the torque and instantaneous angular momentum at every posture. The primitives are precomputed off-line, but might be used by on-line planner either to provide a fixed set of maneuvers or to bias a probabilistic, sample-based search for motions

A biped walking pattern generator which allows an additional ZMP control (auxiliary ZMP) is presented. An auxiliary ZMP is realized by an inverse system added to a pattern generator based on the ZMP preview control. To compensate the effect of the auxiliary ZMP, we apply virtual time shifting of the reference ZMP. As an application of the proposed method, a walking control on uneven terrain is simulated. The simulated robot can walk successfully by changing its walking speed as the side effect of the auxiliary ZMP control

This paper presents a motion suspension system to suspend humanoid motion in case of emergency. Once humanoids start their motions in human daily environments, there is a possibility that humanoids will meet with several emergencies such as hurting humans and injuring themselves. Even so, humanoids should be controlled so that they avert such emergencies in real-time. To realize this demand, we propose a method of real-time judgment of emergency prediction by humanoids. We also propose a simple and effective method of real-time pattern generation to force humanoids to stop immediately by one step without falling. To verify the validity of the proposed method, we finally present experimental results using a humanoid robot HRP-2, which include experiments at 2.8 [km/h] walks

A motion planner of emergency stop must make an operating humanoid robot to a stationary state under the emergency signal. It plays an important role in prevention of falling over because humanoid robots fall over easily. Immediately after the emergency signal, it must generate the emergency stop motion in real-time. We modeled a humanoid robot as a simple dynamic system consists of ZMP (zero-moment point) and COG (center of gravity) as its states. The emergency stop motion is generated by a state feedback. We determined optimal feedback gains in terms of the initial conditions and the pole assignment. The proposed method realized a reliable emergency stop with low computational cost. Furthermore, it can easily predict the possibility of the successful emergency stop at any time. The validity of the proposed method is confirmed by an experiment using humanoid robot HRP-2

This paper discusses the whole arm manipulation allowing the contact state transition. For manipulation of an object under fully constrained, the contact state transition becomes necessary. In order to realize the object manipulation, we first derive the feasible direction of the object manipulation by analyzing the active/passive closure properties for every combination of contact states. Second, we derive the set of joint torque to move the object in the feasible direction. These analyses also provide the joint torque to realize the manipulation at the planned contact states. Effectiveness of the proposed method is confirmed by some simulation results

In this paper, we analyze the balance of a humanoid robot during manipulation tasks. By defining the generalized zero-moment point (GZMP), we obtain the region of it for keeping the balance of the robot during manipulation. During manipulation, the convex hull of the supporting points forms the 3-D convex polyhedron. The region of the GZMP is obtained by considering the infinitesimal displacement and the moment about the edges of the convex hull. We show that we can determine whether or not the robot may keep balance for several styles of manipulation tasks, such as pushing and pulling an object. The effectiveness of our proposed method is demonstrated by simulation.

This paper studies the balance of a humanoid robot carrying a heavy object. Without knowing the mass and the position of the center of gravity of the object, the humanoid robot carries a heavy object stably by using the force sensor information attached at the wrists and the ankles. We first show how to generate the motion of a humamoid robot by taking the force sensor information into consideration. We also show the method for generating the gait pattern by taking the dynamics of the carried object. The effectiveness of the proposed method is shown by experiments.

This paper proposes a method for a humanoid robot to generate 3D model of the environment using a stereo vision, find a movable space using it and plan feasible locomotion online. The model is generated by an accumulation of 3D grid maps which are made from the range data of the field of view obtained by a correlation based stereo vision. The locomotion is planned by an online whole body pattern generator which can modify robot’s waist height, an upper body posture and so on according to the size of the movable space.

This paper describes a generation method for spatially natural biped walking. By limiting the COG (Center of Gravity) motion space to a sculptured surface, the degree of freedom of the COG matches to the number of the ZMP (Zero Moment Point) equations. The COG motion can be uniquely generated along a specified surface satisfying the ZMP constraint with low calculation cost. Spatial and time parts are separable by representing motion surface of the COG as parametric variables. The motion surface defines the relative height of the COG from the landing foot position. Thus, the proposed method reflects geometric information directly to the motion planning without considering walk stability. In this paper, we show two actual examples, walking pattern including mostly stretched knee and going up stairs. The validity of the proposed method is confirmed by simulation. Walking with mostly stretched knee is also shown in experiment using HRP-2.

In this paper, we discuss the manipulation of an object under hybrid active/passive closure. We show the orthogonality between the directions of active and passive force closures for general grasping systems. Based on the orthogonality, we decompose the dynamics of grasping system into the” active part” and the” passive part”. By using the decomposition, we show that the grasped object can be manipulated only by considering the dynamics of the active part. We also consider how to determine the desired internal forces in order to satisfy frictional constraints during the manipulation. In order to verify the validity of our approach, some simulation results are shown.

This paper presents a slip observer towards stabilizing biped walks on a low friction floor. Although biped humanoid robots are expected to easily adapt to environments designed for human, in fact they tend to tip over easily on real environments. For a practical use, it is one of important issues to stabilize a biped walking on an unexpected slippery floor with a low friction. In this paper, we propose the slip observer detecting skids that would occur at walking on unexpected slippery floor. We also propose a basic study of slip stabilizer towards reducing posture rolling caused by skids. Finally, we present experimental results using a humanoid robot HRP-2 to verify the validity of the proposed control scheme.

This paper presents an emergency stop algorithm of a walking humanoid robot. There are many cases which force a walking robot to stop quickly without falling. Since an emergency occurs at unpredictable timing and at any state of robot, the stopping motion must be generated in real-time. To overcome these problems, our emergency stop motion is divided into four phases according to the role of the zero-moment point (ZMP). In each phase, approximate analytical solutions of the center of gravity (COG) dynamics is used to generate the motion. During the single support phase, a landing time and position are determined by evaluating the average velocity of the swing leg and the horizontal position of the COG. During the double support phase, the travel distance of the COG and the ZMP are evaluated. The validity of the proposed method is confirmed by simulation and experiment using a humanoid robot HRP-2.

We previously reported the first human-sized humanoid robot that can fall down safely and stand up again. This paper examines the falling motion control by supplementary simulations and presents an improvement of the control by optimizing the control parameters. The UKEMI falling over control of a human-sized humanoid robot is simulated, the simulation results are compared with the experimental results of real humanoid robot.

This work proposes a new style of manipulation by a humanoid robot. Focusing on the task of pushing an object, the foot placement of it is planed in real-time according to the result of manipulation of an object. By controlling the arms using the impedance control, a humanoid robot can push an object stably regardless of the mass of an object. If an object is heavy, a humanoid robot pushes an object with walking slowly, and vice versa. Also, for planning the gait in real-time, we newly propose an analytical method where the newly calculated trajectory of the robot motion is smoothly connected to the current one. The effectiveness of the proposed method is confirmed by simulation and experiment.

This work studies locomotion planning of humanoid robots to pass through narrow spaces. Humanoid robots can alter the style of the locomotion while wheeled robots can not. The proposed method generates a 3D local map from visual information and plans the appropriate locomotion based on the map from biped walking with the variable height and the width and from crawling.

This paper shows some preliminary results on the dynamical balance of a humanoid robot grasping an environment. By grasping an environment, it becomes easier for the robot to keep balance. By using the linear programming, a necessary condition for keeping balance of the robot is formulated taking the grasping force into consideration. We show that the occasion exists where the stronger the hand of a humanoid robot grasps the handrail, the larger the region of ZMP for keeping balance becomes. We further show an experimental result of a humanoid robot climbing up a big gap increasing the stability by grasping a handrail.

Biped walking on a low friction floor is analyzed in this paper. For a given walking pattern, we can calculate a necessary friction coefficient which allows the robot to perform the expected motion. To reduce the maximum necessary friction coefficient, a pattern generation based on preview control theory is explained. We also describe a calculation of slip concerned ZMP, which provides a good prediction of falling caused by slips. Finally, we test a walk of 135 [km/h] on low friction environment using a humanoid robot HRP-2. The robot could successfully walk over the slippery area whose friction coefficient is 0.14.

This paper discusses the pushing manipulation of an object by a humanoid robot. For such a pushing task, we show that there are two kinds of ZMPs, i.e., the conventional "Zero Moment Point (ZMP)" considering all sources of the force/moment acting in the foot supporting area, and the "Generalized Zero Moment Point (GZMP)" which is an generalization of ZMP for a humanoid robot whose hands do not contact with an object. We first obtain the stable region of the GZMP on the floor. Moreover, since the difference between these two ZMPs corresponds to the magnitude of contact force applied by the hands, we propose the pushing manipulation by a humanoid robot by modifying the desired ZMP trajectory for a humanoid. The effectiveness of the proposed method is confirmed by simulation results.

We have developed a software platform, called OpenHRP, for humanoid robotics which consists of a dynamic simulator and motion control library for humanoid robots. This paper attempts to answer a frequently asked question "do the dynamic simulations and the experiments of biped walking of humanoid robots correspond?". Using OpenHRP and humanoid robots HRP-1S and HRP-2P, the comparisons between the simulations and experiments are shown at various aspects.

We introduce a new method of a biped walking pattern generation by using a preview control of the zero-moment point (ZMP). First, the dynamics of a biped robot is modeled as a running cart on a table which gives a convenient representation to treat ZMP. After reviewing conventional methods of ZMP based pattern generation, we formalize the problem as the design of a ZMP tracking servo controller. It is shown that we can realize such controller by adopting the preview control theory that uses the future reference. It is also shown that a preview controller can be used to compensate the ZMP error caused by the difference between a simple model and the precise multibody model. The effectiveness of the proposed method is demonstrated by a simulation of walking on spiral stairs.

This paper presents a humanoid robot that has the same size as a human and that can lie down to the floor and get up from the floor with the robot face upward and downward. We believe that the robot is the first life-size humanoid robot with the capability. The motions are realized by the combination of novel hardware and software. The features of the hardware are a human-like proportion and joints with wide movable ranges including two waist joints. The software segments the motion into the sequence of the contact states between the robot and the floor and assigns an appropriate controller to each transition between the consecutive states. The experimental results are presented.

This paper investigates a method through which human-size humanoid robot can fall over backwards safely. Squatting-extending motion of legs reduce impact of falling and shock-absorbing parts of the robot keep the force at a permissible range. The robot could stand up itself again after falling.

This paper newly considers the ZMP (zero moment point) of a humanoid robot under arm/leg coordination. By considering the infinitesimal displacement and the moment acting on the convex hull of the supporting points, we show that our method for determining the region of ZMP can be applicable to several cases of the arm/leg coordination tasks. We first express two kinds of ZMPs for such coordination tasks, i.e., the conventional ZMP, and the "generalized zero moment point (GZMP)" which is a generalization of the ZMP to the arm/leg coordination tasks. By projecting the edges of the convex hull of the supporting points onto the floor, we show that the position and the region of the GZMP for keeping the dynamical balance can be uniquely obtained. The effectiveness of the proposed method is shown by simulation results.

We introduce a method to generate whole body motion of a humanoid robot such that the resulted total linear/angular momenta become specified values. First, we derive a linear equation, which gives to total momentum of a robot from its physical parameters, the base link speed and the joint speeds. Constraints between the legs and the environment are also considered. The whole body motion is calculated from a given momentum reference by using a pseudo-inverse of the inertia matrix. As examples, we generated the kicking and walking motions and tested on the actual humanoid robot HRP-2. This method, the resolved momentum control, gives us a unified framework to generate various maneuvers of humanoid robots.

Discusses the manipulation of multiple objects by pushing. When manipulating ultiple objects, relative motion between two objects such as sliding and rotation may occur. We propose velocity based pushing where the region of the pusher's velocity causing the desired relative motion at each contact point is obtained as a common space of regions causing the relative motion at one of the contact points. When obtaining the velocity region, we use the equivalent center of friction (ECOF) which is an extension of the conventional COF and which is same as COF when the object translate. Experimental results are also included to show the effectiveness of our idea.

This paper newly proposes dynamic friction closure, under which a moving object is completely stopped within a predetermined length through the work due to Coulomb friction between an object and a gripper. There are two conditions for achieving it: The geometrical condition, ensuring the robot motion to meet with an object at an appropriate catching point, and frictional stopping capability, ensuring the moving distance of object is less than the predetermined distance under a limited grasping force. Through the simple analysis on catching a 2D circular object by a parallel jaw gripper, we newly found that even under the same total energy of object, the distance changes depending upon the ratio between initial rotational and translational velocity. For a 3D sphere object, the maximum moving distance is provided with the initial kinetic energy.

This paper discusses the manipulation of 3D enveloped object. By assigning all fingers as either the position controlled finger (P-finger) or the torque controlled finger (T-finger), we propose a method for letting the object to move in the desired direction along the surface of the P-fingers. For this purpose, we obtain the joint torque command for the T-fingers. The formulation of the total force/moment set can be applied to general 3D enveloping grasp. Also, we newly provide a sufficient condition for the total force/moment set of the object moving toward the desired direction, where it can be applied to general 3D contact configurations between the object and the P-fingers.

This paper discusses the determination of torque commands for always guaranteeing the body propelling motion in Hugging Walk where the environment is enveloped by legs and the body. There are two key issues, one is to obtain the sub-total force set with the function of the joint torque in each leg, and the other is to obtain the constraint space where the body propelling motion is guaranteed without the body rotating motion when the sub-total force set is put in it. The basic idea is demonstrated by simulation and experiment.

This paper discusses a sufficient condition for the manipulation of the Envelope Family, where multiple contacts are allowed between object and chains (or the environment). All chains are assigned to either the position controlled chain (P-chain) or the torque controlled chain (T-chain). Although the object motion under multiple contacts can not be uniquely specified only by the T-chains, we are able to provide a sufficient condition ensuring that a given set of torque commands for the T-chains always moves the object within the designated direction along the surface of a P-chain (or a fixed environment). Experiments as well as simulations are shown to verify this basic idea.

This paper discusses the manipulation of an object under neighborhood equilibrium (NE), where an object with rolling contact can automatically find another equilibrium state close to the original one even when the current one is broken due the change of input. For a guaranteeing NE, we newly define the contact stable region (CSR) on the object surface. By utilizing the CSR, we propose a control scheme for manipulating an object from one spot to the desired position on a plate whose posture is controllable. Numerical example is also shown to explain our idea.

Through group experiments by human, we found an interesting grasping pattern, where the human can easily captures a small object placed on a table by changing the finger posture from upright to curved ones after each finger making contact with the object. A series of this motion is called as detaching assist motion (DAM). This paper discusses a condition for achieving DAM by taking the idea of self-posture changing motion. A sufficient condition for lifting up an object from the table is discussed. Some experiments are also shown to verify the condition.

The enveloping grasp feasibility inequality is formulated in this article using a coordinate transformation based on the concept of frictional degrees-of-freedom. Various analysis and synthesis problems pertinent to enveloping grasp of whole-limb systems can be handled within the same mathematical framework, even under the situation where statically indeterminate contact forces exists. Simulations are also shown to explain the basic idea.

In this paper, a new training method for robotic rehabilitation aids is proposed, in which only one neural network (NN) is simultaneously used in order to identify the dynamic properties of a human-robot system and give an assist to the trainee. The model used for identification of the dynamics of the human-robot system consists of the NN and a reference model which represents the main characteristic of a skilful operator. This paper explains the working principle of the training method and shows the validity of the proposed method through experiments by non-skilled operators.

The problem due to the statically indeterminate contact forces arising in indeterminate frictional enveloping grasps is addressed. First, we show that the statical model for the contact forces is incomplete in the sense that the mathematical frictional forces may have an infeasible component. To resolve this infeasibility, we directly derive the enveloping grasp infeasibility condition on the frictional forces based on the coordinate transformation developed for frictional enveloping grasps. Then the indeterminate grasp can be analyzed in cooperation with the enveloping grasp feasibility inequality, as shown in a numerical example.

Discusses the neighborhood equilibrium grasp for multiple objects. Assuming rolling contact at each contact point, we first define rolling based redundancy which means that the grasped objects have degrees of freedom through the rolling motion even if the finger posture is fixed. For the grasp having rolling based redundancy, we say that there exists the neighborhood equilibrium if the system can be shifted to another equilibrium state close to the original one even when the current equilibrium is broken. We evaluate the robustness of the equilibrium state by utilizing the rotating angle, where the system loses the neighborhood equilibrium. We show several numerical examples to verify our idea.

Discusses the manipulation of multiple objects under rolling contacts. For manipulating multiple objects, there are two key issues which do not arise in the manipulation, of a single object, (1) each object's motion is restricted by the other objects, and (2) the contact force among objects is not controlled directly. As for (1), we first formulate the motion constraint for the whole grasp system, and then provide a necessary condition for manipulating multiple objects uniquely. As for (2), we provide a condition for determining the contact forces among objects uniquely. We further show a sufficient condition for manipulating multiple objects within the object motion constraint. Under this sufficient condition, we propose a control scheme for object motion by taking the motion constraint into account. An experimental result is provided to confirm our idea.

This paper discusses a sufficient condition for manipulation of envelope family, where multiple contacts are allowed between object and chains (or environment). All chains are assigned by either position controlled chain (P-chain) or torque controlled chain (T-chain). While the object motion under multiple contacts can not be uniquely specified by T-chains only, we show a sufficient condition ensuring that a given set of torque commands for T-chains always move the object toward the goal along the surface of P-chain (or a fixed environment). Experiments as well as simulations are also shown to verify the basic idea.

This paper discusses the manipulation of multiple objects under rolling contacts. For manipulating multiple objects, the following two key issues do not arise in the manipulation of a single object: 1) each object's motion is restricted by the other objects; and 2) the contact force among objects is not controlled directly. As for (1), we first formulate the motion constraint for the whole grasp system, and then provide a necessary condition for manipulating multiple objects uniquely. As for (2), we provide a condition for determining the contact forces among objects uniquely. We further show a sufficient condition for manipulating multiple objects within the object motion constraint. Under this sufficient condition, we propose a control scheme for object motion by taking the motion constraint into account. Simulation and experimental results are provided to confirm our idea.

This paper discusses the enveloping grasp of multiple objects under rolling contacts. We first provide a general mathematical formulation on the kinematic relationship for multiple objects enveloped by a multifingered robot hand, and then derive a condition for judging whether the rolling condition can be satisfied at each contact point. We also show a sufficient condition for rolling up any two objects grasped by a multifingered robot hand in contact with them. Finally, an experimental result is shown to confirm how easily two cylindrical objects can be enveloped by a simple grasping motion.

Discusses the analysis of kinematics and internal force under the gravitational field in grasping multiple objects. We first provide a general, mathematical formulation on the kinematic relationships for multiple objects grasped by a multifingered hand. We show a sufficient condition for lifting up two objects toward the palm. An experimental result verifies that the sufficient condition can be easily satisfied. We also analyze the internal forces for grasped objects, and introduce an index for evaluating the degrees of freedom of internal forces. Finally, we show a couple of examples to explain the physical interpretation of internal forces in connection with the index.

This paper discusses how many fingers are necessary and sufficient for capturing a pyramidal-like object placed on a table under the gravitational field. Allowing that the contact friction is small enough to ensure that any direct grasp may fail in achieving an equilibrium grasp, we prove that a planar two-fingered hand are necessary and sufficient for achieving the task for an idealized 2D triangle object placed vertically on a table. We also consider 3D pyramidal-like objects, and prove that a spatial two-fingered hand are necessary and sufficient for achieving the task.

We deal with the problem of internal stabilization in dynamic trajectory control of flexible manipulators. We compare two configurations of flexible manipulators. One is the conventional flexible manipulator, and the other is the macro-micro system that has a small rigid manipulator at the tip of the flexible manipulator. In order to preserve the stability, the precise end point tracking is impossible for the conventional flexible manipulator. On the other hand, utilizing the redundancy, the precise end point tracking with internal stabilization is available by using the macro-micro system. First, we construct a dynamic trajectory controller for the flexible manipulator considering the internal stabilization, and analyze the problems in controllers of conventional flexible manipulators. Second, we propose a controller for the macro-micro system, and show that the problems can be solved by using the macro-micro system. Lastly, simulation and experimental results are shown.

In this paper, hybrid position/force control algorithms of combined flexible-macro/rigid-micro manipulator systems are proposed, In the proposed system, the micro manipulator is attached at the tip of the flexible macro manipulator. The macro manipulator can move widely, but cannot realize fast and precise motion because of its flexibility. On the contrary, the micro manipulator cannot move widely, but can move fast and precisely. By taking advantage of the macro/micro system, both the end point position and the force exerted by its end effector can be easily controlled in spite of the flexibility in the macro part. This paper first discusses trajectory planning for the macro/micro system. Second, a quasi-static hybrid control algorithm and a dynamic hybrid control algorithm are developed, In our control algorithms, the macro part is controlled roughly to realize the desired trajectory, and suppress vibration. The micro part is controlled to compensate for the position and force errors due to the elasticity in the macro part. Finally, to verify the effectiveness of the proposed control algorithms, experimental results are shown.

In this paper, to realize the hybrid position/force control of a flexible manipulator, the macro-micro manipulator system is used. The macro manipulator can move widely, but can not realize fast and precise motion because of its flexibility. To compensate the position and force errors, a micro manipulator is attached at the tip of the macro manipulator. The kinematic equations of the macro-micro manipulator system are introduced. By utilizing the redundancy and flexibility of the system, the trajectory planning is discussed. A quasi-static hybrid position/force control algorithm and a dynamic hybrid position/force control algorithm are proposed. Lastly, experimental results are shown to verify the effectiveness of the proposed control algorithms.>

In this paper, a method for PTP and trajectory control of Cartesian type three DOF industrial manipulator with flexible joints are proposed. The elasticity of the manipulator is modeled using the spring-mass model. To establish the equation of motion, the Lagrangian formulation is used. The state equation is derived. Then it is linearized approximately. The PTP control system is formulated. A minimal order observer is used in this PTP control system, and the trajectory control is realized based on the optimal control theory. Furthermore, the desired trajectory is modified to improve the tracking performance. Finally, computer simulation results are given to show the validity of the proposed tracking controller.