Synergy provides a practical approach for expressing various postures of a multi-fingered hand. However, a conventional synergy defined for reproducing grasping postures cannot perform in-hand manipulation, e.g., tasks that involve simultaneously grasping and manipulating an object. Locking the position of particular fingers of a multi-fingered hand is essential for in-hand manipulation tasks either to hold an object or to fix unnecessary fingers. When using conventional synergy based control to manipulate an object, which requires locking some fingers, the coordination of joints is heavily restricted, decreasing the dexterity of the hand. We propose a functionally divided manipulation synergy (FDMS) method, which provides a synergy-based control to achieves both dimensionality reduction and in-hand manipulation. In FDMS, first, we define the function of each finger of the hand as either "manipulation" or "fixed." Then, we apply synergy control only to the fingers having the manipulation function, so that dexterous manipulations can be realized with a few control inputs. Furthermore, we propose the Synergy Switching Framework as a method for applying a finely defined FDMS to sequential task changes. The effectiveness of our method is experimentally verified.

The purpose of this study is for a robot to learn picking motions in a logistics warehouse environment. The picking operation performed by a robot often fails owing to the inclination of items placed on a shelf, as well as the minimum clearance between the products and their vinyl packaging. Therefore, we considered acquiring a specific motion trajectory by reinforcement learning. However, because numerous types of items are handled in logistics warehouses, efficient learning is required. Therefore, in this research, we propose a method to efficiently exploration for learning picking an object by determining a focus exploration area for learning based on previous results of different objects.

In caging, an object is geometrically confined by position-controlled robots and never escapes from the constraint. Caging has some advantages over conventional grasping, and its applications have been performed not only in 2D but also in 3D scenes with various actual robots. However, the conditions of complete caging are not always satisfied due to limited robot configuration. This paper studies partial caging, in which an object is incompletely confined by robots or obstacles and is able to escape from the constraint. As an example of partial caging, a circular object moving in the planar hand is considered. We investigate an effect of arrangement of its fingertips, which prevents the object from escaping outside through the gap between the fingertips. Some simulation results show differences of difficulty of escaping for the object according to width of the gap and angle of the fingers. In addition, ease of entering the hand through the gap of the fingers is also evaluated. From these two scores on partial caging, we define an ability index for the hand, which represents the hand can easily capture an object and confine it without any finger motion.

We discuss grasp stability under gravity where each finger makes soft-finger contact with an object. By clustering polygon models of a finger and an object, the contact area between a finger and an object is obtained as the common area between an object cluster and a finger cluster. Then, by assuming the Winkler elastic foundation, the pressure distribution within the contact area is obtained. By using this pressure distribution, we show that we can judge grasp stability under soft-finger contact. We further consider defining a quality measure of a soft-finger grasp by assuming that although the gravitational force is applied to an object, the direction of gravity is unknown. To demonstrate the effectiveness of the proposed approach, we show several numerical examples.

Failure detection and correction is essential in robust systems. In robotics, failure detection has focused on traditional parts assembly, tool breakage, and threaded fastener assembly. However, not much work has focused on sub-mode failure classification. This is an important step in order to provide accurate failure recovery. Our work implemented a novel failure characterization scheme for cantilever snap assemblies. The approach identified exemplars that characterized salient features for specific deviations from a nominal trajectory. Then, a rule based approach with statistical measures was used to identify failure and classify failure sub-modes. Failure sub-mode classification was evaluated by using a reliability measure. Our work classified failure deviations with 88% accuracy. Varying success was experienced in correlating failure deviation modes. Cases with only 1-deviation had 86% accuracy, cases with 2-deviations had 67% accuracy, and cases with 3 deviations had 55% accuracy. Our work is an important step in failure characterization of complex geometrical parts and serves as a stepping stone to enact failure recovery.

This paper presents a knowledge model of everyday objects for semantic grasp. This model is intended for extracting the grasp areas of everyday objects and approach directions for grasping when the 3D point cloud data and the intended purpose are given. Parts that make up everyday objects have functions related to their manipulation. We therefore represent everyday objects in terms of connected parts of functional units. This knowledge model describes the structure of everyday objects and information on their manipulation. The structure of an everyday object describes component parts of the object in terms of simple shape primitives to provide geometrical information and describes connections between parts with kinematic attributes. The information on the structure is used to map the manipulation knowledge onto the 3D point cloud data. The manipulation knowledge of the object includes the grasp areas and approach directions for the intended purpose. Fine grasps suitable for the intended task can be generated by performing a grasp planning with consideration for stable grasp and the kinematics of the robot in the grasp areas and approach directions.

This paper proposes a method for bin-picking for objects without assuming the precise geometrical model of objects. We consider the case where the shape of objects are not uniform but are similarly approximated by cylinders. By using the point cloud of a single object, we extract the probabilistic properties with respect to the difference between an object and a cylinder and consider applying the probabilistic properties to the pick-and-place motion planner of an object stacked on a table. By using the probabilistic properties, we can also realize the contact state where a finger maintain contact with the target object while avoiding contact with other objects. We further consider approximating the region occupied by fingers by a rectangular parallelepiped. The pick-and-place motion is planned by using a set of regions in combination with the probabilistic properties. Finally, the effectiveness of the proposed method is confirmed by some numerical examples and experimental result.

Dexterous manipulation is an important function for working robots. Manipulator tasks such as grasping, assembly and disassembly can generally be divided into several motion primitives. We call such motion primitives “skills” and explain how most manipulator tasks can be composed of sequences of these skills. We will address the issues involved with various types of robots such as maintenance robots and service robots. We have considered hierarchizing the manipulation tasks of these robots since their tasks have become more complex than ever before. Additionally, as errors are seen likely to increase in complex tasks, it is important to implement effective error recovery technology. This paper presents our proposal for a new type of error recovery that uses the concepts of task stratification and error classification which can be expressed specifically using flow charts.

This paper proposes a method for planning the pick-and-place motion of an object by dual-arm manipulators. Our planner is composed of the offline and the online phases. The offline phase generates a set of regions on the object and the environment surfaces and calculates several parameters needed in the online phase. In the online phase, the planner selects a grasping pose of the robot and a putting posture of the object by searching for the regions calculated in the offline phase. By using the proposed method, we can also plan the trajectory of the robot, and the regrasping strategy of the dual-arm. Here, the putting posture of the object can be planned by considering stability of the object placed on the environment. The effectiveness of the proposed method is confirmed by simulation and experimental results by using the dual-arm robot NX-HIRO.

This paper proposes an object placement planner for a grasped object during pick-and-place tasks. The proposed planner automatically determines the pose of an object stably placed near a user assigned point on an environment surface. The proposed method first constructs a polygon model of the surrounding environment, and then clusters the polygon model of both the environment and the object where each cluster is approximated by a planar region. The placement of the object can be determined by selecting a pair of clusters between the object and the environment. We further impose several conditions to determine the pose of the object placed on the environment. We show that we can determine the position/orientation of the object placed on the environment for several cases such as hanging a mug cup on a bar. The effectiveness of the proposed research is confirmed through several numerical examples.

Snap assembly automation remains a challenging task. While progress is being made in localization of parts, force controllers, and control strategies, little work has been done to help the robot reason about its current state, such that if necessary, the robot can assume corrective actions to accomplish the task. Error prone situations caused by the unexpected motion of parts, localization errors, jamming or wedging, cannot be solved by force controllers alone. For this reason we propose a snap assemblies verification system for cantilever-snap fasteners. The verification works in concert with a control strategy that makes use of constraint designs embedded in the snap parts' physical design. The constrained assembly motion generates similar sensory-signal patterns across trials that facilitates force signal discrimination into higher level abstractions of intuitive behavior. This work's contribution is the design of a hierarchical taxonomy for cantilever-snap verification based on increasingly abstract layers that encode relative-change in the task's force signatures. A five-layered taxonomy is built on the concept that relative-change patterns can be classified through a small category set and aided by contextual information. The verification system yielded human apropos intuitive categorizations of task behavior for every state and effectively determined the assembly result. This simple yet effective approach will be expanded to perform probabilistic online system verification to aid in fault tolerance and the automation of cantilever-based snap assemblies.

This paper addresses the use of a service robot system working for humans in a daily life environment. The robot system described here always interacts with humans, and is therefore required to consider the satisfaction of users to provide suitable services. Our goal is to develop a robot system offering a hand-over service according to the preference of users. Here, we construct a task instruction system for the hand-over task utilizing the idea of Service Engineering as the first step in our research. The system can be used to instruct a robot to bring the indicated objects only by selecting task information, such as grasping position and posture, which can be utilized to estimate the user's preference.

This paper presents a grasping system for picking up an indicated object in a complex real-world environment using a parallel jaw gripper. The proposed grasping scheme comprises the following three main steps: (1) A user indicates a target object and provides the system with a task instruction on how to grasp it, (2) the system acquires geometric information about the target object and constructs a 3D environment model around the target by stereo vision using the information obtained from the task instruction, and (3) the system finds a grasp point based on grasp evaluation using the acquired information. As an example of the scheme, we examined the picking up of a cylindrical object by grasping at the brim. An important and advantageous feature of this scheme is that the user can easily instruct the robot on how to perform the object-picking task through simple clicking operations, and the robot can execute the task without exact models of the target object and the environment being available in advance.

This paper presents the development of Robot Technology Components (RTCs) using RT-Middleware (RTM) for designing assistive robotic arms and human-machine interfaces (HMIs) that can be used by the disabled. RTM is an open software platform for a component-oriented robot system. An RTC is the basic functional unit for constructing a robot system using RTM. The interface definition for the basic functionality of RTCs is standardized by the Object Management Group (OMG). Therefore, software modules written as RTCs can be shared and reused in other robot systems. One feature of an assistive robotic arm is that the arm is controlled by manual operation. Various HMIs for manually operating the robotic arms have been developed according to the abilities of each user and the requirements regarding the actions that the robot can perform. These HMIs can be interchangeably connected to the assistive robotic arms by implementing the software modules for the robotic arm and the HMI as RTCs with a common software interface.

In this paper, a training system designed to familiarize persons who have severe upper limb disabilities with robotic arm operation is described. The system combines a robotic arm simulator with various control interfaces and provides training tasks to learn how to operate robotic arms. In our training system, four test subjects used a virtual system to practice how to execute peg insertion tasks, and thus improved their ability to operate actual robotic arms. The training results, along with the results of post-training experiments conducted using an actual robotic arm, demonstrate the effectiveness of the training system.

A grasping and manipulation with a multi-fingered hand is comprised of one or more functional finger actions. We define a functional finger action as a ¿primitive action,¿ and fingers that are used in separate primitive actions in a grasping and manipulation as ¿functional finger isolation.¿ Various grasping and manipulations can be realized by assigning different primitive actions to the functional finger isolation form. This paper presents a towel picking task with a multi-fingered hand by assigning different primitive actions to fingers. In this paper, we describe the towel picking task by a series of primitive actions applied to the towel. Experimental results are given, and we also show that the specification of a hand for the towel picking can be derived by analyzing the primitive actions assigned to the fingers. By describing a grasping and manipulation task with primitive actions, it can be used with other type of robot hands.

To support disabled persons with less muscle strength like muscular dystrophy patients, a single switch and a scanning menu panel is introduced to input method for the manipulator control. The iArm manipulator and its simulator are connected to evaluate the input performance. The evaluation task is the handling of pegs on the pegboard with the manipulator. Muscle dystrophy patients tested our interface to control both of the simulator and the real manipulator with a pegboard. The view of the robot control by the subject is shown on the video.

This paper presents the haptic interface system that consists of a base object and a haptic device. The desired force response is achieved by the combination of the real force response of the base object and the virtual force exerted by the haptic device. The proposed haptic augmented reality (AR) system can easily generate the force response of a visco-elastic object with a cheap haptic device and a base object that has the similar visco-elastic property to the target object. In the demonstration, the force response of the target object was generated by using a haptic device only (VR) and using both a haptic device and a base object (AR), respectively. The evaluation experiments by participants show that the AR method has better performance than the VR method. This result indicates the potential of the proposed haptic AR interface.

We are developing a manipulator system in order to support disabled people with less muscle strength such as muscular dystrophy patients. Such a manipulator should have an easy interface for the users to control it. In this paper, we report on the construction of the user interface of the system using RT-middleware. RT-middleware is an open software platform for robot systems. Therefore other interface components or robot components which are adapted to other symptoms can be replaced with the interface without any change of the contents. A single switch is introduced as the input device for the manual control of the robot arm in our interface. The scanning menu panel is designed to perform various actions of the robot arm with the single switch. A manipulator simulation system was constructed to evaluate the input performance. Two muscular dystrophy patients tried of our interface to control the robot simulator and made comments.

This paper presents a task instruction scheme for a service robot operating in a daily life environment. It is difficult for the robot to execute tasks automatically in the daily life environment, because it should handle various objects and the working environment is complex. In such an environment, the concept of shared autonomy by which the robot can share a task with a human is important. The task instruction that we propose is as follows: An operator clicks on a target object in a camera image and the system gets the position of the clicked point in the work space from the corresponding range image. Then, the operator selects the task model and attaches the task information to the position and the robot can perform its task safely and reliably using this task information. The features of the task instruction are as follows. An object model is composed of a list of task models which can be applied to the object category. The user recognizes the object type and its situation in the environment, and instructs a robot to execute a task by selecting a task model according to the object type and situation.

This paper presents a simple task instruction scheme for a supporting robot in a daily life environment. It is difficult for a robot to perform an automatic task in the daily life environment because its sensors are not perfect in such an environment and the world model of the robot cannot cover the specific details of this environment. On the other hand, a user of the robot may control the robot by full teleoperation scheme. But the teleoperation puts the burden on the user. Instead, we propose a new instruction scheme for the supporting robot in which the user selects a task model which the robot can perform and indicates the key points on the monitoring image of the robot task environment according to the robot's request. The monitoring image is captured by a stereo camera and the range image is stored simultaneously. Therefore the system can detect the important position parameters of the target object with the indication provided by the user and the robot can complete the task with the information which the user simply gives the system.

In this paper, the CPG-based control method for the rotating manipulation that can adaptively change the issuing cycle of motion-triggers is proposed. The CPG model consists of the mutual inhibition network and the rotating manipulation is performed by the motion-triggers that are issued from the CPG model. The smooth change of the desired velocity can be given by the output of neurons that keep smooth increase and decrease. Experimental results using 4-fingered hand system suggest that the issuing cycle of the motion-triggers can be adaptively changed by the feedback of joint angle according to an object size.

Analysis of human hand grasp provides information that can be used to develop a grasping algorithm of robot hands that takes advantage of human knowledge and experience. In analysing the human hand grasp, not only motion of the fingers but also contact force acting on the fingertips should be measured. This paper presents a new master hand which can be attached to a human hand and measures motion of the human fingertips and contact force acting on the fingertips. Our purpose in developing the master hand is to clarify the strategy of a human hand grasp, which is subjected to the same physical constraints as those of a robot hand, and use the obtained information to develop a grasping algorithm of robot hands. Concepts of the development, mechanism and features of the master hand are described.

We present an exchangeable hand system designed for portable manipulators. It is developed for achieving lightness in weight especially at the end of a robot arm to improve performance and safety of a complete manipulation system. We prepared several end effectors with simple functions that can be used interchangeably. We describe concepts and details of our system including a new exchanger mechanism and a new versatile hand called WitchHand.

This paper analyzes the indeterminate grasp force in power grasps with a rigid body model. A power grasp overconstrains an object with multiple contact points using the surfaces of finger links and palm. It is known that the overconstrained grasp results in the static indeterminacy of the grasp force. This paper shows that it also results in an infeasible combination of the sliding directions at the contact points. A static friction force acts only in the opposite direction of the trend of sliding. This characteristic of friction restricts the grasp force in power grasps although it is indeterminate. We first show an example to illustrate the restriction on the grasp force, and then formulate it formally. This analysis leads to a special case where the grasp force is unique.

We demonstrate the robust execution of two regrasping primitives, rotation and pivoting on a four-fingered hand, based on the real-time sensors data based control. We indicate frequently occurring grasp errors and faults during execution of the two regrasping primitives. The sensors data is obtained from two sources. One is force-torque sensors mounted on the finger tips of the four fingers. The other is a laser range sensor mounted on the palm of the hand. The grasp information acquired through these two sources is useful for both sensing the errors and controlling the fingers. The paper proposes strategies to recover or prevent faulty grasps during manipulation, which may lead to unwanted object movements or grasp failures. The object to be manipulated is a polygonal prism, the model of which is known a priori.

A method is proposed to classify and describe grasping and manipulation. It is based on the three functions of grasping surfaces, namely object-supporting, pressing and wrapping functions. The method provides a simple description for grasping and manipulation and it can express their physical fundamentals. The description is illustrated through example grasp and manipulation by a human hand. Because it is independent from specific hardware, the obtained interpretations of tasks by a human hand can be easily applied to robotic hand systems. The new method can be expected to change current approaches to designing robot hands.

Presents a delivery by hand between a multifingered hand and a human based on force-torque sensing. Of all cooperated tasks between humans and robots, the most important and basic factor is delivery of objects by hand. To insure positive delivery, the robot should open its hand only after making sure that the object has been received by the human. The human releases the object only after confirming that the robot can actually grasp the object stably. The features of the hand-delivery system introduced in the paper include that (1) the robot recognizes successful transfer of the object to a human when the human changes the grasping condition, (2) the human guides the position and orientation of the grasped object to a stable grasp, and (3) the robot, in response to the above guide, regrasps the object to realize a stable grasping condition. The paper introduces evaluation for stable grasp, and presents a new technique for sensor based regrasp manipulation and delivery by hand between a human and robot based on force-torque sensing which reflects the result of stability evaluation. Finally, experimental results are shown to demonstrate the effectiveness of the proposed scheme.

The grasp force in power grasps is not necessarily determined uniquely by joint torques when Coulomb friction is assumed in the contact points between a grasped object and finger links. This paper analyzes the indeterminate grasp force with their rigid body models. This paper shows that contact sliding directions are constrained in power grasps. The frictional force acts only in the opposite direction of sliding. This characteristic of friction restricts the grasp force in power grasps although it is indeterminate in general. First the authors show an example to illustrate this restriction imposed on a grasp force. A special case where it is unique is shown.

A multifingered hand with touch sensors has grasping and manipulating functions, and has the function of a sensor probe. We have been studying the merging of these two functions, the acquisition of an object model by manipulation with a multifingered hand, and determining the next finger action based on the acquired information. By manipulating objects while acquiring the object models, we expect to be able to manipulate many types of objects and unknown objects. This paper discusses a multifingered hand system for acquiring an object model while manipulating the object, by using the contact point information between the fingertips and object.

This paper presents dextrous manipulation by a non-articulated multifingered hand. To achieve dextrous manipulation by a non-articulated multifingered hand, a parallel-jaw gripper which has a turntable at each fingertip has been developed. This hand has only one free rotation at each fingertip, but can achieve various manipulations. The design of this hand and manipulation using this hand are easier than with an articulated multifingered hand. In this paper, the mechanism of the new hand is presented and the manipulations possible with this hand when an object is placed in the lower pair are analyzed. The required conditions for achieving these manipulations are calculated.>

A multifingered hand can reorient an object by regrasping it in a hand. This paper presents a planner which plans a sequence of repositioning of fingers for horizontal rotation of an object for a desired angle. The authors previously developed an algorithm for computing contact positions of fingers where they can maintain equilibrium. The algorithm is used to compute a new contact position of a removed finger where there is a finger which can be removed next. Based on these computations, the authors develop an algorithm for searching for a sequence of repositioning of fingers. Examples show that the algorithm can efficiently find sequences which require the number of repositioning steps.>