We aim to develop an efficient programming method for equipping service robots with the skill of performing sign language motions. This paper addresses the problem of transferring complex dual-arm sign language motions characterized by the coordination among arms and hands from human to robot, which is seldom considered in previous studies of motion retargeting techniques. In this paper, we propose a novel motion retargeting method that leverages graph optimization and Dynamic Movement Primitives (DMPs) for this problem. We employ DMPs in a leader-follower manner to parameterize the original trajectories while preserving motion rhythm and relative movements between human body parts, and adopt a three-step optimization procedure to find deformed trajectories for robot motion planning while ensuring feasibility for robot execution. Experimental results of several Chinese Sign Language (CSL) motions have been successfully performed on ABB’s YuMi dual-arm collaborative robot (14-DOF) with two 6-DOF Inspire-Robotics’ multi-fingered hands, a system with 26 DOFs in total.

Programming by demonstration (PBD) transforms the robot programming from the code level to automated interface between robot and human, promoting the flexibility of robotized automation. In this paper, we focus on programming the industrial robot for assembly tasks by parsing the human demonstration into a series of assembly skills and compiling the skill to the robot executables. To achieve this goal, an identification system using multimodal information to recognize the assembly skill, called MASD, is proposed including: 1) an initial learning stage using a hierarchical model to recognize the action by considering the features from action-object effect, gesture, and trajectory and 2) a retrospective thinking stage using a segmentation method to cut the continuous demonstrations into multiple assembly skills optimally. Using MASD, the demonstration of assembly tasks can be explained with high accuracy in real time, driving a hypothesis that a PBD system on the top of MASD can be extended to more realistic assembly tasks beyond pure positional moving and picking. In experiments, the skill identification module is used to recognize the five kinds of assembly skills in demonstrations of both single and multiple assembly skills, and outperforms the comparative action identification methods. Besides integrated with the MASD, the PBD system can generate the program based on the demonstration and successfully enable an ABB industrial robotic arm simulator to assemble a flashlight and a switch, verifying the initial hypothesis. Note to Practitioners-In the conventional robotized automation, the key role of the robot mainly owes to its capacity for repeating a wide variety of tasks with high speed and accuracy in long term, with a cost of days to months of programming for deployment. On the other hand, the new trend of customization brings the new characteristics: production in short cycle and small volume. This irreversible momentum urges the robot to switch from task to task efficiently. The biggest bottleneck here is the tedious programming, which also has high prerequisites for most practitioners in manufacturing. This situation motivates the development of a PBD system that can understand the assembly skills performed by the human experts in the demonstration and accordingly generate the program for robot's execution of the taught task. In this paper, we present a skill decoding system to parse the observational raw demonstration into symbolic sequences, which is the crucial bridge to enable the automatic programming. The system achieves high performance in recognition and is tailored for the PBD in assembly tasks by considering both advantages and disadvantages in the background of assembly, such as controllable environment and limited computational resources. It is particularly useful for assembly tasks with modularized actions based on a set of standard parts. At the perspective of industrial application, the PBD upon the proposed system is a promising solution to improve the flexibility of manufacture, which is expected to be true in midterm but an important step toward this goal.

In robot programming by demonstration (PBD) for assembly tasks, one of the important topics is to inference the poses and spatial relations of parts during the demonstration. In this paper, we propose a world model called assembly graph (AG) to achieve this task. The model is able to represent the poses of all parts, the relations, observations provided by vision techniques and prior knowledge in a unified probabilistic graph. Then the problem is stated as likelihood maximization estimation of pose parameters with the relations being the latent variables. Classification expectation maximization algorithm (CEM) is employed to solve the model. Besides, the contradiction between relations is incorporated as prior knowledge to better shape the posterior, thus guiding the algorithm find a more accurate solution. In experiments, both simulated and real world datasets are applied to evaluate the performance of our proposed method. The experimental results show that the AG gives better accuracy than the relations as deterministic variables (RDV) employed in some previous works due to the robustness and global consistency. Finally, the solution is implemented into a PBD system with ABB industrial robotic arm simulator as the execution stage, succeeding in real world captured assembly tasks.

Robot-centered cells play a more and more important role in the fields of incorporate automation and repetitive processing in order to increase productivity and to improve quality. In practice, it is always desired to achieve maximum/near-maximum throughput in a robotic cell. Sometimes, even a small improvement in throughput will be one of the highlighted objectives in robot scheduling, for instance, in the 3C industry of communication, computer and consumer electronics. In this paper, the scheduling problem in an n-workstation (n<;=8) m-stage robotic cell with parallel workstations is discussed by considering the constraints in real engineering practice, including free/non-free pick-up, allowed time window, and free/non-free process. A new method is proposed for optimal scheduling by means of optimally arranging each blocked cycles in every LCM cycle (Least Common Multiple), and assembling them in a proper order. This method is applied to a practical scheduling problem for a 7-workstation robotic cell. It is shown that the throughput is dramatically enhanced after optimization in the engineering scenario. Moreover, by comparing the resultant increase in revenue with the additional equipment costs, the result diagrams also provide managerial insights into links between throughput and cell layout flexibility.