Employing skin-like tactile sensors on robots enhances both the safety and usability of collaborative robots by adding the capability to detect human contact. Unfortunately, simple binary tactile sensors alone cannot determine the context of the human contact—whether it is a deliberate interaction or an unintended collision that requires safety manoeuvres. Many published methods classify discrete interactions using more advanced tactile sensors or by analysing joint torques. Instead, we propose to augment the intention recognition capabilities of simple binary tactile sensors by adding a robot-mounted camera for human posture analysis. Different interaction characteristics, including touch location, human pose, and gaze direction, are used to train a supervised machine learning algorithm to classify whether a touch is intentional or not with an F1-score of 86%. We demonstrate that multimodal intention recognition is significantly more accurate than monomodal analyses with the collaborative robot Baxter. Furthermore, our method can also continuously monitor interactions that fluidly change between intentional or unintentional by gauging the user’s attention through gaze. If a user stops paying attention mid-task, the proposed intention and attention recognition algorithm can activate safety features to prevent unsafe interactions. We also employ a feature reduction technique that reduces the number of inputs to five to achieve a more generalized low-dimensional classifier. This simplification both reduces the amount of training data required and improves real-world classification accuracy. It also renders the method potentially agnostic to the robot and touch sensor architectures while achieving a high degree of task adaptability.Note to Practitioners—Whenever a user interacts physically with a robot, such as in collaborative manufacturing, the robot may respond to unintended touch inputs from the user. This may be through body collisions or that the user is suddenly distracted and is no longer paying attention to what they are doing. We propose an easy-to-implement method to augment safety of physical human-robot collaboration by determining whether the touch from a user is intentional or not through the use of robot-mounted basic touch sensors and computer vision. The algorithm examines the location of the user’s hands relative to the touched sensors in addition to observing where the user is looking. Machine learning is then used to classify in real-time, with an F1-score of 86%, whether a touch is intentional or not such that the robot can react accordingly. The method is particularly applicable in collaborative manufacturing contexts, but can also be applied anywhere where a user physically interacts with a robot. We demonstrate the utility of the method in enhancing safety during human-robot collaboration through a simulated collaborative manufacturing scenario with the robot Baxter, but the method can easily be adapted to other systems.

We propose a novel performance metric for articulated robots with distributed directional sensors called thesensor observability analysis(SOA). These robot-mounted distributed directional sensors (e.g., joint torque sensors) change their individual sensing directions as the joints move. SOA transforms individual sensor axes in joint space to provide the cumulative sensing quality of these sensors to observe each task-space axis, akin to forward kinematics for sensors. For example, certain joint configurations may align joint torque sensors in such a way that they are unable to observe interaction forces in one or more task-space (e.g., Cartesian) axes. The resultant sensor observability performance metrics can then be used in optimization and in null-space control to avoid sensor observability in singular configurations or to maximize sensor observability in particular directions. We use the specific case of force sensing in serial robot manipulators to showcase the analysis. Parallels are drawn between sensor observability and traditional kinematic manipulability; SOA is shown to be more generalizable in terms of analyzing non-joint-mounted sensors and can potentially be applied to sensor types other than for force sensing. Simulations and experiments using a custom 3-degree of freedom robot and the Baxter robot demonstrate the utility and importance of sensor observability in physical interactions.

In order to properly integrate humanoid robots in real-life situations, they must be able to collaborate with humans in completing tasks. One of these tasks is the cooperative transportation of a heavy object, which has been widely studied in the humanoids literature. However, the proposed methods rely heavily on six-axis force/torque (F/T) sensors at the wrists, which medium-sized or even some full-sized humanoid robots do not have. This paper proposes an observer to overcome the lack of F/T sensors. The observer is then coupled with a simplified dynamic model of the transportation task allowing the humanoid robot to carry out the task in a stable way. The method is tested in simulation using a humanoid robot that does not have F/T sensors, a NAO robot, to demonstrate its performance. These tests pointed out that the proposed method successfully estimated the interaction forces while generating stable walking patterns.

In this paper, we propose a preliminary definition and analysis of the novel concept of sensor observability index. The goal is to analyse and evaluate the performance of distributed directional or axial-based sensors to observe specific axes in task space as a function of joint configuration in serial robot manipulators. For example, joint torque sensors are often used in serial robot manipulators and assumed to be perfectly capable of estimating end effector forces, but certain joint configurations may cause one or more task-space axes to be unobservable as a result of how the joint torque sensors are aligned. The proposed sensor observability provides a method to analyse the quality of the current robot configuration to observe the task space. Parallels are drawn between sensor observability and the traditional kinematic Jacobian for the particular case of joint torque sensors in serial robot manipulators. Although similar information can be retrieved from kinematic analysis of the Jacobian transpose in serial manipulators, sensor observability is shown to be more generalizable in terms of analysing non-joint-mounted sensors and other sensor types. In addition, null-space analysis of the Jacobian transpose is susceptible to false observability singularities. Simulations and experiments using the robot Baxter demonstrate the importance of maintaining proper sensor observability in physical interactions.

Rather than systematically programming joint or task trajectories, having a human physically manipulate the robot for direct adjustments is more intuitive, saves time, and increases usability, especially for nonexperts. Interactive motion generation or repositioning of humanoid robots through direct human-touch manipulation is not an easy task, especially for high-level multijoint maneuvers. We propose a set of design rules for generating intuitive touch semantics called the “two-touch kinematic chain paradigm.” Our method interprets user touch intentions to allow motions ranging from low-level single joint control to high-level whole-body task control with posture generation, stepping, and walking. The goal is to provide the user with an intuitive protocol for physical humanoid manipulation that can serve the purpose of any application. The generated set of touch semantics is embodied in a finite state machine-based framework using a task-space quadratic programming controller to interpret human touch using capacitive sensors embedded in the humanoid shell, and force-torque sensors located at the ankles and wrists. A position-controlled humanoid robot is used to assess the utility and function of our proposed touch semantics for physical manipulation. Furthermore, a user study with nonexperts examines how our approach is perceived in practice.

This paper presents an inverse kinematics (IK) method which can control future velocities and accelerations for multi-body systems. The proposed IK method is formulated as a quadratic programing (QP) that optimizes future joint trajectories. The features of the proposed IK are: (1) the evaluation of accelerations at future time instances, (2) the trajectory representation that can implicitly integrate the time integral formula into QP, (3) the computation of future Jacobian matrices based on the comprehensive theory of differential kinematics proposed in our previous work. Those features enable a stable and fast IK computation while evaluating the future accelerations. We also conducted thorough numerical studies to show the efficiency of the proposed method.

In this paper, we adapt the Continuous Collision Detection (CCD) method proposed in [1] to efficiently handle the case of spherical and two revolute joints, this kind of joints is very common in modern robotic systems. The new formulations provide more tight motion bounds, thus increase the success rate of checking collision-free paths. We also propose an extension to get the minimum distance to obstacles along a path, this information is primordial as it allows sampling-based motion planning techniques to sort collision-free paths according to their minimum clearance. We have integrated our implementation into a sampling-based motion planning technique and validated it through simulation and on the real Baxter research robot. The experiments revealed that the method not only does not miss any collision between the robot and the obstacles, but also the minimum distance extension provides the path with the maximum clearance at no additional computational cost.

External force observer for humanoid robots has been widely studied in the literature. However, most of the proposed approaches generally rely on information from six-axis force/torque sensors, which the small or medium-sized humanoid robots usually do not have. As a result, those approaches cannot be applied to this category of humanoid robots, which is widely used nowadays in education or research. In this paper, we improve the external force observer in [1] to handle the case of an external force applied in any direction and at an arbitrary point of the robot structure. The new observer is based on Kalman filter formulation and it allows the estimation of the three force components. The observer is simple to implement and can easily run in real time using the embedded processor of a medium-sized humanoid robot such as Nao or Darwin-OP. Moreover, the observer does not require any change to the robot hardware as it only uses measurements from the available force-sensing resistors (FSR) inserted under the feet of the humanoid robot and from the robot inertial measurement unit (IMU). The proposed observer was extensively validated on a Nao humanoid robot. In all conducted experiments, the observer successfully estimated the external force within a reasonable margin of error.

This paper presents a method to maximize the manipulability index of a redundant industrial manipulator while solving the Inverse Kinematics (IK) problem as an optimization problem. Even though the IK problem is a widely studied topic, the integration of the manipulability index into IK has rarely been taken into account. As the relation between this index and the joint variables of the robot is not straightforward, we have tested different formulations using approximated derivatives. Obstacle avoidance has also been considered and the effect of modifying the Cartesian trajectory during the execution of a task has been thoroughly analyzed. Different scenarios have been conducted in simulation and have proven that our modified inverse kinematics solver is efficient to maximize the manipulability index, even with the additional constraint of obstacle avoidance. Moreover, the relaxation of the trajectory constraints leads to a greater manipulability while ensuring the motion smoothness and satisfying the robot physical limitations.

In this paper, we introduce a control strategy aimed at generating a stable walking pattern for a humanoid pushing a heavy load on a cart. In contrast to previous approaches that rely on force/torque sensors to measure the interaction between the robot and the pushed object, we present a simple model-based controller that can be implemented on most robots due to its computationally efficient design. Every aspect of the controller design is covered, from the formulation and validation of the dynamic model, to the implementation and validation on an actual humanoid robot. The experimental results show that the controller can efficiently make a NAO humanoid transport, in a stable way, the equivalent of its own weight on a rolling cart.

Although in recent years several studies aimed at the navigation of robots in cluttered environments, just a few have addressed the problem of robots navigating while moving a large or heavy object. This is especially useful when transporting loads with variable weights and shapes without having to change the robot hardware. On one hand, a major advantage of using a humanoid robot to move an object is that it has arms to firmly grasp it and control it. On the other hand, humanoid robots tend to have higher drift than their wheeled counterparts as well as having significant lateral swing while walking, which propagates to anything they carry. In this work, we present algorithms for a humanoid robot navigating in a cluttered environment while pushing a cart-like object. In addition, the algorithms make use of the hands and arms to articulate the cart when executing tight turns using whole body control scheme to reduce the lateral swing effect on the load and ensure a safe transport. Experiments conducted on a real Nao robot assessed the proposed approach and algorithms, they show that the payload of a humanoid robot can be significantly increased without changing the humanoid robot's hardware, and therefore enact the capacity of humanoid robots in real-life situations.

This paper proposes a reactive leg motion generation method which integrates geometric constraints into its generation process. In order to react given instructions instantaneously or to keep balance against external disturbances, feasible steps must be generated automatically in real-time for safety. In many cases this feasibility has been realized by using predefined steps or admissible stepping regions. However, these predefinitions are often too conservative or valid only in limited situations. The proposed method considers geometric constraints in addition to joint limits during its generation process and it can utilize the ability of the robot to a maximum extent. It can generate feasible walking pattern in real-time by modifying the swing leg motion and the next landing position at each control cycle. The proposed method is validated by experiments using a humanoid robot HRP-2.

This paper proposes a unified optimization framework to solve the time-parameterization problem of humanoid-robot paths. Even though the time-parameterization problem is well known in robotics, the application to humanoid robots has not been addressed. This is because of the complexity of the kinematical structure as well as the dynamical motion equation. The main contribution of this paper is to show that the time parameterization of a statically stablepathto be transformed into a dynamically stabletrajectorywithin the humanoid-robot capacities can be expressed as an optimization problem. Furthermore, we propose an efficient method to solve the obtained optimization problem. The proposed method has been successfully validated on the humanoid robot HRP-2 by conducting several experiments. These results have revealed the effectiveness and the robustness of the proposed method.

In this paper, the imitation of human captured motions by a humanoid robot is considered. The main objective is to reproduce an imitated motion which should be as close as possible to the original human captured motion. To achieve this goal, the imitation problem is formulated as an optimization problem and the physical limits of the humanoid robot are considered as constraints. The optimization problem is then solved recursively by using an efficient dynamics algorithm, which allows the calculation of the gradient function with respect to the control parameters analytically. The simulation results using OpenHRP platform, which is a dynamical simulator for humanoid robot motions, have pointed out that the imitated motions preserve the salient characteristics of the original human captured motion. Moreover the optimization procedure converges well thanks to the analytical calculation of the gradient function.

This paper proposes an whole body motion planning method for humanoid robots in which dynamics is integrated. The method consists of two stages. A collision-free and statically stable path is planned in the first stage and it is transformed into a dynamically stable trajectory in the second stage. Contributions of the method is summarized as follows. (1) A local method plans a C1path while avoiding collisions between non-strictly convex objects. (2) The second stage gives the minimum time trajectory by time parameterization under dynamic balance constraints. (3) Any path reshaping for recovering collision-freeness is not required since the second stage doesnpsilat change shape of the path. Effectiveness of the method is examined by applying it to scenarios of a humanoid robot HRP-2.

Quadratic systems are the first kind of nonlinear systems in which we are interested in order to study polynomial nonlinearities. They can be approximated by bilinear models with nilpotent structure that approximate certain nonlinearities and generate finite degree Volterra series. The hereditary identification algorithm limited until now to linear systems is extended here for identification of nonlinear systems by implementing a canonical structure to the approximant of degree two (quadratic). A NARX (Nonlinear Autoregressive eXogenous input) multidimensional expression is employed in order to perform identification by hereditary computation.

Synthesizing human motion signals is difficult because of its multi-dimensional and nonlinear nature. However, the locomotion is a synchronized motion, which means that these signals are related. Using this property, we propose a new method for modeling the human locomotion and identifying this model. The input signals of our model is the trajectory of pelvis and the outputs are corresponding motion signals of whole human body. To identify this system, we considered it as black-box, for which we propose an adapted method of identification using motion capture. We show that the identified model allows to generate various human locomotion in a fast and efficient way