sponsorship: This work was supported by Flanders Make (Flanders Make is the Flemish Strategic Research Center for the manufacturing industry), through the Flanders Make SBO project AUTOCRAFT. This article was recommended by Associate Editor C. P. Hung. (Flanders Make (Flanders Make is the Flemish Strategic Research Center for the manufacturing industry) through Flanders Make SBO project AUTOCRAFT)

This paper describes a solution for pick-and-place tasks that do not require high precision throughout the execution, using a robotised gantry crane. Two common issues that occur when using a crane are addressed, and solutions are provided. First, the lack of rotational controllability is overcome by designing a gripper that passively aligns itself with the handle of the payload using a simple, robust control strategy. Second, the active workspace is expanded by using controlled, dynamic motions, based on a variable-length pendulum model. Thus, the workspace is no longer limited to positions directly accessible from above, as is the case with quasi-static control methods. The robustness and effectiveness of the proposed solution was validated by a grasping, and a shelf insertion experiment. The robot was able to grasp the payload during all 48 trials. Failures were detected and recovery strategies were implemented. The handle could also be reliably ungrasped. During the shelf insertion, imperfect trajectory tracking caused a significant error during the unobservable part of the trajectory. Nevertheless, the actual placement position was always close to the desired position.

This paper studies sand profile grading, a manipulation task to obtain a desired geometric curve in sand. Manipulating sand is challenging because like other amorphous materials, its properties are difficult to estimate and emergent effects such as collapses may occur which both influence the manipulation outcome. To tackle these challenges, humans iterate and adapt their manual actions to the observed material states. In this paper, we propose to replicate this adaptive and iterative approach on a robotic profile grading task. Our results demonstrate that (1) tool insertion adaptation reduces force limit violations during tool-material interactions, (2) grading angle adaptation ensures no undercutting or collisions while allowing for cutting or smoothing the sand profile, and (3) adapting progress speed to task evolution provides a balance between grading precision and execution time. This paper's findings pave the way for generalized and transferable robotic systems manipulating various amorphous materials and automating a larger set of construction tasks and beyond.

This paper presents a framework for multi-agent navigation in structured but dynamic environments, integrating three key components: a shared semantic map encoding metric and semantic environmental knowledge, a claim policy for coordinating access to areas within the environment, and a Model Predictive Controller for generating motion trajectories that respect environmental and coordination constraints. The main advantages of this approach include: (i) enforcing area occupancy constraints derived from specific task requirements; (ii) enhancing computational scalability by eliminating the need for collision avoidance constraints between robotic agents; and (iii) the ability to anticipate and avoid deadlocks between agents. The paper includes both simulations and physical experiments demonstrating the framework's effectiveness in various representative scenarios.

We propose a shared semantic map architecture to construct and configure Model Predictive Controllers (MPC) dynamically, that solve navigation problems for multiple robotic agents sharing parts of the same environment. The navigation task is represented as a sequence of semantically labeled areas in the map, that must be traversed sequentially, i.e. a route. Each semantic label represents one or more constraints on the robots’ motion behaviour in that area. The advantages of this approach are: (i) an MPC-based motion controller in each individual robot can be (re-)configured, at runtime, with the locally and temporally relevant parameters; (ii) the application can influence, also at runtime, the navigation behaviour of the robots, just by adapting the semantic labels; and (iii) the robots can reason about their need for coordination, through analyzing over which horizon in time and space their routes overlap. The paper provides simulations of various representative situations, showing that the approach of runtime configuration of the MPC drastically decreases computation time, while retaining task execution performance similar to an approach in which each robot always includes all other robots in its MPC computations.

We derive a family of efficient constrained dynamics algorithms by formulating an equivalent linear quadratic regulator (LQR) problem using Gauss' principle of least constraint and solving it using dynamic programming. Our approach builds upon the pioneering (but largely unknown)$O(n + m^{2}\;d + m^{3})$solver by Popov and Vereshchagin (PV), where$n$,$m$, and$d$are the number of joints, number of constraints, and the kinematic tree depth, respectively. We provide an expository derivation for the original PV solver and extend it to floating-base kinematic trees with constraints allowed on any link. We make new connections between the LQR's dual Hessian and the inverse operational space inertia matrix (OSIM), permitting efficient OSIM computation, which we further accelerate using matrix inversion lemma. By generalizing the elimination ordering and accounting forMuJoCo-type soft constraints, we derive two original$O(n + m)$complexity solvers. Our numerical results indicate that significant simulation speed-up can be achieved for high dimensional robots like quadrupeds and humanoids using our algorithms as they scale better than the widely used$O(nd^{2} + m^{2}\;d + d^{2}\;m)$LTL algorithm of Featherstone. The derivation through the LQR-constrained dynamics connection can make our algorithm accessible to a wider audience and enable cross fertilization of software and research results between the fields.

The Unified Robot Description Format (URDF) and, to a lesser extent, the COLLAborative Design Activity (COLLADA) format are two of the most popular domain-specific languages (DSLs) to represent kinematic chains in robotics with support in many tools including Gazebo, MoveIt!, KDL or IKFast. In this paper we analyse both DSLs with respect to their structure and semantics as seen by tools that produce or consume such representations. For the former, we notice a tight coupling of various unrelated domains like kinematics and dynamics with visualisation, control or even specific simulators. For the latter, a key insight is that both DSLs target human developers and leave important design decisions like the choice of joint attachment frames implicit or hidden in the documentation. The lessons learned from this analysis guide us to an improved interchange format by designing composable, loosely coupled models with complete metamodels that unambiguously define the model semantics. We substantiate our findings with concrete examples. Furthermore, we compose solver algorithms on top of the kinematic chain representation. As a consequence of the above analysis and decomposition we can systematically apply structure- and semantics-conserving model-to-code transformations to those algorithms.

As robots are equipped with software that makes them increasingly autonomous, it becomes harder for humans to understand and control these robots. Human users should be able to understand and, to a certain amount, predict what the robot will do. The software that drives a robotic system is often very complex, hard to understand for human users, and there is only limited support for ensuring robotic systems are also intelligible. Adding intelligibility to the behavior of a robotic system improves the predictability, trust, safety, usability, and acceptance of such autonomous robotic systems. Applying intelligibility to the interface design can be challenging for developers and designers of robotic systems, as they are expert users in robot programming but not necessarily experts on interaction design. We propose Choreobot, an interactive, online, and visual dashboard to use with our reference framework to help identify where and when adding intelligibility to the interface design is required, desired, or optional. The reference framework and accompanying input cards allow developers and designers of robotic systems to specify a usage scenario as a set of actions and, for each action, capture the context data that is indispensable for revealing when feedforward is required. The Choreobot interactive dashboard generates a visualization that presents this data on a timeline for the sequence of actions that make up the usage scenario. A set of heuristics and rules are included that highlight where and when feedforward is desired. Based on these insights, the developers and designers can adjust the interactions to improve the interaction for the human users working with the robotic system.

This paper proposes a method to integrate the rich semantic data-set provided by Building Information Modeling (BIM) with robotics world models, taking as use case indoor semantic localization in a large university building. We convert a subset of semantic entities with associated geometry present in BIM models and represented in the Industry Foundation Classes (IFC) data format to a robot-specific world model representation. This representation is then stored in a spatial database from which the robot can query semantic objects in its immediate surroundings. The contribution of this work is that, from this query, the robot’s feature detectors are configured and used to make explicit data associations with semantic structural objects from the BIM model that are located near the robot’s current position. A graph-based approach is then used to localize the robot, incorporating the explicit map-feature associations for localization. We show that this explainable model-based approach allows a robot equipped with a 2D LiDAR and odometry to track its pose in a large indoor environment for which a BIM model is available.

In this work a graph-based, semantic mapping approach for indoor robotics applications is presented, which is extending OpenStreetMap (OSM) with robotic-specific, semantic, topological, and geometrical information. Models are introduced for basic indoor structures such as walls, doors, corridors, elevators, etc. The architectural principles support composition with additional domain and application-specific knowledge. As an example, a model for an area is introduced, and it is explained how this can be used in navigation. A key advantage of the proposed graph-based map representation is that it allows exploiting the hierarchical structure of the graphs. Finally, the compatibility of the approach with existing, grid-based motion planning algorithms is shown.

We investigate two major limiting factors in the design and implementation of modern dynamics solvers that interfere with their full utilization in versatile, manipulation-driven robotic software architectures. The first limitation originates from the design of those solvers which aims at computational efficiency while neglecting composability. Instead, we advocate to design the solvers in such a way that they exploit linearity in the equations of motion to fully decompose the state of a kinematic chain. This enables a versatile recomposition and more flexible applications. Secondly, we have observed that most implementations follow the programming principle of information hiding. Consequently, the internal state that is used to compute motion control commands is withheld from other parts of the software architecture. We tackle this problem by following a dataflow programming paradigm and separating the software's dataflow from the control flow. Thereafter, we demonstrate those two simple, yet effective strategies to overcome the limitations along various case studies.

This paper presents a study of haptic perception of virtual stiffness and the influence of visual feedback in virtual reality. In this study, a novel and lightweight haptic glove (ExoTen-Glove) based on Twisted String Actuation (TSA) system is presented and evaluated. This system has two independent TSA modules with integrated force sensors and small-size DC motors. ExoTen-Glove provides force feedback to the users during the execution of grasping virtual objects. The overall design, the controller and the preliminary experimental evaluation of the ExoTen-Glove have been shown in this paper. Different experiments have been performed in virtual reality environment using HTC VIVE headset with 2 degrees of freedom grasping tasks, squeezing a pair of virtual springs with the participant's thumb and index fingers. The aim of this study is to illustrate the benefit of using ExoTen-Glove to distinguish stiffness of a pair of virtual springs and the role of the visual feedback. The results show that the users use not only haptic cues but also use visual cues in detecting spring stiffness difference.

Factories of the future must be agile to adapt to rapidly changing customer needs, market volatility and shortened product life cycles. This requires flexibility in hardware and software at distinct levels of the factory and manufacturing process: multipurpose machines with fast change-overs, easy to use reconfigurable software and distributed decision-making are key. This paper leverages the new Industry 4.0 design principles to cope with these new manufacturing requirements: (i) distributed auction-based coordination allows local decision-making and task allocation, (ii) distributed model predictive control-based transportation enables free space collision avoidance of automated guided vehicles (AGVs), and (iii) distributed vision-based localisation provides scalable and dynamic position information of key resources on the factory floor. Furthermore, these contributions are brought together in a lab-scale reconfigurable manufacturing system to showcase modularity and distributed decision-making at several levels of a manufacturing process' logistics.

This paper presents a novel lightweight and simple TSA-based (twisted string actuation) wearable haptic glove (ExoTen-Glove). This system is using two independent twisted string actuators with integrated force sensors and small-size DC motors. The proposed system can provide users force feedback during the execution of grasping virtual objects. The design of the TSA-based ExoTen-Glove, description of the TSA system, the controller and the preliminary experimental evaluation of the proposed system has been presented in this paper. This device has been evaluated by an experiment in virtual reality environment using HTC VIVE headset with 2 degrees of freedom grasping tasks, where the participants were squeezing a real spring with their thumb and index finger and compare it with a virtual spring stiffness. The results prove the applicability of the ExoTen-Glove for rehabilitation and haptic purposes.

Symbolic reasoners generate plans which are often not exploiting the robot capabilities and are sensitive to runtime disturbances. This work proposes a scheduler as an interface between a discrete, symbolic plan and a motion control based on constraint optimization. Acting as a local reasoner, the scheduler valuates a set of predicates to decide when an action will be executed. Given a task specification which describes how the action should be realized, the scheduler configures the controller at runtime. A demonstration will be provided considering an “open drawer” scenario.

This paper presents a distributed world model that is able to adapt to changes in the Quality of Service (QoS) of the communication layer by online reconfiguration of perception algorithms. The approach consists of (a) a mechanism for storage, exchange and processing of world model data and (b) a feedback loop that incorporates reasoning techniques to adapt to QoS changes immediately. The latter introduces a Level of Detail (LoD) metric based on a spatial resolution in order to infer an upper bound for the amount of data that can be transmitted without violating an application specific transmission delay. Experiments have been performed with Octree-based subsampling techniques applied to data originating from a RGB-D camera using simulated and real-world data sets for timevarying bandwidth values as employed QoS measure.

In the context of touch-based object localisation, solving the problem of “where to sense next” is a challenging task due to the curse of dimensionality related to belief-state reasoning. We present a constrained optimisation scheme that computes the next best action maximising the trade off between (i) the localisation information gain, (ii) the time required for its computation, and (iii) the motion-execution time. This allows the robot programmer to have a deterministic influence on the length of every sensing action. The proposed methodology is applied to localise a solid object in 3D using a Staubli RX90 robot equipped with a force-torque sensor coupled with a spherical end effector. A case-study comparison of the task executed with two different time constraints is presented.

Combining tasks, melding their activities in sequence and in parallel, in order to achieve the desired goal is a challenging research topic. In many practical applications, tasks could be fulfilled even though the robot does not achieve a perfect matching with a given quantitative objective. Thus, it could be possible to carry out other activities, mildly conflicting with the current goal, yet providing benefits for the overall execution of a complex sequence of tasks. This paper proposes i) a criteria to evaluate the execution of a monitored task, ii) an enriched task specification to express the tolerance with which a goal is fulfilled, so that iii) a sequence of tasks can be executed concurrently, on the basis of monitored quantities. Furthermore, the paper reports a classification of conflicting scenarios for a finer selection of the scheduler policy. Finally, results from experimental scenarios show the potential benefits of the proposed methodology.

This paper presents a novel approach for representing and maintaining a shared 3D world model for robotic applications. This approach is based on the scene graph concept which has been adapted to the requirements of the robotic domain. A key feature is the temporal and centralized sharing of all available 3D data in the leaves of the graph structure. The approach enables tracking of dynamic objects, incorporates uncertainty and allows for annotations by semantic tags. A demonstration is given for a perception application that exploits the temporal sharing of 3D data. A Region of Interest (ROI) is extracted from the stored scene data in order to accelerate processing cycle times.

In constraint-based programming, robot tasks are specified and solved as optimization problems with sets of constraints and one or multiple objective functions. In our previous work, we presented (i) a generic modeling approach for geometrically complex robot tasks, including the modeling of parametric uncertainty, in order to allow the robot task programmer to specify the optimization problem without explicitly writing down the different (possibly numerous and involved) constraint equations, and (ii) methods for solving these optimization problem online in the instantaneous case (reactive control), and offline in the non-instantaneous case (trajectory planning). This paper has two contributions. First, it extends our framework to include task constraints (e.g. tracking a curve) that are not given as explicit functions of time. These constraints are highly relevant in practice, for example to facilitate time-optimal path planning combined with other constraints. Second, it extends our framework to user-configurable task horizons when solving the optimization problem, to allow task programmers to make a trade-off between computational speed and (global) task optimality. Both of these novel framework extensions are illustrated by a time-optimal laser tracing experiment.

Deploying a complex robot software architecture on real robot systems and getting it to run reliably is a challenging task. We argue that software deployment decisions should be separated as much as possible from the core development of software functionalities. This will make the developed software more independent of a particular hardware architecture (and thus more reusable) and allow it to be deployed more flexibly on a wider variety of robot platforms. This paper presents a domain-specific language (DSL) which supports this idea and demonstrates how the DSL is used in a model-driven engineering-based development process. A practical example of applying the DSL to the development of an application for the KUKA youBot platform is given.

This paper presents a scene graph for geometric relations between rigid bodies that keeps track of poses and twists of rigid bodies in the scene. The scene graph relies on semantic pose and twist representation, making it invariant to the actual coordinate representation at hand. This makes the scene graph more general and interoperable than most scene graphs currently available. The presented scene graph takes into account constraints imposed by particular coordinate representations, allows for constant poses, answers semantic pose and twist queries, and provides built-in semantic consistency checks. Since the scene graph also keeps track of the twist, it allows native twist calculations, as opposed to deriving the velocities from the poses in the graph. This paper comes with software released under a dual BSD/LGPLv2.1 license.

This paper presents the application of a Bayesian nonparametric time-series model to process monitoring and fault classification for industrial robotic tasks. By means of an alignment task performed with a real robot, we show how the proposed approach allows to learn a set of sensor signature models encoding the spatial and temporal correlations among wrench measurements recorded during a number of successful task executions. Using these models, it is possible to detect continuously and on-line deviations from the expected sensor readings. Separate models are learned for a set of possible error scenarios involving a human modifying the workspace configuration. These non-nominal task executions are correctly detected and classified with an on-line algorithm, which opens the possibility for the development of error-specific recovery strategies. Our work is complementary to previous approaches in robotics, where process monitors based on probabilistic models, but limited to contact events, were developed for control purposes. Instead, in this paper we focus on capturing dynamic models of sensor signatures throughout the whole task, therefore allowing continuous monitoring and extending the system ability to interpret and react to errors.

Task specification models define the activities to be executed by a robot in order to achieve its goal. Classical examples are the sequences involved in assembly or pick and place tasks. This work introduces the preview coordination execution model, an extension to the traditional way in which the execution of such task specifications is coordinated at runtime. Instead of taking activities one-by-one as defined in the task specification model, preview coordination optimizes the task scheduling based on knowledge about the likelihood that not just the activities required by the current state can be executed, but that also one or more of those related to future states of the system can be activated. An experiment with mobile manipulation tasks illustrates the benefits of preview coordination.

Current state-of-the-art robot program development needs expert programmers. Moreover, most robot programs developed today are robot hardware and software specific, and therefore little reusable without modifications. This paper realizes easier robot (re-)programming, by software framework independent models that can be executed using different hard- and software platforms. First, the paper focuses on the formalization of the tasks to be fulfilled by a robot, more specifically constraint-based programming tasks using a Domain Specific Language (DSL). Second, it gives a reference implementation in Lua [1]. The presented DSL makes it easy to develop applications, yet is powerful to execute. It enables automatic model verification and code generation for different hard- and software platforms, diminishing code debugging efforts. Experimental validation shows the ease of creating an application and adapting it, the reduction of the amount of hand-written code, and the debugging aid offered through meaningful errors returned by model verification.

This paper presents a hybrid pose / wrench control framework for quadrotor helicopters, allowing direct contact of the quadrotor with its environment and stable motion while in contact. The presented work explicitly takes into account the quadrotor's underactuation and utilizes a wrench estimator, able to estimate the wrench generated by the quadrotor using only quadrotor inputs and pose measurements. Experimental results of the quadrotor in pose and hybrid pose-wrench control mode are presented, showing stable behaviour while moving in contact. Whereas a lot of research nowadays is focused mainly on free-flight applications, stable and robust wrench control of quadrotors in direct contact with their environment, opens up new application areas for this class of flying robots.

Modern and future robotic applications will have to integrate many software components. Inevitably, the components will be provided by different vendors, and will not be designed together, and in many cases also not implemented from the same source code repository. This paper presents a discussion on how to deal with this multi-component, multi vendor situation. This situation includes the least optimal case when most or all of the components are "legacy", in the sense that they have to be integrated as binary pieces of code, without the possibility to change the components for the sole purpose of facilitating system-level integration. The presented approach is illustrated by an application that integrates (i) two KUKA Light-Weight Robot FRI control components, (ii) a set of ROS components to control a Willow Garage PR2 robot, (iii) a Blender component for on-line visualization, and (iv) a set of Orocos/RTT components that take care of the data flow communication and some simple coordination between all components.

Removing the safety fences that separate humans and robots, to allow for an effective human-robot interaction, requires innovative safety control systems. An advanced functionality of a safety controller might be to detect the presence of humans entering the robotic cell and to estimate their intention, in order to enforce an effective safety reaction. This paper proposes advanced algorithms for cognitive vision, empowered by a dynamic model of human walking, for detection and tracking of humans. Intention estimation is then addressed as the problem of predicting online the trajectory of the human, given a set of trajectories of walking people learnt offline using an unsupervised classification algorithm. Results of the application of the presented approach to a large number of experiments on volunteers are also reported.

This paper discusses the theoretical background and practical implementation of a large-scale, low-performance haptic remote control setup. The experimental system consists of a pair of KUKA Light Weight Robots (LWR) coupled to a Willow Garage Personal Robot (PR2) via two different robotic frameworks. The haptic “performance” is, of course, not comparable to dedicated haptic applications, but has its use as a test-bed for interaction between “legacy” service robot systems, that have not been especially designed for mutual haptic interaction. We discuss some major application problems, and the future work needed for nonuniform robot coupling. Beside haptic coupling, we provide the human operator with visual feedback. To this end, the head movements of the human operator are coupled to the head movement of the PR2 and the images of the eye cameras are displayed to the human operator using a wearable display. The presented teleoperation application is furthermore an example of the integration of two component-based robotic frameworks namely OROCOS (Open Robot Control Software)and ROS (Robot Operating System) Experimental results regarding the haptic coupling are presented using an “artistic” painting task for qualitative results, and a hard contact at the slave side for quantitative results.

Most of today's robotic task descriptions are designed for a single software and hardware platform and thus can not be reused without modifications. This work follows the meta-model approach of Model Driven Engineering (MDE) to introduce the concepts of Domain Specific Languages (DSL) and of Model Transformations to the domain of hybrid force-velocity controlled robot tasks, as expressed in (i) the Task Frame formalism (TFF), and (ii) a Statechart model representing the discrete coordination between TFF tasks. The result is a representation in MDE's M0, M1, M2 and M3 form, with increasingly robot and software independent representations, that do remain instantaneously executable, except obviously for the M3 metametamodel. The Platform Specific Model information can be added in three steps: (i) the type of the hybrid force-velocity controlled task, (ii) the hardware properties of the robot, tool and sensor, and (iii) the software properties of the applied execution framework. We demonstrate the presented approach by means of an alignment task executed on a Willow Garage PR2 and a KUKA Light Weight Robot (LWR) arm.

This paper proposes a novel online two-level multitarget tracking and detection (MTTD) algorithm. The algorithm focuses on multitarget detection and tracking for the case of multiple measurements per target and for an unknown and varying number of targets. Information is continuously exchanged in both directions between the two levels. Using the high level target position and shape information, the low level clusters the measurements. Furthermore, the low level features automatic relevance detection (ARD), as it automatically determines the optimal number of clusters from the measurements taking into account the expected target shapes. The high level's data association allows for a varying number of targets. A joint probabilistic data association algorithm looks for associations between clusters of measurements and targets. These associations are used to update the target trackers and the target shapes with the individual measurements. No information is lost in the two-level approach since the measurement information is not summarized into features. The target trackers are based on an underlying motion model, while the high level is supplemented with a filter estimating the number of targets. The algorithm is verified using both simulations and experiments using two sensor modalities, video and laser scanner, for detection and tracking of people and ants.

We presented our constraint-based programming approach, iTaSC, that formulates instantaneous sensor-based robot tasks as constraint sets, and subsequently solves a corresponding least-squares problem to obtain control set points, such as desired joint velocities or joint torques. This paper further extends this approach, (i) by explicitly supporting the inclusion of inequality constraints in the task and (ii) by supporting a broader class of objective functions for translating the task constraints into robot motion. These extensions are made while retaining a tractable mathematical problem structure (a convex program). Furthermore, first results on extending the approach to non-instantaneous tasks are presented. As illustrated in the paper, the power of the approach lies (i) at its versatility to specify a wide range of robot behaviors and the ease of making task adjustments, and (ii) at its generic nature, that permits using systematic procedures to derive the underlying control equations.

This paper investigates and demonstrates the feasibility of identifying contact dynamics parameters forstiffroboticpayloadsusing a robotic system. The contact dynamics model for stiff payloads is motivated, and theoretical parameter values and bounds are provided. Then, the effect of nonidealities such as surface roughness and plastic deformation on the theoretical values is demonstrated. A row-wise-scaled total least-squares parameter estimation algorithm is proposed and applied to experimental data measured using the special purpose dexterous manipulator task verification facility manipulator at the Canadian Space Agency. The experimental results are compared to a separate set of experiments with a material testing machine as well as finite-element modeling results. Finally, the experimental findings are generalized by providing guidelines for the maximum identifiable payload stiffness as a function of the position resolution, the maximum exertable force, and the structural stiffness of the robotic system.

This paper proposes and experimentally validates a Bayesian network model of a range finder adapted to dynamic environments. The modeling rigorously explains all model assumptions and parameters, improving the physical interpretation of all parameters and the intuition behind the model choices. With respect to the state of the art model [1], this paper proposes: (i) a different functional form for the probability of range measurements caused by unexpected objects, (ii) an intuitive explanation for the discontinuity encountered in the cited paper, and (iii) a reduction in the number of model parameters, while maintaining the same representational power for experimentally obtained data. The proposed beam model is called RBBM, short for rigorously Bayesian beam model. A maximum-likelihood estimation and a variational Bayesian estimation algorithm (both based on expectation-maximization) are proposed to learn the model parameters.

Robots are increasingly used to perform complex tasks, which often involve interaction and contact with unstructured environments. By identifying geometric uncertainties and the dynamic behavior of the environment on-line, the autonomy of intelligent robot systems can be considerably improved. This paper considers the 2D case of an industrial robot equipped with a probe to explore an unknown environment. The goal is to estimate from the measured end-effector position, velocity and forces not only the environmental contact dynamics parameters, but also geometric parameters such as the environment position and orientation, and the position of the probe end-point with respect to the robot end-effector. To this end, a Kalman filter based algorithm is proposed, which enforces physical constraints and which is executed in an event-triggered way to improve convergence and robustness. Experimental results illustrate the viability of the proposed algorithm.

This paper proposes and experimentally validates a Bayesian network model of a range finder adapted to dynamic environments. All modeling assumptions are rigorously explained, and all model parameters have a physical interpretation. This approach results in a transparent and intuitive model. With respect to the state of the art beam model this paper: (i) proposes a different functional form for the probability of range measurements caused by unmodeled objects, (ii) intuitively explains the discontinuity encountered in te state of the art beam model, and (iii) reduces the number of model parameters, while maintaining the same representational power for experimental data. The proposed beam model is called RBBM, short for Rigorously Bayesian Beam Model. A maximum likelihood and a variational Bayesian estimator (both based on expectation-maximization) are proposed to learn the model parameters. Furthermore, the RBBM is extended to a full scan model in two steps: first, to a full scan model for static environments and next, to a full scan model for general, dynamic environments. The full scan model accounts for the dependency between beams and adapts to the local sample density when using a particle filter. In contrast to Gaussian-based state of the art models, the proposed full scan model uses a sample-based approximation. This sample-based approximation enables handling dynamic environments and capturing multi-modality, which occurs even in simple static environments.

A first step towards truly versatile robot assistants consists of building up experience with simple tasks such as the cooperative manipulation of objects. This paper extends the state-of-the-art by developing an assistant which actively cooperates during the point-to-point transportation of an object. Besides using admittance control to react to interaction forces generated by its operator, the robot estimates the intended human motion and uses this identified motion to move along with the operator. The offered level of assistance can be scaled, which is vital to give the operator the opportunity to gradually learn how to interact with the system. Experiments revealed that, while the robot is programmed to adapt to the human motion, the operator also adapts to the offered assistance. When using the robot assistant the required forces to move the load are greatly reduced and the operators report that the assistance feels comfortable and natural.

This paper shows the application of a systematic approach for constraint-based task specification for sensorbased robot systems [1] to a laser tracing example. This approach integrates both task specification and estimation of geometric uncertainty in a unified framework. The framework consists of an application independent control and estimation scheme. An automatic derivation of controller and estimator equations is achieved, based on a geometric task model that is obtained using a systematic task modeling procedure. The paper details the systematic modeling procedure for the laser tracing task and elaborates on the task specific choice of two types of task coordinates: feature coordinates, defined with respect to object and feature frames, which facilitate the task specification, and uncertainty coordinates to model geometric uncertainty. Furthermore, the control and estimation scheme for this specific task is studied. Simulation and real world experimental results are presented for the laser tracing example.

This paper presents a contribution to programming by human demonstration, in the context of compliant-motion task specification for sensor-controlled robot systems that physically interact with the environment. One wants to learn about the geometric parameters of the task and segment the total motion executed by the human into subtasks for the robot, that can each be executed with simple compliant-motion task specifications. The motion of the human demonstration tool is sensed with a 3-D camera, and the interaction with the environment is sensed with a force sensor in the human demonstration tool. Both measurements are uncertain, and do not give direct information about the geometric parameters of the contacting surfaces, or about the contact formations (CFs) encountered during the human demonstration. The paper uses a Bayesian sequential Monte Carlo method (also known as a particle filter) to do the simultaneous estimation of the CF (discrete information) and the geometric parameters (continuous information). The simultaneous CF segmentation and the geometric parameter estimation are helped by the availability of a contact state graph of all possible CFs. The presented approach applies to all compliant-motion tasks involving polyhedral objects with a known geometry, where the uncertain geometric parameters are the poses of the objects. This work improves the state of the art by scaling the contact estimation to all possible contacts, by presenting a prediction step based on the topological information of a contact state graph, and by presenting efficient algorithms that allow the estimation to operate in real time. In real-world experiments, it is shown that the approach is able to discriminate in real time between some 250 different CFs in the graph

This paper describes the construction of a local polyhedral 3-D feature model derived from pose and wrench sensor measurements collected during a force-controlled execution of a sequence of polyhedral contact formations. The procedure consists of two steps: 1) the sensor measurements are filtered, resulting in a (nonminimal) representation of a 3-D feature model; and 2) identification of a reduced set of geometric parameters (vertices and faces) of the rigid polyhedral objects in the environment is performed. The following improvements with respect to the state of the art are made: 1) creation of a polyhedral 3-D feature model of a previously unknown polyhedral environment; 2) estimation of a force decomposition useful for feedback in a force controller or for monitoring the contact forces; and 3) reduction of the number of model parameters, leading to a computational reduction, a better precision of the geometric parameters, and a higher level description of the contact topology

This paper presents a unified task specification formalism and a unified control scheme for the lowest control level of sensor-based robot tasks. The formalism is based on: (i) the integration of any sensor that provides (direct or indirect) distance (and time derivatives) and force information; (ii) the possibility to use multiple "Tool Centre Points", e.g. defined relative to the robot end effector, other links or the environment; (iii) the integration of optimization functions for underconstrained as well as overconstrained specifications with linear constraints; (iv) the integration of on-line estimators; and (v) compatibility with all major low level control approaches. The unified formalism applies to the whole range from industrial manipulators over cooperating robots to humanoid robots, and from pure position control tasks over industrial processes to interaction between a humanoid robot and its environment.

This paper presents work in the field of hard realtime robotics and machine control. We analyse the requirements of a hybrid realtime control task specification allowing the integration of discrete and continuous control tasks. We propose an application independent task structure providing data flow consistency under simulataneous access by different control layers. We provide an execution flow mechanism to guarantee execution time determinism, yet allowing flexibility to react to a changing environment. We use state machines for process monitoring and a thread-safe realtime event system to communicate changes. The tasks can be distributed over a network and communicate using interfaces or manipulate streams of data in the loop. The presented task structure is applied to a real world example.

This paper presents the compliant task generator: a new approach for the automatic conversion of a geometric path generated by a compliant path planner to a force based task specification for a compliant robot controller. Based on the geometric model of a moving object and its environment, a compliant path planner generates a set of six-dimensional positions x/sub 1...m/ and their corresponding contact formations CF/sub 1...n/. The compliant force controller, which executes a planned path under force feedback using the hybrid control paradigm, expects a desired force w/sub d/, velocity t/sub d/ and position x/sub d/ at each time-step, together with their force and velocity controlled subspaces W and T. To specify these controller primitives, we add information about the desired dynamic interaction between the moving object and its environment, in the form of the desired kinetic energy E/sub kin/ of the moving object and the potential energy E/sub pot/ in the contacts with the environment, together with the inertia and stiffness matrix M and S. We fully automated the conversion process of the compliant planner output together with the added information about the dynamic interaction, to a force based task specification. This eliminates the requirement of human intervention between the planning and execution phase. The presented approach applies to all compliant motions between polyhedral objects, and is verified in a real world experiment.

This paper presents a modular demonstration tool for robot programming by human demonstration and an approach for the calibration of the tool's sensors. The tool is equipped with a wrench sensor, twelve LED markers for fast and accurate six dimensional position tracking with the Krypton K600 camera system, a compact camera and a laser distance sensor. A gripper mechanism is mounted on the tool for grasping and manipulating objects. The design of the tool specifically focused on the demonstration of compliant motion task, with applications in manipulation and assembly tasks. The calibration approach first uses an extended Kalman Filter to convert the measured positions of three to twelve visible LED's into the pose of the tool frame relative to the Krypton camera frame. Then, using a non minimal state Kalman filter, the force sensor calibration parameters are calculated, and the orientation of the Krypton camera frame relative to the world frame is defined. This calibration approach is verified in a real world experiment.

This work presents new experimental results for the estimation of large position and orientation inaccuracies of contacting objects during force-controlled compliant motion. The estimation is based on position, velocity, and force measurements. The authors have described the contact modeling and presented some Kalman filter identification results for small inaccuracies. However, when dealing with larger inaccuracies, the nonlinear estimation problem remained unsolved. This problem has now been solved satisfactorily by applying a new Bayesian estimator. The Bayesian filter is valid for static systems (parameter estimation) with any kind of nonlinear measurement equation, subject to Gaussian measurement uncertainty and for a limited class of dynamic systems. Experimental results of this new filter are given for the estimation of the positions and orientations of contacting objects during the cube-in-corner assembly described in the first reference.

Autonomous execution of robot tasks requires the ability to deal online with uncertainties such as partially unknown environments, inaccurate models, and measurement noise. This is especially true for the execution of motions maintaining stiff contacts ("compliant motions"), as contact forces become very high even for small position errors. The autonomy during compliant motion tasks is based on i) a force controller, dealing with small misalignments and keeping the contact forces within safe limits, and ii) an estimator, which recognizes the model (e.g., the type of contact) and estimates the system state (e.g., the relative position of the contacting objects). This paper focuses on Bayesian model-based solutions to the model recognition problem. We discuss Bayesian hypothesis testing and practical approximations. Experimental results are provided for two autonomous-compliant motion tasks by applying consistency testing and likelihood ratio testing. The system state is estimated simultaneously with the model recognition. This estimation is performed by the Iterated Extended Kalman filter for (approximate) linear problems and by the nonminimal state Kalman filter for nonlinear problems.

This paper presents a new filter developed for exact nonlinear Bayesian state estimation and model recognition. The filter is able to deal (i) with static systems with any kind of nonlinear measurement function subject to additive Gaussian measurement uncertainty, and (ii) with a limited class of dynamic systems. Experimental position estimation and shape recognition results during a contouring task show the strength of the new nonlinear filtering approach.

Compliant motion is required or desirable in many robotic tasks, especially assembly tasks. Both planning and execution of autonomous compliant motion requires the knowledge of contact states between parts in contact beforehand. Previous research has addressed automatic generation of contact states between rigid objects. However, not all such contact states can be possibly reached if a rigid object is attached to and moved by a manipulator due to the manipulator constraints. In this paper, we study the problem of finding feasible contact states between a polyhedral part A held by a manipulator with a fixed base and a fixed polyhedral part B. Given a contact state graph between the unattached part A and the fixed part B, our approach then attaches A to the manipulator model and checks the reachability of each contact state and the connection between two neighboring contact states by applying a virtual compliant controller to the manipulator to test possible compliant motions of A. Implementation results validate the effectiveness of our method.

This paper describes the current state, and the ongoing developments, of the hard real-time motion control core, of the OROCOS project (Open Robot Control Software). This core is a component-based, distributed and configurable software framework. The presentation discusses both the design and the current implementation. The design separates the structure of the control system from its functionality. Its hard real-time core provides a generic control structure, with plug-in facilities for customization.

This paper describes the optimization of a compliant motion task plan which guarantees the estimation of the inaccurately known position and orientation of the contacting objects to the required accuracy ("active sensing"). The task plan consists of: (1) the desired sequence of contact formations; and (2) the compliant path to execute during each of these contact formations. The plan is optimized to minimize the expected execution time, an important criterion in industrial applications. The theory is implemented for contacts between polyhedral objects.

This paper describes the contact formation modeling for the identification of geometrical parameters (positions, orientations, and dimensions) of rigid polyhedral objects during the force-controlled execution of contact formation sequences. The following improvements with respect to the state of the art are made: 1) the modeling effort is reduced considerably; 2) the generation of the measurement equations for the online estimators can be automated more easily; 3) the propagation of the geometrical parameter estimates over sequences of contact formations becomes straightforward; and 4) the measurement equations are valid for large uncertainties on the geometrical parameter estimates.

In contrast to most hybrid vision/force research, this work uses eye-in-hand vision and force control. Mounting both sensors on the same end effector gives rise to new constraints, control issues and advantages, which are discussed in the paper. Four meaningful tool/camera configurations, being parallel or non-parallel endpoint closed-loop and fixed or variable endpoint open-loop are suggested. Several task examples (in 3D space), specified in the task frame formalism, illustrate the use of these four configurations. Experimental results for the fixed EOL configuration are presented.

Hybrid position/force control can be extended to incorporate visual servoing. Together with the task frame formalism (TFF), it forms the basis to easily model, implement and execute robotic tasks in an uncalibrated workspace, using traded, hybrid or shared vision/force control. This paper emphasizes shared control in which both vision and force simultaneously control a given task frame direction. Our work shows the usefulness and feasibility of a range of tasks using shared control. Moreover, it offers a framework based on the TFF to distinguish between different (and partly new) forms of shared control. Each basic form is illustrated by a robotic task with shared control in one direction. In addition, an extension to classify multi-dimensional shared control tasks is presented. As shown by the experimental results, the tasks at hand benefit from the shared control approach.

Robotics systems and software are becoming more and more complex: the need of standard specifications is certainly a key issue in the near future. There is a need to define, from the software point of view, a generic robot. The Orocos project was started to address this problem. It aims at developing a set of robotics software in particular domains and, as a first step, will define a software framework to do that. This paper presents the future evolutions of G/sup en/oM (component generator) which we propose as a programming framework in the context of the project. G/sup en/oM proposes the definition of generic components that are used to implement robotics functionalities (vision, control, motion planning, ...), and the paper presents the definition of those components. They have been designed so that they can be connected together and externally controlled to form a modular functional layer on robots. We have defined three main entities of which components are made: a set of algorithms, an execution engine, and a communication library. The paper explains their role and presents a formal description language that let us achieves a real decoupling between code of the components (the algorithms) and the execution engine.

This paper presents a multisine approach for trajectory optimization based on information gain, with distance and orientation sensing to known beacons. It addresses the problem of active sensing, i.e. the selection of a robot motion or sequence of motions, which make the robot arrive in its desired goal configuration (position and orientation) with maximum accuracy, given the available sensor information. The optimal trajectory is parameterized as a linear combination of sine functions. An appropriate optimality criterion is selected which takes into account various requirements (e.g. maximum accuracy and minimum time). Several constraints can be formulated, e.g. with respect to collision avoidance. The optimal trajectory is then determined by numerical optimization techniques. The approach is applicable to both nonholonomic and holonomic robots. Its effectiveness is illustrated here for a nonholonomic wheeled mobile robot (WMR) in an environment with and without obstacles.

This paper introduces the OROCOS project that aims at becoming a general-purpose and open robot control software package. OROCOS follows the open source development model that has been proven to work in many other general-purpose software packages, such as Linux, Apache, Perl or LATEX. The paper focuses on the long-term vision of this start-up project, motivates which strategic and innovative design decisions are to be taken (a CORBA-like component architecture being the most important one), and lists other projects on which OROCOS could build. The success of OROCOS depends critically on how many researchers and engineers can be motivated to contribute code, documentation and feedback to the project.

This article shows that time derivatives of twists and wrenches are indeed screws, in contrast to many classical kinematicians' believe. Furthermore, it is proven that the "centripetal screw" as well as the momentum of a rigid body together with all its derivatives, are also screws, and that a rigid body's dynamics can be geometrically expressed as a screw equation. The paper relies on a somewhat more formal treatment of the screw theory than usual, in order to clarify these "controversial" issues concerning the motion of rigid systems, and in order to make the link with the more general (and historically much richer) field of differential geometry.

This paper uses Gauss' principle of least constraint to derive the "natural" dynamic equations for redundant manipulators. This approach is the fastest way to the result that the operational space inertia matrix of the manipulator is the natural weighting matrix for the projection used in solving the redundancy problem. Force-controlled robots form a special case of redundant robots, such that the results can be applied straightforwardly to solve the long-standing problem of the "non-invariance" of the selection matrices in the hybrid force/position control paradigm.

Shows that in the use of Lie groups for the study of the relative motion of rigid bodies some assumptions are not explicitly stated. A commutation diagram is shown which points out the "reference problem" and its simplification to the usual Lie group approach under certain conditions which are made explicit.

This paper gives an overview of the different levels of sensor processing complexity found in force controlled tasks, and explains which techniques from Bayesian probability theory are appropriate to cope with uncertainties and missing information at each of the different levels. The paper reduces all approaches for "intelligent" compliant, motion sensor processing to a basis set of just four classes. Some of these algorithms have already been tested experimentally, while others are still beyond the current state-of-the-art. The major contribution of this paper is to bring a clear structure to the field, which should eventually result in an easier integration of different research results, and a more precise discussion about their relative merits and innovations.

This paper uses the differential-geometric concepts of tangent and co-tangent space to describe the well-known duality between twists of, and wrenches acting on, a rigid body. This duality is the basis for formal and mathematically rigorous definitions of the duality for serial kinematic chains, parallel kinematic chains, and finally the duality between a serial and a parallel chain.

This paper presents dualities between the "elbow manipulator" architecture for serial robot arms, and the parallel "321" architecture. These dualities go beyond the well-known twist-wrench dualities, and include topological, singularity and assembly mode dualities.

The paper above by Yang and Geng (1998) presents a class of parallel manipulators that has an efficient closed form solution to its forward kinematics. This Correspondence points to some missing insights in that paper (and in many others), especially with respect to singular configurations.

We develop a geometric framework for the stability analysis of multifingered grasps and propose a measure of grasp stability for arbitrary perturbations and loading conditions. The measure requires a choice of metric on the group of rigid body displacements. We show that although the stability of a grasp itself does not depend on the choice of metric, comparison of the stability of different grasps depends on the metric. Finally, we provide some insight into the choice of metrics for stability analysis.

This paper describes the modelling and motion specification of compliant motion tasks with two or three contact points. These tasks cannot be done with Mason's (1981) classical "taskframe" (TF) or "compliance frame" approach. Hence, a more flexible and versatile motion constraint model is introduced, that maintains most of the intuitiveness of the TF approach.

The choice of "where to look next" is a special case of an optimal experiment design. The paper proposes the tolerance-optimal experiment design, which is a special instance of the well-known L-optimal design, that minimises the weighted trace of the covariance matrix of the estimated state under Gaussian assumptions. The weighting matrix is chosen such that the design is invariant to transformations with nonsingular Jacobians, and such that the emerging sensing sequence reflects the information needs of the task. This tolerance-optimal design does not require more calculations than existing optimal experiment designs. Existing optimal experiment designs do not reflect the information needs of the task. In addition, some of them physically do not make sense if the estimated state has inconsistent units. The tolerance-optimal design is illustrated with two examples.

This paper derives closed-form equations to solve the forward position kinematics of fully parallel manipulators with a 3-1-1-1 architecture on both the (planar) base and top platforms; the tetrahedrons emerging from the base and top platforms must have one leg in common. The closed-form solutions allow real-time execution on standard computer hardware: since only a few linear and quadratic equations have to be solved, all eight configurations can be found within milliseconds.

This paper presents analytical solutions for the forward displacement and velocity kinematics of the "321-HEXA", a fully-parallel manipulator design with R-RR-RRR legs as in the "HEXA" design and with a "3-2-1" top platform. The forward position and velocity kinematics rely on simple geometric algorithms that find the position and velocity of the top of a tetrahedron from the knowledge of the tetrahedron's side lengths as well as the position and velocity of the base triangle points. The forward position kinematics give eight solutions, requiring linear and quadratic equations only. The inverse kinematics requires the solution of one quartic equation for each leg.

This paper presents analytical solutions for the forward displacement kinematics of the "31-12" fully parallel manipulator design. In this design, three points in the base platform are linked to one single point in the top platform, while two of the remaining top platform points are linked to a single point in the base. Neither the base nor the top platforms of the manipulator have to be planar. The forward position kinematics give eight solutions, requiring nothing more difficult than the solution of quadratic equations.

This paper finds an analytical expression for the Jacobian matrix of a parallel manipulator as well as for its derivatives (with respect to time or to a change in joint angles) of any order. The most important part in the presented procedures is the closed-form expression for the velocity closure equations of the manipulator.

This paper extends previous work on (suboptimal) Kalman filter based identification of geometric uncertainties that occur during the manipulation of an object in contact with its environment. The setup consists of a force-controlled robot which has grasped a cylindrical peg; the relative position and orientation of the peg with respect to the robot end effector are only approximately known. The peg is moved over a surface, whose position and orientation are also uncertain. This paper describes how to estimate the above-mentioned uncertainties, even if some of the geometric uncertainties are instantaneously indeterminate, i.e., their influences on the force and/or velocity errors are instantaneously indistinguishable. Experimental results illustrate the theoretical background of the estimation problem.

This article presents a systematic and fully general model-based approach to identify geometrical uncertainties and to detect topological transitions in contact situations during compliant motion robot tasks. The emphasis is not on control but on modelling, online uncertainty identification and model validation (monitoring). The identification and monitoring methods presented are based on energy considerations and thus are invariant with respect to changes in the mathematical representations. Only one signal has to be monitored in order to validate the correctness of the contact model. The theoretical developments are illustrated by a real world experiment: a complete assembly of a peg into a hole. Due to the computational complexity, only off-line identification and monitoring are executed.

This paper presents a general approach for the identification of contact and grasping uncertainties, and the monitoring of contact situation changes in force controlled assembly operations. The identification problem is solved using virtual contact manipulators and Kalman filter techniques. Monitoring is solved by carrying out a statistical test on the sum of normalized and squared innovations of the Kalman filter, within a moving window, identification and monitoring are verified by experimental results. The paper explains how the error covariance matrix of the Kalman filter is interpreted to analyse the observability of the Kalman filter's states. Preliminary simulation results are presented for an ad hoc active sensing strategy to achieve complete state observability.

Autonomous robot tasks involving contacts with the environment must be performed under active force control if the geometric uncertainties in the task models are too large to cope with by means of passive compliance only. In practice, task specification of force-controlled actions is closely linked to the task frame formalism (TFF), also known as the compliance frame formalism. The TFF is a very intuitive and controller-independent approach to model a motion constraint, and to specify the desired forces and motions compatible with this constraint. However, it has never been defined clearly and unambiguously, and it cannot cope with all possible constrained motion tasks. This paper provides, for the first time, a formal definition of what makes up a TFF task specification. It gives also a synthesis of which tasks the TFF can cope with, and proposes a generic textual task specification formalism. Finally, it describes an example constrained motion task that the TFF cannot handle.

Most of the literature on the classical "peg-in-hole" problem concentrates on the insertion phase. In unstructured environments, however, the preceding phases of finding the hole and aligning the axes of peg and hole are equally important. This paper describes how to model the "peg-on-hole" contact situation, and how to specify the alignment motion for arbitrarily large alignment errors between the axes of peg and hole. The results are given for any radius of peg and hole. They are applied in a real world experiment, in which some uncertainties in the location of the hole are also resolved online by active force sensing.

The authors derive a theoretical framework, based on the feature frame formalism, in which the tracking equations for any general motion constraint configuration are derived in a systematic way. The resulting equations are compared to previous research, which applied intuitively derived tracking equations to simple constrained motion tasks. Their applicability to complex tasks is shown. The authors describe the elements of a kinestatic model for ideal motion constraints, and introduce one possible representation: The Jacobian, or virtual manipulator representation. The modeling of uncertainties and their influence on the Jacobian representation are discussed. Observation and identification are considered. The enhanced observation and identification potential of the feature frame formalism with respect to the task frame formalism is discussed.>