To mitigate unforeseen operational interruptions caused by potential malfunctions in robotic systems employed in industrial automation, we propose an innovative strategy for anomaly detection that incorporates a Transformer-based reconstruction network for identifying irregularities in skill-oriented manufacturing. Leveraging a semantic representation of processes, products, and resources, a semantic manufacturing execution system synthesizes an appropriate robot program and carries out the process. Our technique utilizes these descriptions to partition and automatically assign pertinent process data, facilitating the automated configuration of the anomaly detection pipeline. To overcome limited data availability, we employ a sliding window technique for data augmentation and capitalize on the attention mechanism of the Transformer to effectively extract semantic interdependencies from the time series data. By examining the discrepancies between the reconstructed time series data and the original, we can detect anomalies related to the manufacturing process. Through experiments conducted on an actual robot workcell, we demonstrate that our approach surpasses alternative competitive concepts.

This work presents a novel method for point registration in 3D space. The proposed algorithm utilizes transformation-invariant geometry information to estimate the pose of objects based on correspondences between points in two sets. Conventional methods use geometry descriptors to find these correspondences, which can result in a large number of outliers. Most existing algorithms are error-prone when outliers are present. Instead of formulating point registration as a non-convex optimization problem, we propose an intuitive method that filters out spurious correspondences. This is achieved by evaluating three different geometry-based transformation-invariant descriptors for outlier removal. We construct fully connected graphs with the proposed descriptors on correspondences, and convert the outlier removal problem into a subgraph isomorphism problem that is solved using a binary clustering approach. The resulting inlier clustering is used to estimate the transformation between the two point sets. The effectiveness of the proposed approach is evaluated on standard 3D data and the 3DMatch scan matching dataset, and compared against existing state-of-the-art methods. Results show that our method effectively reduces outliers and performs similarly to these methods.

Modern vehicles include an increasing amount of software, e.g., for autonomous driving capabilities, connectivity, and personalized user experience. The capabilities in current vehicles are still mostly provided by multiple separated embedded systems, while the current trend goes toward purely software-defined vehicles (SDV). Traditional distributed electrical/electronic (E/E) architectures have tightly coupled hardware/software, and the computational power is optimized for the included feature set. For SDVs, a centralized E/E architecture utilizing high-performance computers has been proposed. In contrast to individual embedded systems with limited and fixed functionality, combing a large set of individual software components in a single system leads to a high complexity in the proper allocation of resources. Model-based system engineering (MBSE) has been promoted in the automotive industry to handle complex system design. However, existing MBSE approaches focus mainly on traditional E/E architectures. In this work, we propose an automated and model-based approach that can address the resource allocation problem in SDVs. Users can formally describe the vehicle’s resources, safety/non-safety requirements, and optimization objectives based on existing software engineering standards. The proposed method is not restricted to specific system models, requirements, or optimization goals and is, therefore, compatible with other E/E architectures. By introducing a model-independent transformation from the model information to solver-independent optimization formulas, the resource allocation problem can be solved automatically by a wide range of state-of-the-art solvers. We demonstrate the applicability of this approach in a SDV scenario with a high-performance computer and multiple applications.

Software-defined vehicles (SDV) play an important role in future electrical and electronic (E&E) architectures. Their increased flexibility compared to traditional architectures is a crucial factor in the rapid development cycles of autonomous driving. Containerization and virtualization are two key technologies that enable rapid software installation and updates under the SDV framework. These two technologies have been widely adopted in cloud computing, but their performance and suitability in intelligent vehicles still has to be evaluated. In this work, we look at generic performance experiments of containerization and virtualization on both embedded and general-purpose computer systems regarding CPU, memory, network, and disk. We further investigate the impact of virtualization and containerization on the Autoware framework to evaluate scenarios that are close to real-world automotive applications. Additionally, we evaluate performance by splitting the Autoware framework into several dependent service parts, which are installed in separate containers. Extensive experimental results show that virtualization and containerization have no significant performance drop with 0-5% loss compared to a bare-metal setup in terms of CPU, memory, and network. However, both technologies suffer dramatic performance degradation on the disk side, losing 5-15% in containers and 35% in virtualization.

Rotation invariance is a crucial property for 3D object classification, which is still a challenging task. State-of-the-art deep learning-based works require a massive amount of data augmentation to tackle this problem. This is however inefficient and classification accuracy suffers a sharp drop in experiments with arbitrary rotations. We introduce a new descriptor that can globally and locally capture the surface geometry properties and is based on a combination of spherical harmonics energy and point feature representation. The proposed descriptor is proven to fulfill the rotation-invariant property. A limited bandwidth spherical harmonics energy descriptor globally describes a 3D shape and its rotation-invariant property is proven by utilizing the properties of a Wigner D-matrix, while the point feature representation captures the local features with a KNN to build the connection to its neighborhood. We propose a new network structure by extending PointNet++ with several adaptations that can hierarchically and efficiently exploit local rotation-invariant features. Extensive experimental results show that our proposed method dramatically outperforms most state-of-the-art approaches on standard rotation-augmented 3D object classification benchmarks as well as in robustness experiments on point perturbation, point density, and partial point clouds.

This paper presents two different approaches to generate a time local-optimal and jerk-limited trajectory with blends for a robot manipulator under consideration of kinematic constraints. The first approach generates a trajectory with blends based on the trapezoidal acceleration model by formulating the problem as a nonlinear constraint and a non-convex optimization problem. The resultant trajectory is locally optimal and approximates straight-line movement while satisfying the robot manipulator’s constraints. We apply the bridged optimization strategy to reduce the computational complexity, which borrows an idea from model predictive control by dividing all waypoints into consecutive batches with an overlap of multiple waypoints. We successively optimize each batch. The second approach is a combination of a trapezoidal acceleration model with a 7-degree polynomial to form a path with blends. It can be efficiently computed given the specified blending parameters. The same approach is extended to Cartesian space. Furthermore, a quaternion interpolation with a high degree polynomial under consideration of angular kinematics is introduced. Multiple practical scenarios and trajectories are tested and evaluated against other state-of-the-art approaches.

Inferring a complete 3D geometry given an in-complete point cloud is essential in many vision and robotics applications. Previous work mainly relies on a global feature extracted by a Multi-layer Perceptron (MLP) for predicting the shape geometry. This suffers from a loss of structural details, as its point generator fails to capture the detailed topology and structure of point clouds using only the global features. The irregular nature of point clouds makes this task more challenging. This paper presents a novel method for shape completion to address this problem. The Transformer structure is currently a standard approach for natural language processing tasks and its inherent nature of permutation invariance makes it well suited for learning point clouds. Furthermore, the Transformer’s attention mechanism can effectively capture the local context within a point cloud and efficiently exploit its incomplete local structure details. A morphing-atlas-based point generation network further fully utilizes the extracted point Transformer feature to predict the missing region using charts defined on the shape. Shape completion is achieved via the concatenation of all predicting charts on the surface. Extensive experiments on the Completion3D and KITTI data sets demonstrate that the proposed PCTMA-Net outperforms the state-of-the-art shape completion approaches and has a 10% relative improvement over the next best-performing method.

Digital transformation efforts in manufacturing companies bear the potential to reduce product costs and increase the flexibility of production systems. The semantic integration of data and information along the value chain enables the automated interpretation of interrelations between its different aspects such as product design, production process and manufacturing resources. These interrelations can be used to automatically generate semantic process descriptions and execute corresponding robot motions. An initial one-time effort to model the required knowledge of a particular application domain can make the manufacturing of high-variant products in small batches or even lot size one production more efficient.This paper introduces a knowledge-based digital engineering concept to automate engineering and production activities without human involvement. The concept was integrated and evaluated in a physical robot workcell where automotive fuse boxes are autonomously fitted with different fuse configurations.

We propose a surface-to-surface (S2S) point registration algorithm by exploiting the Gaussian Process Implicit Surfaces for partially overlapping 3D surfaces to estimate the 6D pose transformation. Unlike traditional approaches, that separate the corresponding search and update steps in the inner loop, we formulate the point registration as a nonlinear non-constraints optimization problem which does not explicitly use any corresponding points between two point sets. According to the implicit function theorem, we form one point set as a Gaussian Process Implicit Surfaces utilizing the signed distance function, which implicitly creates three manifolds. Points on the same manifold share the same function value, indicated as {1, 0, -1}. The problem is thus converted into finding a rigid transformation that minimizes the inherent function value. This can be solved by using a Gauss-Newton (GN) or Levenberg-Marquardt (LM) solver. In the case of a partially overlapping 3D surface, the Fast Point Feature Histogram (FPFH) algorithm is applied to both point sets and a Principal Component Analysis (PCA) is performed on the result. Based on this, the initial transformation can then be computed. We conduct experiments on multiple point sets to evaluate the effectiveness of our proposed approach against existing state-of-the-art methods.

The effectiveness of cognitive manufacturing systems in agile production environments heavily depend on the automatic assessment of various levels of interoperability between manufacturing resources. For taking informed decisions, a semantically rich representation of all resources in a workcell or production line is required. OPC UA provides means for communication and information exchange in such distributed settings.This paper proposes a semantic representation of a resource's properties, in which we use OWL ontologies to encode the information models that can be found in OPC UA NodeSet specifications. We further combine these models with an OWL-based description of the resource's geometry and - if applicable - its kinematic model. This leads to a comprehensive semantic representation of hardware and software features of a manufacturing resource, which we call semantic digital twin. Among other things, it reduces costs through virtual prototyping and enables the automatic deployment of manufacturing tasks in production lines. As a result, small-batch assemblies become financially viable.In order to minimize the effort of creating OWL-based UA NodeSet descriptions, we provide a software tool for the automatic transformation of XML-based NodeSet specifications that adhere to the OPC Foundation's NodeSet2 XML schema.

The current trend to lot-size-one production requires reduced integration effort and easy reuse of available devices inside the production line. These devices have to offer a uniform interface to fulfill these requirements.This paper presents a hardware-agnostic skill model using the semantic modeling capabilities of OPC UA. The model provides a standardized interface to hardware or software functionality while offering an intuitive way of grouping multiple skills to a higher hierarchical abstraction.Our skill model is based on OPC UA Programs and modeled as an open source NodeSet. We hereby focus on the reusability of the skills for many different domains. The model is evaluated by controlling three different industrial robots and their tools through the same skill interface. The evaluation shows that our generic OPC UA skill model can be used as a standardized control interface for device and software components in industrial manufacturing. With our solution new components can easily be exchanged without changing the interface. This is not only true for industrial robots, but for any device which provides a controllable functionality.

In this paper, we enhance our knowledge-based and constraint-based approach of robot programming with the concept of Semantic Mates. They describe intended mechanical connections between parts of an assembly. This allows deriving appropriate assembly poses from the type of connection and the geometric properties of the involved parts. The paper presents an ontology-based representation of Semantic Mates that is used to augment object models with additional information regarding their potential use in an assembly. Such semantically annotated object models can be used in our instruction framework to program a robot to perform assembly tasks through simple drag-and-drop operations in a graphical user interface. We conducted a user study with 21 participants in order to evaluate the efficiency and usability of the Semantic Mates concept based on a use-case from the domain of mechanical assembly. Across different experience levels in robotics, the participants achieved a significantly faster workflow and improved perceived usability compared to the manual specification of constraint-based assembly operations.

We propose a methodology for designing dependable Artificial Neural Networks (ANNs) by extending the concepts of understandability, correctness, and validity that are crucial ingredients in existing certification standards. We apply the concept in a concrete case study for designing a highway ANN-based motion predictor to guarantee safety properties such as impossibility for the ego vehicle to suggest moving to the right lane if there exists another vehicle on its right.

This paper presents an approach to generate the time-optimal trajectory for a robot manipulator under certain kinematic constraints such as joint position, velocity, acceleration, and jerk limits. This problem of generating a trajectory that takes the minimum time to pass through specified waypoints is formulated as a nonlinear constraint optimization problem. Unlike prior approaches that model the motion of consecutive waypoints as a Cubic Spline, we model this motion with a seven-segment acceleration profile, as this trajectory results in a shorter overall motion time while staying within the bounds of the robot manipulator's constraints. The optimization bottleneck lies in the complexity that increases exponentially with the number of waypoints. To make the optimization scale well with the number of waypoints, we propose an approach that has linear complexity. This approach first divides all waypoints to consecutive batches, each with an overlap of two waypoints. The overlapping waypoints then act as a bridge to concatenate the optimization results of two consecutive batches. The whole trajectory is effectively optimized by successively optimizing every batch. We conduct experiments on practical scenarios and trajectories generated by motion planners to evaluate the effectiveness of our proposed approach over existing state-of-the-art approaches.

We discuss the architecture and software engineering principles of the Robotics Library (RL). Driven by requirements of robot systems, research projects, industrial applications, and education, we identify relevant design requirements and present an approach to manage hardware and real-time, provide a user-friendly, object-oriented interface to powerful kinematics and dynamics calculations, and support various platforms. After over ten years of development that started in 2004 and evaluating many variants of the architecture, we discuss the design choices for the components of the library in its current version.

Reliability and safety are extremely important for autonomous driving in real traffic scenarios. However, due to imperfect control and sensing, the actual state of the vehicle cannot be flawlessly predicted or measured, but estimated with uncertainty. Therefor, it is important to consider the execution risk advance in motion planning for a solution with a high success rate. The Space Exploration Guided Heuristic Search (SEHS) method is extended to deal with perception and control uncertainty in its two planning stages. First, the localization uncertainty is evaluated with a simple probabilistic robot model by the Space Exploration to find a path corridor with sufficient localization quality for the desired motion accuracy. Then, a trajectory controller is modeled with nonholonomic kinematics for the belief propagation of a robot state with primitive motions. The dynamic model and the control feedback are approximated in a close neighborhood of the reference trajectory. In this case, the Heuristic Search can propagate the state uncertainty as a normal distribution in the search tree to guarantee a high probability of safety and to achieve the required final accuracy. The belief-based SEHS is evaluated in several simulated scenarios. Compared to the basic SEHS method that assumes perfection, motions with higher execution successful rate are produced, especially the human-like behaviors for driving through narrow passages and precise parking. This confirms the major contribution of this work in exploiting the uncertainties for motion planning in autonomous driving.

When creating modern and visually appealing user experiences for the interaction with industrial robots, previously known and universally applicable paradigms in app and web design can be utilized to increase accessibility and usability of the to be created service. This is especially the case when the expected user group consists of untrained and inexperienced users and therefore system interaction focus is laid more on build progress overview, safety for human and robot, as well as overall simplification of complicated features. In this paper, we present four of the most important paradigms of modern graphical user experiences in web and app design that can be used to forward the concept of interacting with an industrial robot without any experience-related thresholds. By redesigning an existing interaction concept of a working robot cell system for assembly tasks in a small and medium-sized enterprise environment the presented paradigms are being utilized. The achieved improvements are then examined in a before-after user study to analyze the paradigm's success in suiting the user's expectation and anticipation using the redesigned service.

In this paper, we introduce a novel robot programming paradigm. It focuses on reducing the required expertise in robotics to a level that allows shop floor workers to use robots in their application domain without the need of extensive training. Our approach is user-centric and can interpret underspecified robot tasks, enabling communication on an abstract level. Such high-level task descriptions make the system amenable for users that are experts in a particular domain, but have limited knowledge about robotics and are thus not able to specify low-level details and instructions. Semantic models for all involved entities, i.e., processes, workpieces, and workcells, enable automatic reasoning about underspecified tasks and missing pieces of information. We showcase and evaluate this methodology on two industrial use cases from the domains of assembly and woodworking, comparing it to state-of-the-art solutions provided by robot manufacturers.

In this paper, we propose a framework for prioritized constraint-based specification of robot tasks. This framework is integrated with a cognitive robotic system based on semantic models of processes, objects, and workcells. The target is to enable intuitive (re-)programming of robot tasks, in a way that is suitable for non-expert users typically found in SMEs. Using CAD semantics, robot tasks are specified as geometric inter-relational constraints. During execution, these are combined with constraints from the environment and the workcell, and solved in real-time. Our constraint model and solving approach supports a variety of constraint functions that can be non-linear and also include bounds in the form of inequalities, e.g., geometric inter-relations, distance, collision avoidance and posture constraints. It is a hierarchical approach where priority levels can be specified for the constraints, and the nullspace of higher priority constraints is exploited to optimize the lower priority constraints. The presented approach has been applied to several typical industrial robotic use-cases to highlight its advantages compared to other state-of-the-art approaches.

A hybrid planning approach is developed for intersection assistance systems up to fully automated driving through intersections. Route planning, task planning and motion planning methods are integrated in a hierarchical planning framework to deal with the various information and constraints in different layers. The navigation agent provides a global driving direction at an intersection according to the selected route. The task planner decides a sequence of actions to accomplish the driving mission taking into consideration traffic rules and semantic conditions. The motion planner generates detailed trajectories to execute the tasks. Meanwhile, the task sequence and the motion trajectory are verified periodically against the actual traffic situation, and re-planning is triggered when necessary in the motion planning or task planning level. The hierarchical planning framework is evaluated in several intersection scenarios. The result shows that it can handle the complex planning problems with dynamic objects and provide a modular solution for automated driving that can be easily extended for different traffic rules and applications.

We present an approach for monitoring and interpreting human activities based on a novel multimodal vision-based interface, aiming at improving the efficiency of human-robot interaction (HRI) in industrial environments. Multi-modality is an important concept in this design, where we combine inputs from several state-of-the-art sensors to provide a variety of information, e.g. skeleton and fingertip poses. Based on typical industrial workflows, we derived multiple levels of human activity labels, including large-scale activities (e.g. assembly) and simpler sub-activities (e.g. hand gestures), creating a duration- and complexity-based hierarchy. We train supervised generative classifiers for each activity level and combine the output of this stage with a trained Hierarchical Hidden Markov Model (HHMM), which models not only the temporal aspects between the activities on the same level, but also the hierarchical relationships between the levels.

The Space Exploration Guided Heuristic Search (SEHS) method solves the motion planning problem, especially for car-like robots, in two steps: a circle-based space exploration in the workspace followed by a circle-guided heuristic search in the configuration space. This paper extends this approach for kinodynamic planning in dynamic environments by performing the exploration in both space and time domains. Thus, a time-dependent heuristic is constructed to guide the search algorithm applying a kinodynamic vehicle model. Furthermore, the search step-size and state resolution are adapted incrementally to guarantee resolution completeness with a trade-off for efficiency. The performance of Space-Time Exploration Guided Heuristic Search (STEHS) approach is verified in two scenarios and compared with several search-based and sampling-based methods.

In order to fully exploit the capabilities of a robotic systems, it is necessary to consider the limitations and errors of actuators and sensors already during the motion planning phase. In this paper, a framework for path planning is introduced, that uses heuristic search to build up a search graph in belief space, an extension to the deterministic state space considering the uncertainty associated with this space. As sources of uncertainty actuator errors and map uncertainties are considered. We apply this framework to various scenarios for a non-holonomic vehicle and compare the resulting paths to heuristic state space planners and LQG-MP[1] with the help of simulations. As a result, paths generated with this framework could either not be found with worst-case assumptions or have a higher probability of being successfully executed compared to planners with more relaxed constraints.

In this paper, we introduce an approach for leveraging CAD description to a semantic level, in order to link additional knowledge to CAD models and to exploit resulting synergy effects. This has been achieved by designing a description language, based on the Web Ontology Language (OWL), that is used to define boundary representations (BREP) of objects. This involves representing geometric entities in a semantic meaningful way, e.g., a circle is defined by a coordinate frame and a radius instead of a set of polygons. Furthermore, the scope of this semantic description language also covers geometric constraints between multiple objects. Constraints can be specified not only on the object level, but down to single edges or faces of an object. This semantic representation is used to improve a variety of applications, ranging from shape-based object recognition to constraint-based robot task descriptions. Results from a quantitative evaluation are presented to assess the practicability of this approach.

Intuitive programming of industrial robots is especially important for small and medium-sized enterprises. We evaluated four different input modalities (touch, gesture, speech, 3D tracking device) regarding their preference, usability, and intuitiveness for robot programming.

In this paper, we propose a framework for intuitive task-based programming of robots using geometric inter-relational constraints. The intended applications of this framework are robot programming interfaces that use semantically rich task descriptions, allow intuitive (re-)programming, and are suitable for non-expert users typically found in SMEs. A key concept in this work is the use of CAD semantics to represent geometric entities in the robotic workcell. The robot tasks are then represented as a set of geometrical inter-relational constraints, which are solved in real-time to be executed on the robot. Since these constraints often specify the target pose only partially, the robot can be controlled to move in the constraints' null space in order to handle external disturbances or further optimize the robot's pose during runtime. Geometrical inter-relational constraints are easy to understand and can be intuitively specified using CAD software. A number of applications common in industrial robotic scenarios have been chosen to highlight the advantages of the presented approach vis-à-vis the state-of-the-art approaches.

A hybrid planning approach is presented in this paper with the focus of integrating task planning and motion planning for highly automated driving. In the context of task planning, the vehicle and environment states are transformed from the continuous configuration space to a discrete state space. A planning problem is solved by a search algorithm for an optimal task sequence to reach the goal conditions in the symbolic space, regarding constraints such as space topology, place occupation, and traffic rules. Each task can be mapped to a specific driving maneuver and solved with a dedicated motion planning method in the continuous configuration space. The task planning approach not only bridges the gap between high-level navigation and low-level motion planning, but also provides a modular domain description that can be developed and verified individually. Our task planner for automated driving is evaluated in several scenarios with prior knowledge about the road-map and sensing range of the vehicle. Behavior that is otherwise complex to achieve is planned according to traffic rules and re-planned regarding the on-line perception.

Due to the nonholonomic constraints of the vehicle kinematics, parking and maneuvering a car in a narrow clustered space are among the most challenging driving tasks. This paper introduces an extended version of Space Exploration Guided Heuristic Search (SEHS) method, called Orientation-Aware Space Exploration Guided Heuristic Search (OSEHS), to solve the path planning problems for parking and maneuvering. This method considers the orientation of a vehicle in the space exploration phase to achieve knowledge about driving directions. Such information is exploited later in the heuristic search phase to improve the planning efficiency in maneuvering scenarios. This approach is not bound to the specific domain knowledge about a parking or maneuvering task, but obtains the space dimension and orientation information through a generic exploration procedure. Therefore, it is convenient to integrate the maneuvering ability into a general SEHS motion planning framework. Experiments show that the OSEHS approach produces better results than common random-sampling methods and general heuristic search methods.

Today's industrial robots require expert knowledge and are not profitable for small and medium sized enterprises with their small lot sizes. It is our strong belief that more intuitive robot programming in an augmented reality robot work cell can dramatically simplify re-programming and leverage robotics technology in short production cycles. In this paper, we present a novel augmented reality system for defining virtual obstacles, specifying tool positions, and specifying robot tasks. We evaluate the system in a user study and, more specifically, investigate the input of robot end-effector orientations in general.

A real-time vehicle motion planning engine is presented in this paper, with the focus on exploiting the prior and online traffic knowledge, e.g., predefined roadmap, prior environment information, behaviour-based motion primitives, within the space exploration guided heuristic search (SEHS) framework. The SEHS algorithm plans a kinodynamic vehicle motion in two steps: a geometric investigation of the free space, followed by a grid-free heuristic search employing primitive motions. These two procedures are generic and possible to take advantage of traffic knowledge. In this paper, the space exploration is supported by a roadmap and the heuristic search benefits from the behaviour-based primitives. Based on this idea, a light weighted motion planning engine is built, with the purpose to handle the traffic knowledge and the planning time in real-time motion planning. The experiments demonstrate that this SEHS motion planning engine is flexible and scalable for practical traffic scenarios with better results than the baseline SEHS motion planner regarding the provided traffic knowledge.

We present the exploring/exploiting tree (EET) algorithm for motion planning. The EET planner deliberately trades probabilistic completeness for computational efficiency. This tradeoff enables the EET planner to outperform state-of-the-art sampling-based planners by up to three orders of magnitude. We show that these considerable speedups apply for a variety of challenging real-world motion planning problems. The performance improvements are achieved by leveraging work space information to continuously adjust the sampling behavior of the planner. When the available information captures the planning problem's inherent structure, the planner's sampler becomes increasingly exploitative. When the available information is less accurate, the planner automatically compensates by increasing local configuration space exploration. We show that active balancing of exploration and exploitation based on workspace information can be a key ingredient to enabling highly efficient motion planning in practical scenarios.

Robot task planning is an inherently challenging problem, as it covers both continuous-space geometric reasoning about robot motion and perception, as well as purely symbolic knowledge about actions and objects. This paper presents a novel “knowledge of volumes” framework for solving generic robot tasks in partially known environments. In particular, this approach (abbreviated, KVP) combines the power of symbolic, knowledge-level AI planning with the efficient computation of volumes, which serve as an intermediate representation for both robot action and perception. While we demonstrate the effectiveness of our framework in a bimanual robot bartender scenario, our approach is also more generally applicable to tasks in automation and mobile manipulation, involving arbitrary numbers of manipulators.

This paper presents an efficient motion planning method for nonholonomic vehicles, which combines space exploration and heuristic search to achieve online performance. The space exploration employs simple geometric shapes to investigate the collision-free space for the dimension and topology information. Then, the heuristic search is guided by this knowledge to generate vehicle motions under kinodynamic constraints. The overall performance of this framework greatly benefits from the cooperation of these two simple generic algorithms in suitable domains, which sequentially handles the free-space information and kinodynamic constraints. Experimental results show that this method is able to generate motions for nonholonomic vehicles in a time frame of less than 100 milliseconds for the given problem settings. The contribution of this work is the development of a Space Exploration Guided Heuristic Search with a circle-path based heuristics and adaptable search step size. The approach is grid-free and able to plan nonholonomic vehicle motions under kinodynamic constraints.

The development of increasingly complex robots in recent years has been characterized by an extensive use of physics-based simulations for controller design and optimization. Today, a variety of open-source and commercial simulators exist for this purpose for mobile and industrial robots. However, existing simulation engines still lack support for the emerging class of tendon-driven robots. In this paper, an innovative simulation framework for the simulation of tendon-driven robots is presented. It consists of a generic physics simulator capable of utilizing CAD robot models and a set of additional tools for simulation control, data acquisition and system investigation. The framework software architecture has been designed using component-based development principles to facilitate the framework extension and customization. Furthermore, for inter-component communication, the operating-system and programming language independent Common Object Request Broker Architecture (CORBA) [1] has been used which simplifies the integration of the framework into existing software environments.

Direct physical human-robot interaction has become a central part in the research field of robotics today. To use the advantages of the potential for humans and robots to work together as a team in industrial settings, the most important issues are safety for the human and an easy way to describe tasks for the robot. In this work, we present an approach of a hierarchical structured control of industrial robots for joint-action scenarios. Multiple atomic tasks including dynamic collision avoidance, operational position, and posture can be combined in an arbitrary order respecting constraints of higher priority tasks. The controller flow is based on the theory of orthogonal projection using nullspaces and constraint least-square optimization. To proof the approach, we present three collaboration scenarios between a human and an industrial robot.

In many future joint-action scenarios, humans and robots will have to interact physically in order to cooperate successfully. Ideally, human-robot interaction should not require training on the human side, but should be intuitive and simple. Previously, we reported on a simple case of physical human-robot interaction, a hand-over task. Even such a basic task as manually handing over an object from one agent to another requires that both partners agree upon certain basic prerequisites and boundary conditions. While some of them are negotiated explicitly, e.g. by verbal communication, others are determined indirectly and adaptively in the course of the cooperation. In the previous study we compared a human-human hand-over interaction with the same task performed by a human and a robot. However, the trajectories used for the robot, a conventional trapezoidal velocity profile in joint coordinates and a minimum-jerk profile of the end-effector, have little resemblance to the natural movements of humans. In this study we introduce a novel trajectory generator that is a variation of the traditional minimum-jerk profile, the `decoupled minimum-jerk' profile. Its trajectory is much closer to those observed in human-human experiments. We evaluated its performance concerning human comfort and acceptance in a simple hand-over experiment by using a post-test questionnaire. The evaluation of the questionnaire revealed no difference with respect to comfort, human-likeness, or subjective safety of the new planner compared to the minimum-jerk profile. Thus, the `decoupled minimum-jerk' planner, which offers important advantages with respect to target approach, proved to be a promising alternative to the previously used minimum-jerk profile.

Computationally efficient motion planning must avoid exhaustive exploration of configuration space. We argue that this can be accomplished most effectively by carefully balancing exploration and exploitation. Exploration seeks to understand configuration space, irrespective of the planning problem, while exploitation acts to solve the problem given the available information obtained by exploration. We present an exploring/exploiting tree (EET) planner that balances its exploration and exploitation behavior. The planner acquires workspace information and subsequently uses this information for exploitation in configuration space. If exploitation fails in difficult regions, the planner gradually shifts its behavior towards exploration. We present experimental results demonstrating that adaptive balancing of exploration and exploitation leads to significant performance improvements compared to other state-of-the-art sampling-based planners.

In many future joint-action scenarios, humans and robots will have to interact physically in order to successfully cooperate. Ideally, seamless human-robot interaction should not require training for the human, but should be intuitively simple. Nonetheless, seamless interaction and cooperation involve some degree of learning and adaptation. Here, we report on a simple case of physical human-robot interaction, a hand-over task. Even such a basic task as manually handing over an object from one agent to another requires that both partners agree upon certain basic prerequisites and boundary conditions. While some of them are negotiated explicitly, e.g. by verbal communication, others are determined indirectly and adaptively in the course of the cooperation. In the present study, we compared human-human hand-over interaction with the same task done by a robot and a human. To evaluate the importance of biological motion, the robot human interaction was tested with two different velocity profiles: a conventional trapezoidal velocity profile in joint coordinates and a minimum-jerk profile of the end-effector. Our results show a significantly shorter reaction time for minimum jerk profiles, which decreased over the first three hand-overs. The results of our comparison provide the background for implementing effective joint-action strategies in humanoid robot systems.

This paper presents a concept of a smart working environment designed to allow true joint-actions of humans and industrial robots. The proposed system perceives its environment with multiple sensor modalities and acts in it with an industrial robot manipulator to assemble capital goods together with a human worker. In combination with the reactive behavior of the robot, safe collaboration between the human and the robot is possible. Furthermore, the system anticipates human behavior, based on knowledge databases and decision processes, ensuring an effective collaboration between the human and robot. As a proof of concept, we introduce a use case where an arm is assembled and mounted on a robotpsilas body.

We describe a human-robot dialogue system that allows a human to collaborate with a robot agent on assembling construction toys. The human and the robot are fully equal peers in the interaction, rather than simply partners. Joint action is supported at all stages of the interaction: the participants agree on a construction task, jointly decide how to proceed to proceed with the task, and also implement the selected plans jointly. The symmetry provides novel challenges for a dialogue system, and also makes it possible for findings from human-human joint-action dialogues to be easily implemented and tested.