The specification and validation of robotics applications require bridging the gap between formulating requirements and systematic testing. This often involves manual and error-prone tasks that become more complex as requirements, design, and implementation evolve. To address this challenge systematically, we propose extending behaviour-driven development (BDD) to define and verify acceptance criteria for robotic systems. In this context, we use domain-specific modelling and represent composable BDD models as knowledge graphs for robust querying and manipulation, facilitating the generation of executable testing models. A domain-specific language helps to efficiently specify robotic acceptance criteria. We explore the potential for automated generation and execution of acceptance tests through a software architecture that integrates a BDD framework, Isaac Sim, and model transformations, focusing on acceptance criteria for pick-and-place applications. We tested this architecture with an existing pick-and-place implementation and evaluated the execution results, which shows how this application behaves and fails differently when tested against variations of the agent and environment. This research advances the rigorous and automated evaluation of robotic systems, contributing to their reliability and trustworthiness.

Autonomous learning of robotic manipulation tasks is a promising approach to reduce manual engineering effort and increase flexibility in the future of industrial manufacturing. Although a lot of research has been done especially robotic assembly tasks requiring contact-rich compliant interaction remain a challenge for learning-based methods, since large amounts of interaction data are required. Incorporation of prior knowledge has long been seen as a possibility to make learning-based approaches tractable. The question is how can we enable process experts to encode their prior knowledge in grey-box models so that it can be used for learning robotic manipulation tasks? For that reason we propose a new grey-box learning approach, “Adaptive Manipulation Primitives” (AMP), introduced in this paper. AMPs combine compliant manipulation task specifications based on Manipulation Primitives Nets with Policy Gradient Reinforcement Learning. Our framework is evaluated in a real-world robotic assembly task. It is shown that learning to assemble industrial connector modules is possible with comparatively few real-world trials.

Collaborative robots (cobots) are being applied in areas such as healthcare, rehabilitation, agriculture and logistics, beyond the typical manufacturing setting. This is leading to a marked increase in the number of cobot stakeholders with little or no experience in traditional safety engineering. Considering the importance of human safety in collaborative robotic applications, this is currently proving to be a barrier to more widespread cobot usage. A web-based Toolkit that targets cobot end-users and manufacturers with varying levels of safety expertise was developed, helping them to understand how to consider the safety of their cobot applications. In this work, we will provide an overview of the state of the art for ensuring cobot safety, highlight the support provided by the “COVR Toolkit” and introduce three examples where third parties applied the Toolkit for their collaborative robotics application.

Recent studies on assistance systems show that adaptive support for users is becoming increasingly important, as this is a promising approach to ensure acceptance, learning support, and appropriate assistance in the long term. Developers of assistance systems are therefore faced with the challenge of designing various dimensions of adaptivity in such a way that they are conducive to acceptance and support. In this paper, we consider the different dimensions and options for the adaptation of assembly assistance and present the results of a first survey to investigate users’ expectations regarding the adaptability of assembly assistance systems. Our evaluation shows that users have clear preferences concerning the control over adaptation and adaptation goals, while other aspects of adaptation assistance do not indicate a clear tendency. The results of this survey should serve as a decision support in the design phase of adaptive assistance systems to choose between different implementation options and concepts.

Autonomous learning of robotic manipulation tasks is a desirable proposition for the future of industrial manufacturing to increase flexibility and reduce manual engineering effort. In particular assembly tasks that require contact-rich manipulation skills are challenging to accomplish with classical robotic control methods. The Reinforcement Learning (RL) framework provides a possibility to learn complex behaviors based on interaction with the environment. Although a lot of research has been done robotic assembly tasks remain a challenge for pure learning-based systems. In this paper we give an overview on grey-box learning approaches that integrate prior knowledge and learning based methods. Different dimensions of knowledge injection are identified, and knowledge representations are described. These representations are discussed in the context of industrial assembly processes to answer the question: how can process experts model their knowledge to boost RL approaches in the context of industrial assembly?

Modern production requires flexible systems to fulfill customer desires of highly specialized products down to a batch size of one. Collaborative robots are considered as the key element for a versatile and flexible production in a state-of-the-art and modular production environment. Although modularization is a well known concept since the 80s, the composition of modular fabrication units and the flexible programming of robots are still challenging and depend strongly on the experience of the industrial engineer. For this reason, we present an assisted approach for interactive workspace setup targeting the operator to plan and commission efficiently applications containing collaborative robots in flexible production systems. As a central contribution, we introduce a planning workflow to take the human-in-the-loop for the workspace configuration of the robot and the arrangement of components in the modular production system. We evaluate our contribution quantitatively in an initial experimental setup performing the automatized layout in complex configurations to showcase its effectiveness.

In recent years the need for manipulation tasks in the industrial as well as in the service robotics domain that require compliant interaction with the environment rose. Since then, an increased number of publications use a model-driven approach to describe these tasks. High-level tasks and sequences of skills are coordinated to achieve a desired motion for e.g., screwing, polishing, or snap mounting. Even though the awareness of the environment, especially in terms of contact situations, is essential for successful task execution, it is too often neglected or considered insufficiently.In this paper, we present a model-based approach, using domain-specific languages (DSL), that enables the explicit modeling of the environment in terms of contact situations. Decoupling the environment model from the skills, fosters exchangeability and allows the adaptation to different environmental situations. This way, an explicit but non-invasive link is established to the skills, enabling the environment model to provide a context to constrain the execution of the skills. Further, we present a synthesis from the modeled contact situations to a real-time component-based control architecture, which executes the skills subject to the active environmental context. A dual arm yoga mat rolling task is used to show the impact of the environment model on the skill execution.

The paper explores the interactional dynamics within groups of visitors (children & adults) who are interacting with an autonomous museum guide robot. Using the example of question-answer sequences, analysis reveals that 'answering' tends to be a process of interactional negotiation at the end of which a 'result' is addressed to the robot. This process, however, is often not transparent to the robot and considered solely as "silence" which leads to the disregard of the users' attempts to provide an answer.

On the basis of a video corpus of human-robot-interaction in a museum guide scenario, we address the challenge how a robot could determine the appropriate moment of opening an interaction with a visitor. Considering such "openings" as a stepwise coordinated process, we used a wizard-of-oz setup to investigate the wizard's timely decisions about when to look at the visitor resp. to verbally greet him/her (during early stages called "pre-beginnings"). Analysis is based on both robot internal and external data and combines qualitative sequential case analysis and quantification. Analysis reveals features for classification of two visitors types and corresponding opening strategies.

The first workshop on embodied interaction with smart environments aims to bring together the very active community of multi-modal interaction research and the rapidly evolving field of smart home technologies. Besides addressing the software architecture of such very complex systems, it puts an emphasis on questions regarding an intuitive interaction with the environment. Thereby, especially the role of agency leads to interesting challenges in the light of user interactions. We therefore encourage a lively discussion on the design and concepts of social robots and virtual avatars as well as innovative ambient devices and their implementation into smart environments.

Even though research on autonomous robots and human-robot interaction accomplished great progress in recent years, and reusable soft- and hardware components are available, many of the reported findings are only hardly reproducible by fellow scientists. Usually, reproducibility is impeded because required information, such as the specification of software versions and their configuration, required data sets, and experiment protocols are not mentioned or referenced in most publications. In order to address these issues, we recently introduced an integrated tool chain and its underlying development process to facilitate reproducibility in robotics. In this contribution we instantiate the complete tool chain in a unique user study in order to assess its applicability and usability. To this end, we chose three different robotic systems from independent institutions and modeled them in our tool chain, including three exemplary experiments. Subsequently, we asked twelve researchers to reproduce one of the formerly unknown systems and the associated experiment. We show that all twelve scientists were able to replicate a formerly unknown robotics experiment using our tool chain.

We present a novel fault detection method for application in component-based robotic systems. In contrast to existing work, our method specifically addresses faults in the software system of the robot using a data-driven methodology which exploits the inter-process communication of the system. This enables an application of the approach without expert knowledge or availability of complex software models. We specifically focus on performance bugs, which slowly degrade the performance of the system and are thereby harder to detect but also most valuable for automatic recovery. Using a data set recorded on a RoboCup@Home platform we demonstrate the performance and applicability of our method and analyze the types of faults that can be detected by the method.

An autonomous robot system was equipped with basic means to monitor the users' success/failure in following a robot's verbal-gestural deictic reference to an object and - in case of problems - to provide additional help, i.e. to suggest a `repair' action. A real-world field trial with the robot acting as museum guide constitutes the basis for analysis of the users' reactions to the first reference and the subsequent `repair' action in two structurally different conditions. Results indicate that deictic `repair' actions are mostly successful helping users to correctly orient, but they may also lead to confusion for initially correctly oriented visitors in situations of small groups of users. A recurring user practice is revealed which consists of displaying to co-visitors their understanding of the reference by pointing to the location and thus providing additional orientational help.

This paper introduces a model-driven approach for engineering complex movement control architectures based on motion primitives, which in recent years have been a central development towards adaptive and flexible control of complex and compliant robots. We consider rich motor skills realized through the composition of motion primitives as our domain. In this domain we analyze the control architectures of representative example systems to identify common abstractions. It turns out that the introduced notion of motion primitives implemented as dynamical systems with machine learning capabilities, provide the computational building block for a large class of such control architectures. Building on the identified concepts, we introduce domain-specific languages that allow the compact specification of movement control architectures based on motion primitives and their coordination respectively. Using a proper tool chain, we show how to employ this model-driven approach in a case study for the real world example of automatic laundry grasping with the KUKA LWR-IV, where executable source-code is automatically generated from the domain-specific language specification.

On the basis of a videocorpus of human-robot-interactions in a museum guide scenario and with a combined approach of qualitative case analysis and quantitative evaluation we examine two research questions: (1) What type of group dynamics result from a robot's unexpected next move - especially focussing on gaze patterns? (2) What should a robot detect when searching for trouble events and how should it identify patterns of visitor behavior that are related to the robots actions? Systematically studying opening sequences, we extract structural gaze patterns out of two cases considered from an interactional linguistics perspective as well as keeping in mind a system's capabilities of perception.

Once a museum guide robot is deployed in the real world, it is faced with the difficulty to deal with user utterances within a noisy environment. We present explorative analysis of the ways in which visitors signal trouble on a multimodal level when they have difficulties in understanding the robot. Such moments of trouble are investigated in the context of robot-to-group situations. Analysis will give insights into the visitors conduct at moments of trouble in robot-to-single-user and robot-to-group interactions. In a limited data set, we found that members of a group show internal gaze coordination between each other when trouble occurs due to the robot's inadequate behavior. Implications will be drawn for further developing the interactional capabilities of an autonomous robot system to be attentive and responsive to the users' needs.

A basic task for robots interacting with humans consists in guiding their focus of attention. Existing guidelines for a robot's multimodal deixis are primarily focused on the speaker (talk-gesture-coordination, handshape). Conducting a field trial with a museum guide robot, we tested these individualistic referential strategies in the dynamic conditions of real-world HRI and found that their success ranges between 27% and 95%. Qualitative video-based micro-analysis revealed that the users experienced problems when they were not facing the robot at the moment of the deictic gesture. Also the importance of the robot's head orientation became evident. Implications are drawn as design guidelines for an interactional account of modeling referential strategies for HRI.

Facilitating efficient programming-by-demonstration methods for advanced robot systems is an ongoing research challenge. This paper addresses one important challenge in this area, which is the programming of kinematically redundant robots. We argue that standard programming-by-demonstration methods for teaching task-space trajectories on a redundant robot using physical human-robot interaction are too complex for non-expert human tutors. We therefore introduce a new interaction and control concept for redundant robot systems, Assisted Gravity Compensation, based on a hierarchical control scheme, separating task-space programming from the redundancy resolution. The user is actively assisted by a given redundancy resolution while kinesthetically teaching task-space trajectories. This control scheme is implemented on our experimental robot system called FlexIRob and we briefly present results of a kinesthetic teaching experiment obtained in a larger field study on physical Human-Robot Interaction with 48 industrial workers. These results show, that the Assisted Gravity Compensation reduces the complexity of a kinesthetic teaching task, which is revealed by an improved task performance, making kinesthetic teaching an efficient programming-by-demonstration method for redundant robots.

The recent advent of compliant and kinematically redundant robots poses new research challenges for human-robot interaction. While these robots provide a great degree of flexibility for the realization of complex applications, the flexibility gained generates the need for additional modeling steps and definition of criteria for redundancy resolution constraining the robot's movement generation. The explicit modeling of such criteria usually require experts to adapt the robot's movement generation subsystem. A typical way of dealing with this configuration challenge is to utilize kinesthetic teaching by guiding the robot to implicitly model the specific constraints in task and configuration space. We argue that current programming-by-demonstration approaches are not efficient for kinesthetic teaching of redundant robots and show that typical teach-in procedures are too complex for novice users. In order to enable non-experts to master the configuration and programming of a redundant robot in the presence of non-trivial constraints such as confined spaces, we propose a new interaction scheme combining kinesthetic teaching and learning within an integrated system architecture. We evaluated this approach in a user study with 49 industrial workers at HARTING, a medium-sized manufacturing company. The results show that the interaction concepts implemented on a KUKA Lightweight Robot IV are easy to handle for novice users, demonstrate the feasibility of kinesthetic teaching for implicit constraint modeling in configuration space, and yield significantly improved performance for the teach-in of trajectories in task space.

A major goal of current robotics research is to enable robots to become co-workers that collaborate with humans efficiently and adapt to changing environments or workflows. We present an approach utilizing the physical interaction capabilities of compliant robots with data-driven and model-free learning in a coherent system in order to make fast reconfiguration of redundant robots feasible. Users with no particular robotics knowledge can perform this task in physical interaction with the compliant robot, for example to reconfigure a work cell due to changes in the environment. For fast and efficient learning of the respective null-space constraints, a reservoir neural network is employed. It is embedded in the motion controller of the system, hence allowing for execution of arbitrary motions in task space. We describe the training, exploration and the control architecture of the systems as well as present an evaluation on the KUKA Light-Weight Robot. Our results show that the learned model solves the redundancy resolution problem under the given constraints with sufficient accuracy and generalizes to generate valid joint-space trajectories even in untrained areas of the workspace.

As a testbed for real-world experimentation on HRI and dynamic interaction models, this paper presents an autonomous robot system acting as guide in a German arts museum. The visitors' evaluation of this system is analyzed using a questionnaire and reveals issues for subsequent analysis of the real-time interaction.

In this paper we demonstrate the online applicability of the fault detection and diagnosis approach which we previously developed and published in [1]. In our former work we showed that a purely data driven fault detection approach can be successfully built based on monitored inter-component communication data of a robotic system and used for a-posteriori fault detection. Here we propose an extension to this approach which is capable of online learning of the fault model as well as for online fault detection. We evaluate the application of our approach in the context of a RoboCup task executed by our service robot BIRON in corporation with an expert user.

In order to address the problem of failure detection in the robotics domain, we present in this contribution a so-called self-awareness model, based on the system's internal data exchange and the inherent dynamics of inter-component communication. The model is strongly data driven and provides an anomaly detector for robotics systems both applicable in-situ at runtime as well as a-posteriori in post-mortem analysis. Current architectures or methods for failure detection in autonomous robots are either implementations of watch dog concepts or are based on excessive amounts of domain-specific error detection code. The approach presented in this contribution provides an avenue for the detection of more subtle anomalies originating from external sources such as the environment itself or system failures such as resource starvation. Additionally, developers are alleviated from explicitly modeling and foreseeing every exceptional situation, instead training the presented probabilistic model with the known normal modes within the specification of the robot system. As we developed and evaluated the self-awareness model on a mobile robot platform featuring an event-driven software architecture, the presented method can easily be applied in other current robotics software architectures.

In order to provide personalized services and to develop human-like interaction capabilities robots need to recognize their human partner. Face recognition has been studied in the past decade exhaustively in the context of security systems and with significant progress on huge datasets. However, these capabilities are not in focus when it comes to social interaction situations. Humans are able to remember people seen for a short moment in time and apply this knowledge directly in their engagement in conversation. In order to equip a robot with capabilities to recall human interlocutors and to provide user- aware services, we adopt human-human interaction schemes to propose a face memory on the basis of active appearance models integrated with the active memory architecture. This paper presents the concept of the interactive face memory, the applied recognition algorithms, and their embedding into the robot's system architecture. Performance measures are discussed for general face databases as well as scenario-specific datasets.

With the increasing interest in computer vision for interactive systems, the challenges of the development process involving many researchers are becoming more prominent. Issues like reuse of algorithms, modularity, and distributed processing are getting more important in the endeavor of building complex vision systems. We present a framework that allows independent development of enclosed components and supports interactive optimization of algorithmic parameters in an online fashion. The communication between components is performed nearly without any slow down compared to a monolithic system. Through the modular concept, all components can be flexibly distributed and reused in other application domains. The suitability of the approach is demonstrated with an example system.

In this paper, we present a case study that exemplifies general ideas of system integration and coordination. The application field of assistant technology provides an ideal test bed for complex computer vision systems including real-time components, human-computer interaction, dynamic 3-d environments, and information retrieval aspects. In our scenario the user is wearing an augmented reality device that supports her/him in everyday tasks by presenting information that is triggered by perceptual and contextual cues. The system integrates a wide variety of visual functions like localization, object tracking and recognition, action recognition, interactive object learning, etc. We show how different kinds of system behavior are realized using the Active Memory Infrastructure that provides the technical basis for distributed computation and a data- and eventdriven integration approach.

The development of robot companions with nat ural human-robot interaction (HRI) capabilities is a challeng ing task as it requires incorporating various functionalities. Consequently, a flexible infrastructure for controlling module operation and data exchange between modules is proposed, taking into account insights from software system integration. This is achieved by combining a three-layer control architec ture containing a flexible control component with a powerful communication framework. The use of XML throughout the whole infrastructure facilitates ongoing evolutionary develop ment of the robot companion's capabilities.