This paper presents an overview about a programming concept for an industrial HRI cell designed for packaging of electronic consumer goods. The focus of this work lies on the interplay of the involved software components. Furthermore, developed methods for programming the software components of the HRI cell are described within a sample use case. Finally, the usability of the programming concept and a short evaluation of relevant components are presented.

Today's manufacturing and assembly systems have to be flexible to adapt quickly to an increasing number and variety of products, and changing market volumes. To manage these dynamics, several production concepts (e.g., flexible, reconfigurable, changeable or autonomous manufacturing and assembly systems) were proposed and partly realized in the past years. This paper presents the general principles of autonomy and the proposed concepts, methods and technologies to realize cognitive planning, cognitive control and cognitive operation of production systems. Starting with an introduction on the historical context of different paradigms of production (e.g., evolution of production and planning systems), different approaches for the design, planning, and operation of production systems are lined out and future trends towards fully autonomous components of an production system as well as autonomous parts and products are discussed. In flexible production systems with manual and automatic assembly tasks, human-robot cooperation is an opportunity for an ergonomic and economic manufacturing system especially for low lot sizes. The state-of-the-art and a cognitive approach in this area are outlined. Furthermore, introducing self-optimizing and self-learning control systems is a crucial factor for cognitive systems. This principles are demonstrated by a quality assurance and process control in laser welding that is used to perform improved quality monitoring. Finally, as the integration of human workers into the workflow of a production system is of the highest priority for an efficient production, worker guidance systems for manual assembly with environmentally and situationally dependent triggered paths on state-based graphs are described in this paper.

Every day human communication relies on a large number of different communication mechanisms like spoken language, facial expressions, body pose and gestures, allowing humans to pass large amounts of information in short time. In contrast, traditional human-machine communication is often unintuitive and requires specifically trained personal. In this paper, we present a real-time capable framework that recognizes traditional visual human communication signals in order to establish a more intuitive human-machine interaction. Humans rely on the interaction partner’s face for identification, which helps them to adapt to the interaction partner and utilize context information. Head gestures (head nodding and head shaking) are a convenient way to show agreement or disagreement. Facial expressions give evidence about the interaction partners’ emotional state and hand gestures are a fast way of passing simple commands. The recognition of all interaction queues is performed in parallel, enabled by a shared memory implementation.

This paper introduces a new visual tracking technique combining particle filtering and Dynamic Bayesian Networks. The particle filter is utilized to robustly track an object in a video sequence and gain sets of descriptive object features. Dynamic Bayesian Networks use feature sequences to determine different motion patterns. A Graphical Model is introduced, which combines particle filter based tracking with Dynamic Bayesian Network-based classification. This unified framework allows for enhancing the tracking by adapting the dynamical model of the tracking process according to the classification results obtained from the Dynamic Bayesian Network. Therefore, the tracking step and classification step form a closed tracking-classification-tracking loop. In the first layer of the Graphical Model a particle filter is set up, whereas the second layer builds up the dynamical model of the particle filter based on the classification process of the Dynamic Bayesian Network.

In everyday live head gestures such as head shaking or nodding and hand gestures like pointing gestures form important aspects of human-human interaction. Therefore, recent research considers integrating these intuitive communication cues into technical systems for improving and easing human-computer interaction. In this paper we present a vision-based system to recognize head gestures (nodding, shaking, neutral) and dynamic hand gestures (hand moving right/left/up/down, fist moving right/left) in real-time. The gestural input delivers a communication modality for a human-robot interaction scenario situated in an assistive household environment. The use of fast low-level image-feature extraction methods contributes to the real-time capability of the system and advanced classification approaches relying on Graphical Models provide high robustness. Graphical Models offer the possibility to group the input features in several sub-nodes resulting in a better classification than obtained via a traditional Hidden Markov Model classification. The applied grouping can regard interdependencies owing to, either physical constraints (like for the head gestures), or interrelations between shape and motion (like for the hand gestures).

In this paper, the development of a framework based on the Real-time Database (RTDB) for processing multimodal data is presented. This framework allows readily integration of input and output modules. Furthermore the asynchronous data streams from different sources can be approximately processed in a synchronous manner. Depending on the included modules, online as well as offline data processing is possible. The idea is to establish a real multimodal interaction system that is able to recognize and react to those situations that are relevant for human-robot interaction.

This paper presents a concept of a smart working environment designed to allow true joint-actions of humans and industrial robots. The proposed system perceives its environment with multiple sensor modalities and acts in it with an industrial robot manipulator to assemble capital goods together with a human worker. In combination with the reactive behavior of the robot, safe collaboration between the human and the robot is possible. Furthermore, the system anticipates human behavior, based on knowledge databases and decision processes, ensuring an effective collaboration between the human and robot. As a proof of concept, we introduce a use case where an arm is assembled and mounted on a robotpsilas body.

This article presents an integrated framework for multi-modal adaptive cognitive technical systems to guide, assist and observe human workers in complex manual assembly environments. The demand for highly flexible construction facilities obviously contradicts longer training and preparation phases of human workers. By giving context-aware building instructions over retina displays, text-to-speech commands or acoustical signals, a non-specialized industrial stand-by men in a production task can precisely be alloted to execute the next processing step without any previous knowledge. Using non-invasive gesture recognizers and object detectors the human worker can be observed in order to track the production line and initiate the subsequent step in the interaction loop. Aiming at testing and evaluating the desired human-machine interfaces and its capabilities a virtual working place together with a concrete use case is introduced.