The growing complexity of commissioning industrial automation systems has increased the demand for simpler and more intuitive operation, often requiring domain-specific expertise. This paper explores task-oriented natural-language interaction with automation systems, a paradigm that is rapidly gaining traction particularly due to the advent of large language models (LLMs). Leveraging a multi-agent system (MAS), the authors introduce a two-phase approach for controlling an industrial gantry for pick-and-place operations. In phase one, the MAS generates an execution plan based on the user input; in phase two, the plan is executed deterministically on real-world hardware. Our preliminary results demonstrate the advantages of a chat-based human-in-the-loop (HiL) workflow for refining and validating execution plans, and they highlight the promise of natural-language interfaces for operating pick-and-place systems (Demo: Video 1, Video 2).

Research into robotics applications of deep reinforcement learning (DRL) has increasingly been focussed on learning precise object manipulation and trajectory planning. Extending these tasks to continuous robot-object interactions with the surface of complex geometries remains an open problem. In this paper we investigate end-to-end DRL solutions for depowdering tasks that work by directing a pressurized air stream onto the object's surfaces using a blast nozzle head mounted on a robotic arm. We develop a GPU accelerated vectorized cleaning effect for integration into RL training and consider ways to expose vision-less trajectory synthesis for surface treatment applications to the RL agent based on UV mapping. Our experimental evaluation demonstrates that DRL has the potential to be used for generating object-specific agents for depowdering tasks on a variety of 3D objects without requiring intermediate path planners even in a full 3D motion setup. Finally, we show that DRL-generated trajectories can be transferred to a real-world setup. Our task formulation lends itself to approximate a wide range of surface treatment applications (e.g., cleaning and spray painting) with various effects.

Human action recognition and motion forecasting is becoming increasingly successful, in particular with utilizing graphs. We aim to transfer this success into the context of industrial Human-Robot Collaboration (HRC), where humans work closely with robots and interact with workpieces in defined workspaces. For this purpose, it is necessary to use all the available information extractable in such a workspace and represent it with a natural structure, such as graphs, that can be used for learning. Since humans are the center of HRC, it is mandatory to construct the graph in a human-centered way and use real-world 3D information as well as object labels to represent their environment. Therefore, we present a novel Graph Neural Network (GNN) architecture which combines, human action recognition and motion forecasting for industrial HRC environments. We evaluate our method with two different and publicly available human action datasets, including one that is a particularly realistic representation of the industrial HRC, and compare the results with baseline methods for classifying the current human action and predicting the human motion. Our experiments show that our combined GNN approach improves the accuracy of action recognition compared to previous work, and significantly on the CoAx dataset by up to 20%. Further, our motion forecasting approach performs better than existing baselines, predicting human trajectories with a Final Displacement Error (FDE) of less than 10cm for a prediction horizon of 1s.

Automated assembly systems are becoming more and more complex, which also increases the effort for their planning. The increasing complexity of the products to be assembled also reinforces this effect. For that reason, concepts and systems must be developed to simplify the planning processes by partially automating them and thus reducing the overall effort. One phase in this planning process is the selection of suitable automation resources to carry out an assembly step. The concept presented in this paper aims to support this selection process. Therefore, the requirements of the production process are modelled and generated based on the product model. The resources, on the other hand, provide assurances to be matched within the selection process. These requirements and assurances are modelled generically and product-oriented without using a process taxonomy or ontology. Furthermore, a concept is presented for selecting optimal resource combinations to perform a single or even multiple assembly steps while considering the property dependencies between the individual resources.

Finding the right solution for a given automation problem is one of the biggest challenges for customers of industrial control and automation companies. This search needs to address customers' demands and preferences such as cost-effectiveness, energy-consumption, durability, working space, etc. Consequently, solutions may comprise different products of different providers. This leads to an enormous search space of diverse potential solutions. As of today, the search process involves human effort. Thus, a good understanding of system engineering is required not only on the manufacturer side but on the customer side as well. Furthermore, integration and configuration of the products into a functional system needs extra effort. In this paper, we present our first ideas on reducing the complexity of this search task. Our approach is laid out in three layers: (i) a customer only needs to describe their desired automation application on an abstract level. (ii) manufacturers need to categorize their products as per functionality and characteristics. (iii) the linking between the requirements as per customer's description to relevant products is done automatically. This paper provides a roadmap for the research necessary to implement these three layers in practice. We lay out the essential research questions and provide a conceptual division of the work, thus pointing out the challenges that need to be solved to allow for further automation in this area.

As a result of the increased demand for bandwidth, current automotive networks are getting more heterogeneous. New technologies like Ethernet as a packet-switched point-to-point network are introduced. Nevertheless, the requirements on stand-by power consumption and short activation times are still the same as for existing field buses. Ethernet does not provide wakeup mechanisms that are sufficient for automotive systems. As a remedy, this paper introduces a novel physical-layer mechanism called Low Frequency Wakeup that is largely independent of the communication technology and topology used. It provides parallel and remote wakeup for all nodes even in a point-to-point network as well as full support of partial networking. The overall wakeup detection time is smaller than 10ms and every node can actively feed a wakeup signal asynchronously to all other nodes. In terms of latency, it is shown that Low Frequency Wakeup reaches a reduction of more than 30 % for a three-hop network and more than 50 % for a five-hop network in comparison to the current state-of-the-art technology for automotive point-to-point networks.