This paper introduces a novel physical annotation system that is designed to generate training data for automated optical inspection. The system uses pointer-based, in-situ interaction to transfer the valuable expertise of trained inspection personnel directly into a machine learning training pipeline. Unlike conventional screen-based annotation methods, our system allows annotation directly on the physical object, providing a more intuitive and efficient way to label data. The core technology uses calibrated, tracked pointers to accurately record user input and convert these spatial interactions into standardised annotation formats compatible with open-source software. A simple projector-based interface also projects visual guidance onto the object to assist users during the annotation process, ensuring greater accuracy and consistency. The proposed concept bridges the gap between human expertise and automated data generation. It enables non-IT experts to contribute to the ML training pipeline. Preliminary evaluation results confirm the feasibility of capturing detailed annotation trajectories and demonstrate that integration with CVAT streamlines the workflow for subsequent ML tasks. This paper details the system architecture, calibration procedures, and interface design, and discusses the concept’s potential contribution to future ML-based automated optical inspection.

We introduce IndEgo, a multimodal egocentric and exocentric dataset addressing common industrial tasks, including assembly/disassembly, logistics and organisation, inspection and repair, woodworking, and others. The dataset contains 3,460 egocentric recordings (approximately 197 hours), along with 1,092 exocentric recordings (approximately 97 hours). A key focus of the dataset is collaborative work, where two workers jointly perform cognitively and physically intensive tasks. The egocentric recordings include rich multimodal data and added context via eye gaze, narration, sound, motion, and others. We provide detailed annotations (actions, summaries, mistake annotations, narrations), metadata, processed outputs (eye gaze, hand pose, semi-dense point cloud), and benchmarks on procedural and non-procedural task understanding, Mistake Detection, and reasoning-based Question Answering. Baseline evaluations for Mistake Detection, Question Answering and collaborative task understanding show that the dataset presents a challenge for the state-of-the-art multimodal models. Our dataset is available at: https://huggingface.co/datasets/FraunhoferIPK/IndEgo

Automating machine tending for high mix - low volume environment provides additional engineering challenges. Current machine tending systems are usually fixed in place and preprogrammed by a human operator. Available mobile robots are limited in their machine tending capabilities as they cannot provide the required accuracy that is necessary for the precise insertion of work pieces into a fixture. This paper presents a mobile machine tending system that uses computer vision and ROS2 to address these limitations and to enable autonomous machine tending. After a brief introduction to the system architecture, the article provides first insights into the systems capabilities based on experimental evaluation. The first experiment addresses the accuracy of the end effector, showing reasonable improvements using low cost sensors. The second experiment evaluates the network load of the distributed control system and discusses the requirements for the operation of a fleet of such robots.

Easy-to-use programming methods for robotic manufacturing systems are a key element in enabling a wider audience to benefit from increasing automation. This paper provides an overview of current programming methods for industrial mating operations. More specifically, the review focuses on programming methods for welding and assembly tasks. While welding is mainly concerned with the exact definition of the free space trajectories, assembly tasks revolve around the contact with the environment. Thus, the two tasks at hand not only represent two of the most common robotic applications, but also two distinct groups of operations that place very different demands on the programming interface.

Classic off- and online programming approaches offer different advantages but cannot provide intuitive programming as it would be needed for frequent reconfiguration of industrial robotic cells and their programs. While a simulation and therewith a collision control can be used offline, highest precision can be achieved by using online teach-in. We propose a system that combines the advantages of both offline and online programming. By creating a programming framework that is independent of in- and output devices we allow the use of augmented as well as virtual reality. Thus, enabling the user to choose for a programming technique that best suits his needs during different stages of programming. Bringing together both methods allows us to not only drastically reduce programming time but simultaneously increase intuitiveness of human robot interaction.

This paper is a work in progress report on a novel system for intuitive gesture based robot programming. The major contribution is the addition of an expert system into the robot programming process. The expert system uses static knowledge, such as seam types, and dynamic knowledge to reason about the intended process. The dynamic knowledge holds information about the environment and is derived from sensor data. The paper outlines the required components and the current state of development of a prototype system. An example of the inference process is given for a robot laser welding application.