Surface finishing is a time-consuming, dangerous task, difficult to automate despite its necessity in many manufacturing processes. Its automation, particularly through robotics, increases productivity and relieves workers from health-critical tasks. However, challenges remain, as automated offline planning tools can result in certain areas being either neglected or overly processed. Ergodic control offers the possibility to cover target probability distributions in an online manner, by taking into account the observed coverage history. However, existing ergodic control approaches provide little flexibility in designing and adapting coverage strategies. Moreover, they come with simplifying assumptions, such as point-based dynamics, which are no longer valid for tasks where the robot is in contact with strongly varying curvatures on non-trivial surface geometries. In this work, we introduce a closed-form ergodic control framework that includes the tool imprint in the system modeling while simultaneously permitting the intuitive transfer of finish strategies, namely preferred motion directions. We build on the Spectral Multiscale Coverage (SMC) approach, augmenting it with a tool imprint model, as well as both target distributions and state-dependent movement directions extracted from human demonstrations. Through evaluations in a surface finishing task using a torque-controlled, 7-DoF, robot arm we show that our approach optimally covers surfaces according to the tool contact area, with robust error convergence.

Explainability is vital for establishing user trust, also in robotics. Recently, foundation models (e.g. vision-language models, VLMs) fostered a wave of embodied agents that answer arbitrary queries about their environment and their interactions with it. However, naively prompting VLMs to answer queries based on camera images does not take into account existing robot architectures which represent the robot's tasks, skills, and beliefs about the state of the world. To overcome this limitation, we propose RACCOON, a framework that combines foundation models' responses with a robot's internal knowledge. Inspired by Retrieval-Augmented Generation (RAG), RACCOON selects relevant context, retrieves information from the robot's state, and utilizes it to refine prompts for an LLM to answer questions accurately. This bridges the gap between the model's adaptability and the robot's domain expertise.

Technology has sufficiently matured to enable, in principle, flexible and autonomous robotic assembly systems. However, in practice, it requires making all the relevant (implicit) knowledge that system engineers and workers have – about products to be assembled, tasks to be performed, as well as robots and their skills – available to the system explicitly. Only then can the planning and execution components of a robotic assembly pipeline communicate with each other in the same language and solve tasks autonomously without human intervention. This is why we have developed the Factory of the Future (FoF) ontology. At its core, this ontology models the tasks that are necessary to assemble a product and the robotic skills that can be employed to complete said tasks. The FoF ontology is based on existing standards. We started with theoretical considerations and iteratively adapted it based on practical experience gained from incorporating more and more components required for automated planning and assembly. Furthermore, we propose tools to extend the ontology for specific scenarios with knowledge about parts, robots, tools, and skills from various sources. The resulting scenario ontology serves us as world model for the robotic systems and other components of the assembly process. A central runtime interface to this world model provides fast and easy access to the knowledge during execution. In this work, we also show the integration of a graphical user front-end, an assembly planner, a workspace reconfigurator, and more components of the assembly pipeline that all communicate with the help of the FoF ontology. Overall, our integration of the FoF ontology with the other components of a robotic assembly pipeline shows that using an ontology is a practical method to establish a common language and understanding between the involved components.