The application of DNNs for CV in industrial environments holds significant promise for enhancing production efficiency. However, SMEs often face adoption barriers due to limited technical resources, cost constraints, and reliability concerns—particularly when dealing with disturbances in image acquisition or transmission. This paper discusses ongoing work within the DeepFrame project, which aims to improve the robustness of deep learning systems for industrial sensor data. Rather than presenting a finalized framework, we outline a research agenda and conceptual design to address challenges posed by real-world data disturbances. Building on existing robustness research and insights gathered through expert interviews we propose a conceptual system with two key components: (1) dataset engineering using automated synthetic data generation and robustness evaluation; and (2) multi-modal neural representation to fuse sensor data and reconstruct disrupted inputs.

We address the problem of robotic grasping of known and unknown objects using implicit behavior cloning. We train a grasp evaluation model from a small number of demonstrations that outputs higher values for grasp candidates that are more likely to succeed in grasping. This evaluation model serves as an objective function, that we maximize to identify successful grasps. Key to our approach is the utilization of learned implicit representations of visual and geometric features derived from a pre-trained NeRF. Though trained exclusively in a simulated environment with simplified objects and 4-DoF topdown grasps, our evaluation model and optimization procedure demonstrate generalization to 6-DoF grasps and novel objects both in simulation and in real-world settings, without the need for additional data. Supplementary material is available at: https://gergely-soti.github.io/grasp

Offloading software modules to the edge/cloud can enhance a robot’s capabilities by leveraging massive computing power. However, determining which software module should be offloaded and scheduled to which robot/edge/cloud node is a challenging task, particularly for robot fleets with diverse tasks. In this paper, we tackle the software scheduling problem and introduce a taxonomy to categorize software modules and classify their applicability and requirements for offloading. Additionally, by using prior measurements, we model the compute cluster and formalize software scheduling as a multi-objective optimization problem which we tackle with a genetic algorithm. To evaluate our approach with a challenging setup, we build a mobile manipulation task using open-source frameworks and libraries in the Robot Operating System (ROS2) community in simulation as well as a mildly simplified real-world variant. Our evaluation shows significant improvements compared to the built-in scheduler of Kubernetes (K8s) regarding robotic specific metrics such as the rate of missed cycle time in both simulated and real-world experiments.

Precise pick-and-place is essential in robotic applications. To this end, we define an exact training method and an iterative inference method that improve pick-and-place precision with Transporter Networks [1]. We conduct a large scale experiment on 8 simulated tasks. A systematic analysis shows, that the proposed modifications have a significant positive effect on model performance. Considering picking and placing independently, our methods achieve up to 60% lower rotation and translation errors than baselines. For the whole pick-and-place process we observe 50% lower rotation errors for most tasks with slight improvements in terms of translation errors. Furthermore, we propose architectural changes that retain model performance and reduce computational costs and time. We validate our methods with an interactive teaching procedure on real hardware. Supplementary material is available at: https://gergely-soti.github.io/p3

As advanced algorithms enable robots to handle more challenging tasks and operate more autonomously, the on-board computer cannot meet the increased demands regarding computing power and memory storage in an efficient way. Leveraging the massive computing power of the cloud and low-latency connectivity to the edge can compensate for this lack of computing resources. However, this introduces a new challenge related to the deployment of complex robotic software across multiple devices, especially in a large-scale system. This paper presents KubeROS, a unified and fully managed platform for automated deployment of robotic applications developed on top of Robot Operating System 2 (ROS2), in a hybrid computing infrastructure with robots, edge and cloud. KubeROS uses Kubernetes from Cloud Native Computing as its underlying software orchestration framework. It aims to help researchers and developers with no prior cloud computing knowledge deploy their ROS2-based robotic applications at any scale. KubeROS eliminates the need for system configuration and network setup. We demonstrate the applicability of KubeROS by deploying a fleet of simulated mobile manipulators in a clas-sical pick-and-place application. The experiments demonstrate the effects of different deployment strategies for vision-based motion planning under different fleet sizes and workloads. In addition, KubeROS improves task performance by using high-performance computing at the edge and in the cloud, and achieves high resource efficiency when using the shared deployment strategy.

This paper proposes a reachability-aware model predictive control with a discrete control barrier function for backward obstacle avoidance for a tractor-trailer system. The framework incorporates the state-variant reachable set obtained through sampling-based reachability analysis and symbolic regression into the objective function of model predictive control. By optimizing the intersection of the reachable set and iterative non-safe region generated by the control barrier function, the system demonstrates better performance in terms of safety with a constant decay rate, while enhancing the feasibility of the optimization problem. The proposed algorithm improves real-time performance due to a shorter horizon and outperforms the state-of-the-art algorithms in the simulation environment and on a real robot.

Presents an approach to parallel path planning for industrial robot arms with six degrees of freedom in an online given 3D environment. The method is based a best-first search algorithm and needs no essential off-line computations. The algorithm works in an implicitly discrete configuration space. Collisions are detected in the Cartesian workspace by hierarchical distance computation based on polyhedral models of the robot and the obstacles. By decomposing the 6D configuration space into hypercubes and cyclically mapping them onto multiple processing units, a good load distribution can be achieved. We have implemented the parallel path planner on a workstation cluster with 9 PCs and tested the planner for several benchmark environments. With optimal discretisation, the new approach usually shows very good speedups. In online provided environments with static obstacles, the parallel planning times are only a few seconds.

The paper is based on a path planning approach for industrial robot arms with 6 degrees of freedom in an online given 3D environment. It has online capabilities by searching in an implicit and discrete configuration space and detecting collisions in the Cartesian workspace by distance computation based on the given CAD model. Here, we present different methods for specifying the C-space discretization. Besides the usual uniform and heuristic discretization, we investigate two versions of an optimal discretization for a user-predefined Cartesian resolution. The different methods are experimentally evaluated. Additionally, we provide a set of 3-dimensional benchmark problems for a fair comparison of the path planner. For each benchmark, the run-times of our planner are between only 3 and 100 seconds on a Pentium PC with 133 MHz.