We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation with a versatile onboarding process. Our pipeline begins with an onboarding stage that generates object representations from either traditional 3D CAD models or, in their absence, by rapidly reconstructing a high-fidelity neural representation (NeRF) from multi-view images. Given a test image, our system first employs the CNOS detector to localize target objects. For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose. The core of OPFormer is a transformer-based architecture that leverages a foundation model for robust feature extraction. It uniquely learns an object representation by jointly encoding multiple template views and enriches these features with explicit 3D geometric priors using Normalized Object Coordinate Space (NOCS). A decoder then establishes robust 2D3D correspondences to determine the final pose. Evaluated on the challenging BOP benchmarks, our integrated system demonstrates a strong balance between accuracy and efficiency, showcasing its practical applicability in both model-based and model-free scenarios.

Current generalizable object pose estimators, i.e., approaches that do not need to be trained per object, rely on accurate 3D models. Predominantly, CAD models are used, which can be hard to obtain in practice. At the same time, it is often possible to acquire images of an object. Naturally, this leads to the question of whether 3D models reconstructed from images are sufficient to facilitate accurate object pose estimation. We aim to answer this question by proposing a novel benchmark for measuring the impact of 3D reconstruction quality on pose estimation accuracy. Our benchmark provides calibrated images suitable for reconstruction and registered with the test images of the YCB-V dataset for pose evaluation under the BOP benchmark format. Detailed experiments with multiple state-of-the-art 3D reconstruction and object pose estimation approaches show that the geometry produced by modern reconstruction methods is often sufficient for accurate pose estimation. Our experiments lead to interesting observations: (1) Standard metrics for measuring 3D reconstruction quality are not necessarily indicative of pose estimation accuracy, which shows the need for dedicated benchmarks such as ours. (2) Classical, non-learning-based approaches can perform on par with modern learning-based reconstruction techniques and can even offer a better reconstruction time-pose accuracy tradeoff. (3) There is still a sizable gap between performance with reconstructed and with CAD models. To foster research on closing this gap, the benchmark is made available at https://github.com/VarunBurde/reconstruction_pose_benchmark.

This paper explores the integration of augmented reality into robotic manufacturing systems, emphasizing the enhancement of connectivity, real-time data processing, and interactive visual interfaces. Utilizing Hololens 2 headsets equipped with OPC UA clients, the system establishes a direct interface between operators and the digital and physical components of the manufacturing environment. The architecture employs augmented reality to facilitate sophisticated operational control and visualization, ranging from robotic additive manufacturing to complex tasks like robotic sanding and screwing with integrated camera systems. This integration showcases significant advancements in manufacturing processes, allowing operators to engage interactively with the systems, optimize workflows, and improve overall precision and efficiency in a smart factory setting.

This study aims to integrate and evaluate digital twin technology with robotic multi-axis additive manufacturing to bridge the gap between virtual planning and physical execution, enhancing the precision and efficiency of manufacturing processes. A laser tracker is employed to calibrate the machine workspace and tools and introduce a comprehensive system involving an industrial robot with a filament extruder head and a rotary-tilt positioner. The digital twin not only encapsulates the geometric and operational intricacies of the multi-axis additive manufacturing process but also automates code generation for robotic operations. The application of this methodology is demonstrated through the production of two complex parts, where their dimensional fidelity is assessed to evaluate the precision limits and potential of digital twin technology in multi-axis additive manufacturing. The findings reveal significant advancements in the alignment of virtual models and physical objects, confirming the potential of digital twins to minimize deviations common in traditional manufacturing setups.

The exponential proliferation of big data and computation-intensive tasks, such as Artificial Intelligence (AI) applications in factories, poses a significant challenge for the current datacenter-focused technological architecture. The “Big data pRocessing and Artificial Intelligence at the Network Edge” (BRAINE) project addresses this problem by introducing an innovative system architecture designed explicitly for compute-intensive edge deployments. BRAINE focuses on decentralizing the computation tasks, enabling a significant reduction in latency, and optimizing the placement of applications within a cloud-edge continuum to ensure optimal operational efficiency. This paper presents the design, implementation, and testing of our novel system architecture in the context of an AI digital twin for factory robotics. Our empirical results indicate substantial improvements in performance metrics such as processing speed and latency compared to traditional architectures and approaches.

Modern manufacturing faces a challenge of integrating data models from various sources/domains which may differ both semantically and technically when particular domain specific data models are designed by different users and stored in different formats. This paper introduces an approach for facilitating the design of domain-specific data models using semantic web technologies. In this approach, all the information required for managing the production (including a description of a product, processes involved in the production, and existing resources and their specifications) is captured in an ontology. The proposed Product, Process, and Resource (PPR) ontology defines fundamental conceptualization of the production that can be easily applied to the arbitrary domain. Application of the PPR ontology is demonstrated in the case of simple truck assembling by means of robots. Capturing the knowledge in the form of ontology provides the advantage of employing supporting tools such as reasoners for consistency checking or query languages for information extraction. The paper demonstrates the utilization of SQWRL for searching resources suitable to manipulate given truck parts on the basis of semantic matching between properties of particular elements.

This study focuses on the energy optimization of industrial robotic cells, which is essential for sustainable production in the long term. A holistic approach that considers a robotic cell as a whole toward minimizing energy consumption is proposed. The mathematical model, which takes into account various robot speeds, positions, power-saving modes, and alternative orders of operations, can be transformed into a mixed-integer linear programming formulation that is, however, suitable only for small instances. To optimize complex robotic cells, a hybrid heuristic accelerated by using multicore processors and the Gurobi simplex method for piecewise linear convex functions is implemented. The experimental results showed that the heuristic solved 93% of instances with a solution quality close to a proven lower bound. Moreover, compared with the existing works, which typically address problems with three to four robots, this study solved real-size problem instances with up to 12 robots and considered more optimization aspects. The proposed algorithms were also applied on an existing robotic cell in řkoda Auto. The outcomes, based on simulations and measurements, indicate that, compared with the previous state (at maximal robot speeds and without deeper power-saving modes), the energy consumption can be reduced by about 20% merely by optimizing the robot speeds and applying power-saving modes. All the software and generated datasets used in this research are publicly available.

This paper presents a way how a simulation model of an industrial Ethernet network is designed and how it is parametrised according to a real network. The parameter set up is done by capturing the Ethernet frames in the network and by performing their detailed analysis. The device types in the simulation have been designed as generic that can be parametrised externally and automatically based on the identification of a real network. Thus the parameters of the real devices such as switch delay are used to adapt the simulation and the simulation results are compared to the real traffic in a series of long-term observations. Such a setting can be used to perform diagnostics of industrial Ethernet networks to identify e.g. bursts of messages and thus discovering potential bottlenecks and problems aiming at interrupting or disturbing the connected application process, eventually.

This paper presents an algorithm that allows one to create a static schedule of the Profinet IO IRT (Isochronous Real Time) communication, which is an industrial Ethernet protocol standardized in IEC 61158. This algorithm offers an alternative to the available commercial tool, providing comparable results with respect to the resulting schedule makespan. Furthermore, the problem is extended by useful temporal constraints (i.e., release dates, deadlines and end-to-end deadlines of the messages) providing a greater flexibility with respect to the individual messages. Due to this flexibility, it is possible to place the selected messages in various parts of the communication cycle (in order to increase the computational time available for the main-controller application, or to retransmit the synchronization message without holdup in the switch, or to add new messages into the original schedule). The solution is based on a formulation of the Profinet IO IRT scheduling problem in terms of the Resource Constrained Project Scheduling with Temporal Constraints.

This paper presents an algorithm that allows one to create a static schedule of the Profinet IO IRT communication, which is an industrial Ethernet protocol standardised in IEC$\;$61158. This algorithm offers an alternative to the available commercial tool, providing comparable results regarding the resulting time schedule length. Furthermore, we extend the problem by useful time constraints providing a greater flexibility with respect to the individual messages. Due to this flexibility, it is possible to place the selected messages in various parts of the communication cycle, to define end-to-end delays, or to increase the computational time available for the main-controller application, for example. The solution is based on a formulation of the Profinet IO IRT scheduling problem in terms of the Resource Constrained Project Scheduling with Temporal Constraints (PS|temp|Cmax).