Abstract This paper proposes an approach to pattern-based modeling and Uppaal-based verification for ROS 2 applications. The proposed verification focuses on callback execution latencies and buffer overflow. We propose formal model templates to model the execution of ROS 2 system components, created using a pattern-based approach. The model templates simplify the formal modeling of an ROS 2 application. Using Uppaal, we model in Uppaal timed automata, allowing the description of computation chains of ROS 2-based applications. Our focus is on execution behavior, including two versions of the mainline single-threaded executor of ROS 2. System traces generated using the formal models are validated in multiple experiments. Furthermore, we compare two approaches to modeling the execution of nodes that are typically the core units of computation of ROS 2. The first approach is a holistic approach to model ROS 2 applications, including communication and execution in computation chains. The second is an approach for individual nodes only, at a higher abstraction level. Additionally, we show the application of the verification by model checking in two ROS 2 system scenarios where we compare generated model traces to actual system executions. Overall, through formal modeling and verification, we showcase the potential for uncovering errors in the execution of distributed robotic systems.

Multi-UAV mission control requires a software solution that helps the operator to supervise individual UAVs and coordinate groups of units. This paper explores the experience and methodology of a User Interface for multi-UAVs systems developed for supporting a European project to aggregate heterogeneous UAVs in farming domain. The aim of the proposed system is to support the operator with the extra workload required for supervision of several UAVs while keeping their Situational Awareness (SA) high. The solution is evaluated with respect to workload and SA. A text-to-speech (TTS) approach is also evaluated to assess its effect on these metrics. The results show that while a TTS solution reduces the operator’s workload, over-reliance on the system can cause loss of SA for properties which are not directly presented by TTS.

Robot operating system 2 (ROS 2) is increasingly popular both in research and commercial robotic systems. ROS 2 is designed to allow real-time execution and data communication, enabling rapid prototyping and deployment of robotic systems. In order to predict and calculate execution times in ROS 2, one needs to analyze its internal scheduler, called executor. The executor has been updated in various distributions of ROS 2, which is shown to impact significantly the periodic execution invoked by the underlying operating system’s timers, potentially causing unexpected latencies. To expose the mentioned impact due to executor differences, in this paper, we present an experimental evaluation of the execution behavior of ROS 2’s schedulable entities, namely callbacks, among the existing versions of the executor. We visualize the differences of callback execution order via simulation, and we create design-level scenarios that impact the execution of periodically scheduled callbacks, negatively. Moreover, we show how such negative impact can be mitigated by using multi-threaded executors. Finally, we illustrate the observed behavior on a real-world centralized multi-agent robot system. Our work aims to raise awareness within the ROS 2 developer community, regarding possible problems of timer blocking, and propose a mitigation solution of the latter.

With the rapidly growing use of Multi-Agent Systems (MASs), which can exponentially increase the system complexity, the problem of planning a mission for MASs became more intricate. In some MASs, human operators are still involved in various decision-making processes, including manual mission planning, which can be an ineffective approach for any non-trivial problem. Mission planning and re-planning can be represented as a combinatorial optimization problem. Computing a solution to these types of problems is notoriously difficult and not scalable, posing a challenge even to cutting-edge solvers. As time is usually considered an essential resource in MASs, automated solvers have a limited time to provide a solution. The downside of this approach is that it can take a substantial amount of time for the automated solver to provide a sub-optimal solution. In this work, we are interested in the interplay between a human operator and an automated solver and whether it is more efficient to let a human or an automated solver handle the planning and re-planning problems, or if the combination of the two is a better approach. We thus propose an experimental setup to evaluate the effect of having a human operator included in the mission planning and re-planning process. Our tests are performed on a series of instances with gradually increasing complexity and involve a group of human operators and a metaheuristic solver based on a genetic algorithm. We measure the effect of the interplay on both the quality and structure of the output solutions. Our results show that the best setup is to let the operator come up with a few solutions, before letting the solver improve them.

A superpixel segment is a group of pixels that carry similar information. The Simple Linear Iterative Clustering (SLIC) is a well-known algorithm for generating superpixels that offers a good balance between accuracy and efficiency. Nevertheless, due to its high computational requirements, the algorithm does not meet the demands of real-time embedded applications in terms of speed and resources. This paper proposes a fully-pipelined FPGA architecture based on SLIC, dubbed FP-SLIC, that exhibits 1) a simplified and efficient algorithm of reduced computational complexity that facilitates algorithm development for FPGAs, 2) a fully pipelined FPGA design operating at 40MHz with a throughput of one pixel per cycle, and 3) a memory-efficient architecture that eliminates the requirement for external memory. FP-SLIC shows promising BSDS500 benchmark results, especially considering boundary recall for less than 1000 superpixels, where it performs better than related works, while, at the same time, accomplishing a throughput of 259 frames per second (fps).

Multi-agent robot systems, specifically mobile robots in dynamic environments interacting with humans, e.g., assisting in production environments, have seen an increased interest over the past years. To better understand the ROS2 communication in a network with a high load of nodes, this paper investigates the communication handling of multiple robots to a single tracking node for centralized multi-agent robot systems using ROS2. Thereore, a quantitative analysis of two publisher-subscriber communication architectures and a comparative study between DDS vendors (CycloneDDS, FastDDS and GurumDDS) using ROS2 Galactic is performed. The architectures of consideration are a many-to-one approach, where multiple robots communicate to a central node over one topic, and the one-to-one communication approach, where multiple robots communicate over particular topics to a central node. Throughout this work, the increase in the number of robots at different publishing rates is simulated on a single computer for the different DDS vendors. A further simulation is done using a distributed setup with CycloneDDS. The simulations show that with an increase in the number of nodes, the average data age and the data miss ratio in the one-to-one approach were significantly lower than in the many-to-one approach. CycloneDDS was shown as the most robust regarding crashes and response time under system launch, while FastDDS showed better results regarding the data ageing.

The use of multiagent systems (MASs) in real-world applications keeps increasing, and diffuses into new domains, thanks to technological advances, increased acceptance, and demanding productivity requirements. Being able to automate the generation of mission plans for MASs is critical for managing complex missions in realistic settings. In addition, finding the right level of abstraction to represent any generic MAS mission is important for being able to provide general solution to the automated planning problem. In this article, we show how a mission for heterogeneous MASs can be cast as an extension of the traveling salesperson problem (TSP), and we propose a mixed-integer linear programming formulation. In order to solve this problem, a genetic mission planner (GMP), with a local plan refinement algorithm, is proposed. In addition, the comparative evaluation of CPLEX and GMP is presented in terms of timing and optimality of the obtained solutions. The algorithms are benchmarked on a proposed set of different problem instances. The results show that, in the presence of timing constraints, GMP outperforms CPLEX in the majority of test instances.

Automation and robotics are two enablers for developing the Smart Factory of the Future, which is based on intelligent machines and collaboration between robots and humans. Especially in final assembly and its material handling, where traditional automation is challenging to use, collaborative robot (cobot) systems may increase the flexibility needed in future production systems. A major obstacle to deploy a truly collaborative application is to design and implement a safe and efficient interaction between humans and robot systems while maintaining industrial requirements such as cost and productivity. Advanced and intelligent control strategies is the enabler when creating this safe, yet efficient, system, but is often hard to design and build.This paper highlights and discusses the challenges in meeting safety requirements according to current safety standards, starting with the mandatory risk assessment and then applying risk reduction measures, when transforming a typical manual final assembly station into an intelligent collaborative station. An important conclusion is that current safety standards and requirements must be updated and improved and the current collaborative modes defined by the standards community should be extended with a new mode, which in this paper is refereed to the deliberative planning and acting mode.

This paper presents a novel path planning system for multiple robots working in an uncontrolled environment in the presence of humans. The approach combines the use of Petri net to plan the movement of multiple robots to prevent the risk of congestion caused by routing several robots into a narrow region, together with a dipole field with dynamic window approach to avoid collisions of a robot with dynamic obstacles. By regarding the velocity and direction of both humans and robots as a source of magnetic dipole moment, the dipole-dipole interaction between the moving objects will generate repulsive forces to prevent collisions. The whole system is presented on robot operating system platform so that its implementation can be extendable into real robots. Experimental results with Gazebo simulator demonstrates the effectiveness of the proposed approach.

During the last five decades, automation and robotics have transformed the automotive industry by increasing efficiency and improving the product quality. However, future trucks that will be autonomous, electrical and connected will require a completely new type of flexibility and intelligence in the production systems, especially in the final assembly. To handle the increased complexity of the products, production processes and logistic systems, final assembly must be transformed into collaborative and intelligent automation systems. These systems will include collaborative and deliberative robots (cobots), advanced vision-based control, adaptive safety systems, online optimization and learning algorithms and connected and well-informed human operators. But it will be a huge undertaking to transform current trucks industry such that they can design, implement and maintain large scale collaborative and intelligent automation systems. This paper presents the challenges with current planning and preparation processes for final assembly as well as the requirement and possible solutions for the future processes. An industrial use case at Volvo Trucks based on Sequence Planner and ROS2 is used to evaluate the proposed planning and preparation processes.

In recent years, multi-agent systems have been widely used in different missions, ranging from underwater to airborne. A mission typically involves a large number of agents and tasks, making it very hard for the human operator to create a good plan. A search for an optimal plan may take too long, and it is hard to make a time estimate of when the planner will finish. A genetic algorithm based planner is proposed in order to overcome this issue. The contribution of this paper is threefold. First, an Integer Linear Programming (ILP) formulation of a novel Extensive Colored Traveling Salesperson Problem (ECTSP) is given. Second, a new objective function suitable for multi-agent mission planning problems is proposed. Finally, a reparation algorithm to allow usage of common variation operators for ECTSP has been developed.

The Census Transform (CT) is a well proven method for stereo vision that provides robust matching, with respect to object boundaries, outliers and radiometric distortion, at a low computational cost. Recent CT methods propose patterns for pixel comparison and sparsity, to increase matching accuracy and reduce resource requirements. However, these methods are bounded with respect to symmetry and/or edge length. In this paper, a Genetic algorithm (GA) is applied to find a new and powerful CT method. The proposed method, Genetic Algorithm Census Transform (GACT), is compared with the established CT methods, showing better results for benchmarking datasets. Additional experiments have been performed to study the search space and the correlation between training and evaluation data.

Adaptive autonomy (AA) is a behavior that allows agents to change their autonomy levels by reasoning on their circumstances. Previous work has modeled AA through the willingness to interact, compos ...

Adaptive autonomous (AA) agents are able to make their own decisions on when and with whom to share their autonomy based on their states.Whereas dependability gives evidence on whether a system, (e.g. an agent team), and its provided services are to be trusted.In this paper, an initial analysis on AA agents with respect to dependability is conducted.Firstly, AA is modeled through a pairwise relationship called willingness of agents to interact, i.e. to ask for and give assistance.Secondly, dependability is evaluated by considering solely the reliability attribute, which presents the continuity of correct services.The failure analysis is realized by modeling the agents through Petri Nets.Simulation results indicate that agents drop slightly more tasks when they are more willing to interact than otherwise, especially when the fail-rate of individual agents increases.Conclusively, the willingness should be tweaked such that there is compromise between performance and helpfulness.

Adaptive autonomy enables agents operating in an environment to change, or adapt, their autonomy levels by relying on tasks executed by others. Moreover, tasks could be delegated between agents, an ...

Fault tolerance has become more and more important in the development of autonomous systems with the aim to help the system to recover its normal activities even when some failures happen. Yet, one ...

This paper proposes a segmentation-based approach for matching of high-resolution stereo images in real time. The approach employs direct region matching in a raster scan fashion influenced by scanline approaches, but with pixel decoupling. To enable real-time performance it is implemented as a heterogeneous system of an FPGA and a sequential processor. Additionally, the approach is designed for low resource usage in order to qualify as part of unified image processing in an embedded system.

This paper presents an embedded vision system based on reconfigurable hardware (FPGA) and two CMOS cameras to perform stereo image processing and 3D mapping for autonomous navigation. We propose an EKF based visual SLAM and sparse feature detectors to achieve 6D localization of the vehicle in non flat scenarios. The system can operate regardless of the odometry information from the vehicle since visual odometry is used. As a result, the final system is compact and easy to install and configure.