The rapid expansion of outdoor traffic camera systems requires efficient methods to accurately estimate the geolocation of objects within their scenes. We present an innovative and scalable framework that completely automates this process by combining easy-to-build 3D world modeling with real-world traffic camera imagery. First, using the Cesium plugin for Unreal Engine, we create detailed and scalable 3D representations of urban environments, leveraging publicly available, highly accurate 3D data. This results in the creation of globally curated 3D content, including terrain, imagery, and photogrammetry. The real-world traffic camera imagery is then matched within our model using state-of-the-art feature matching techniques. By estimating the homography between synthetic images from the 3D model and the real images from traffic cameras, we accurately determine the geolocation of observed objects within the scene. This approach not only enhances geolocation accuracy but also enables seamless scalability across diverse urban settings and camera deployments worldwide. Our method significantly reduces the manual effort required for traffic camera calibration, thus streamlining the deployment of intelligent transportation systems at scale. We demonstrate the high performance of our approach by experimenting in an urban trial site with multiple smart city cameras and publicly available cameras around the world. Additionally, we highlight the adaptability of our framework for a wide range of computer vision-based traffic analytics applications, including its potential for drone-based localization.

In the autonomous vehicles field, localization is a crucial aspect. While the ORB-SLAM algorithm is a recognized solution for these tasks, it poses challenges due to its computational intensity. Although accelerated implementation exists, a bottleneck persists in the Point Filtering phase which relies on the Distribute Octree algorithm that is not suitable for GPU processing. In this paper, we introduce a novel GPU-suitable algorithm designed to enhance the Point Filtering step, surpassing Distribute Octree. We conducted a comprehensive comparison with state-of-the-art CPU and GPU implementations, considering both computational time and trajectory accuracy. Our experimental results, demonstrate significant speed-ups up to 3x compared to previous contributions.

Autonomous vehicles are increasingly utilized in safety-critical and time-sensitive settings like urban environments and competitive racing. Planning maneuvers ahead is pivotal in these scenarios, where the onboard compute platform determines the vehicle’s future actions. This paper introduces an optimized implementation of the Frenet Path Planner, a renowned path planning algorithm, accelerated through GPU processing. Unlike existing methods, our approach expedites the entire algorithm, encompassing path generation and collision avoidance. We gauge the execution time of our implementation, showcasing significant enhancements over the CPU baseline (up to 22x of speedup). Furthermore, we assess the influence of different precision types (double, float, half) on trajectory accuracy, probing the balance between completion speed and computational precision. Moreover, we analyzed the impact on the execution time caused by the use of Nvidia Unified Memory and by the interference caused by other processes running on the same system. We also evaluate our implementation using the F1tenth simulator and in a real race scenario. The results position our implementation as a strong candidate for the new state-of-the-art implementation for the Frenet Path Planner algorithm.

Nowadays, heterogeneous embedded platforms are extensively used in various low-latency applications, including the automotive industry, real-time IoT systems, and automated factories. These platforms utilize specific components, such as CPUs, GPUs, and neural network accelerators for efficient task processing and to solve specific problems with a lower power consumption compared to more traditional systems. However, since these accelerators share resources such as the global memory, it is crucial to understand how workloads behave under high computational loads to determine how parallel computational engines on modern platforms can interfere and adversely affect the system's predictability and performance. One area that remains unclear is the interference effect on shared memory resources between the CPU and GPU: more specifically, the latency degradation experienced by GPU kernels when memory-intensive CPU applications run concurrently. In this work, we first analyze the metrics that characterize the behavior of different kernels under various board conditions caused by CPU memory-intensive workloads on a Nvidia Jetson Xavier. Then, we exploit various machine learning methodologies aiming to estimate the latency degradation of kernels based on their metrics. As a result of this, we are able to identify the metrics that could potentially have the most significant impact when predicting the kernels completion latency degradation.

Reducing the execution time of ORB-SLAM algorithm is a crucial aspect of autonomous vehicles since it is computationally intensive for embedded boards. We propose a parallel GPU-based implementation, able to run on embedded boards, of the Tracking part of the ORB-SLAM2/3 algorithm. Our implementation is not simply a GPU port of the tracking phase. Instead, we propose a novel method to accelerate image Pyramid construction on GPUs. Comparison against state-of-the-art CPU and GPU implementations, considering both computational time and trajectory errors shows improvement on execution time in well-known datasets, such as KITTI and EuRoC.

The widespread deployment of NVIDIA GPUs in latency-sensitive systems today requires predictable GPU multi-tasking, which cannot be trivially achieved. The NVIDIA CUDA API allows programmers to easily exploit the processing power provided by these massively parallel accelerators and is one of the major reasons behind their ubiquity. However, NVIDIA GPUs and the CUDA programming model favor throughput instead of latency and timing predictability. Hence, providing real-time and quality-of-service (QoS) properties to GPU applications presents an interesting research challenge. Such a challenge is paramount when considering simultaneous multikernel (SMK) scenarios, wherein kernels are executed concurrently within each streaming multiprocessor (SM). In this work, we explore QoS-based fine-grained multitasking in SMK via job arbitration at the lowest level of the GPU scheduling hierarchy, i.e., between warps. We present QoS-aware warp scheduling (QAWS) and evaluate it against state-of-the-art, kernel-agnostic policies seen in NVIDIA hardware today. Since the NVIDIA ecosystem lacks a mechanism to specify and enforce kernel priority at the warp granularity, we implement and evaluate our proposed warp scheduling policy on GPGPU-Sim. QAWS not only improves the response time of the higher priority tasks but also has comparable or better throughput than the state-of-the-art policies.

Real-time and latency sensitive applications such as autonomous driving, feature an increasing need of computational power that traditional multi-core platforms can not provide. For this purpose, many heterogeneous embedded platforms have been released recently. They offer a set of diverse processing elements (e.g. GPUs, DSPs, ASICs, etc...) in order to manage the computational demands of data hungry applications. The system engineer, therefore, can choose the fittest processing element for each specific subtask. In this context, timing constraints and related task models are of paramount importance.The HPC-DAG (Heterogeneous Parallel Directed Acyclic Graph) task model has been recently proposed to capture real-time workload execution on modern heterogeneous platforms. It expresses the Instruction Set Architecture (ISA) heterogeneity across the different compute accelerators, but also their differences in terms of possible scheduling policies such as preemption.In this paper, we propose a time-table scheduling approach to allocate and schedule a set of HPC-DAG tasks onto a set of heterogeneous cores, by the mean of Integer Linear Programming (ILP). Our design allows the system engineer to handle heterogeneity of resources, of on-line execution costs, and of a part of the tasks and sub-tasks allocation to cores. It improves the solving time compared to the state of the art by gradually exploring the design space.

Several Application Programming Interfaces (APIs) and frameworks have been proposed to simplify the development of General-Purpose GPU (GPGPU) applications. GPGPU application development typically involves specific customization for the target operating systems and hardware devices. The effort to port applications from one API to the other (or to develop multi-target applications) is complicated by the availability of a plethora of specifications, which in essence offers very similar underlying functionality. In this work we provide an in-depth study of six state-of-the-art GPGPU APIs. From these we derive a taxonomy of the common semantics and propose a unified specification. We describe a methodology to translate this unified specification into different target APIs. This simplifies cross-platform application development and provides a clean framework for benchmarking. Our proposed unified specification is called GUST (GPGPU Unified Specification and Translation) and it captures common functionality found in compute-only APIs (e.g., CUDA and OpenCL), in the compute pipeline of traditional graphic-oriented APIs (e.g., OpenGL and Direct3D11) and in last-generation bare-metal APIs (e.g., Vulkan and Direct3D12). The proposed translation methodology solves differences between specific APIs in a transparent manner, without hiding available tuning knobs for compute kernel optimizations and fostering best programming practices in a simple manner.

Recent commercial hardware platforms for embedded real-time systems feature heterogeneous processing units and computing accelerators on the same System-on-Chip. When designing complex real-time applications for such architectures, the designer is exposed to a number of difficult choices, like deciding on which compute engine to execute a certain task, or what degree of parallelism to adopt for a given function. To help the designer exploring the wide space of design choices and tune the scheduling parameters, we propose a novel real-time application model, called HPC-DAG (Heterogeneous Parallel Condition Directed Acyclic Graph Model), specifically conceived for heterogeneous platforms. An HPC-DAG allows the system designer to specify alternative implementations of a software component for different processing engines, as well as conditional branches to modelif-then-elsestatements. We also propose a schedulability analysis for the HPC-DAG model and a set of heuristic allocation algorithms aimed at improving schedulability for latency sensitive applications. Our analysis takes into account the cost of preempting a task, which can be non-negligible on certain processors. We show the use of our approach on a realistic case study, and we demonstrate its effectiveness by comparing it with state-of-the-art algorithms previously proposed in literature.

Over the last few years, the ever-increasing use of Graphic Processing Units (GPUs) in safety-related domains has opened up many research problems in the real-time community. The closed and proprietary nature of the scheduling mechanisms deployed in NVIDIA GPUs, for instance, represents a major obstacle in deriving a proper schedulability analysis for latency-sensitive applications. Existing literature addresses these issues by either (i) providing simplified models for heterogeneous CPUGPU systems and their associated scheduling policies, or (ii) providing insights about these arbitration mechanisms obtained through reverse engineering. In this paper, we take one step further by correcting and consolidating previously published assumptions about the hierarchical scheduling policies of NVIDIA GPUs and their proprietary CUDA application programming interface. We also discuss how such mechanisms evolved with recently released GPU micro-architectures, and how such changes influence the scheduling models to be exploited by real-time system engineers.

Modern embedded platforms are known to be constrained by size, weight and power (SWaP) requirements. In such contexts, achieving the desired performance-per-watt target calls for increasing the number of processors rather than ramping up their voltage and frequency. Hence, generation after generation, modern heterogeneous System on Chips (SoC) present a higher number of cores within their CPU complexes as well as a wider variety of accelerators that leverages massively parallel compute architectures. Previous literature demonstrated that while increasing parallelism is theoretically optimal for improving on average performance, shared memory hierarchies (i.e. caches and system DRAM) act as a bottleneck by exposing the platform processors to severe contention on memory accesses, hence dramatically impacting performance and timing predictability. In this work we characterize how subsequent generations of embedded platforms from the NVIDIA Tegra family balanced the increasing parallelism of each platform's processors with the consequent higher potential on memory interference. We also present an open-source software for generating test scenarios aimed at measuring memory contention in highly heterogeneous SoCs.

The push towards automated and connected driving functionalities mandates the use of heterogeneous HW platforms in order to provide the required computational resources. For these platforms, the established methods for performance modelling in industry are no longer effective. In this paper, we propose an initial modelling concept for heterogeneous platforms which can then be fed into appropriate tools to derive effective performance predictions. The approach is demonstrated for a prototypical automated driving application on the Nvidia Tegra X2 platform.

Containerisation is becoming a cornerstone of modern distributed systems, thanks to their lightweight virtualisation, high portability, and seamless integration with orchestration tools such as Kubernetes. The usage of containers has also gained traction in real-time cyber-physical systems, such as software-defined vehicles, which are characterised by strict timing requirements to ensure safety and performance. Nevertheless, ensuring real-time execution of co-located containers is challenging because of mutual interference due to the sharing of the same processing hardware. Existing parallel computing frameworks such as Ray and its Kubernetes-enabled variant, KubeRay, excel in distributed computation but lack support for scheduling policies that allow guaranteeing real-time timing constraints and CPU resource isolation between containers, such as the SCHED_DEADLINE policy of Linux. To fill this gap, this paper extends Ray to support real-time containers that leverage SCHED_DEADLINE. To this end, we propose KubeDeadline, a novel, modular Kubernetes extension to support SCHED_DEADLINE. We evaluate our approach through extensive experiments, using synthetic workloads and a case study based on the MobileNet and EfficientNet deep neural networks. Our evaluation shows that KubeDeadline ensures deadline compliance in all synthetic workloads, adds minimal deployment overhead (in the order of milliseconds), and achieves lower worst-case response times, up to 4 times lower, than vanilla Kubernetes under background interference.

One of the main predictability bottlenecks of modern multi-core embedded systems is contention for access to shared memory resources. Partitioning and software-driven allocation of memory resources is an effective strategy to mitigate contention in the memory hierarchy. Unfortunately, however, many of the strategies adopted so far can have unforeseen side-effects when practically implemented latest-generation, high-performance embedded platforms. Predictability is further jeopardized by cache eviction policies based on random replacement, targeting average performance instead of timing determinism. In this paper, we present a framework of software-based techniques to restore memory access determinism in high-performance embedded systems. Our approach leverages OS-transparent and DMA-friendly cache coloring, in combination with an invalidation-driven allocation (IDA) technique. The proposed method allows protecting important cache blocks from (i) external eviction by tasks concurrently executing on different cores, and (ii) internal eviction by tasks running on the same core. A working implementation obtained by extending the Jailhouse partitioning hypervisor is presented and evaluated with a combination of synthetic and real benchmarks.

Modern automotive grade embedded platforms feature high performance Graphics Processing Units (GPUs) to support the massively parallel processing power needed for next-generation autonomous driving applications. Hence, a GPU scheduling approach with strong Real-Time guarantees is needed. While previous research efforts focused on reverse engineering the GPU ecosystem in order to understand and control GPU scheduling on NVIDIA platforms, we provide an in depth explanation of the NVIDIA standard approach to GPU application scheduling on a Drive PX platform. Then, we discuss how a privileged scheduling server can be used to enforce arbitrary scheduling policies in a virtualized environment.

The current trend in designing Advanced Driving Assistance System (ADAS) is to enhance their computing power by using modern multi/many core accelerators. For many critical applications such as pedestrian detection, line following, and path planning the Graphic Processing Unit (GPU) is the most popular choice for obtaining orders of magnitude increases in performance at modest power consumption. This is made possible by exploiting the general purpose nature of today's GPUs, as such devices are known to express unprecedented performance per watt on generic embarrassingly parallel workloads (as opposed of just graphical rendering, as GPUs where only designed to sustain in previous generations). In this work, we explore novel challenges that system engineers have to face in terms of real-time constraints and functional safety when the GPU is the chosen accelerator. More specifically, we investigate how much of the adopted safety standards currently applied for traditional platforms can be translated to a GPU accelerated platform used in critical scenarios.

Modern automotive-grade embedded computing platforms feature high-performance Graphics Processing Units (GPUs) to support the massively parallel processing power needed for next-generation autonomous driving applications (e.g., Deep Neural Network (DNN) inference, sensor fusion, path planning, etc). As these workload-intensive activities are pushed to higher criticality levels, there is a stronger need for more predictable scheduling algorithms that are able to guarantee predictability without overly sacrificing GPU utilization. Unfortunately, the real-rime literature on GPU scheduling mostly considered limited (or null) preemption capabilities, while previous efforts in broader domains were often based on programming models and APIs that were not designed to support the real-rime requirements of recurring workloads. In this paper, we present the design of a prototype real-time scheduler for GPU activities on an embedded System on a Chip (SoC) featuring a cutting edge GPU architecture by NVIDIA adopted in the autonomous driving domain. The scheduler runs as a software partition on top of the NVIDIA hypervisor, and it leverages latest generation architectural features, such as pixel-level preemption and threadlevel preemption. Such a design allowed us to implement and test a preemptive Earliest Deadline First (EDF) scheduler for GPU tasks providing bandwidth isolations by means of a Constant Bandwidth Server (CBS). Our work involved investigating alternative programming models for compute APIs, allowing us to characterize CPU-to-GPU command submission with more detailed scheduling information. A detailed experimental characterization is presented to show the significant schedulability improvement of recurring real-time GPU tasks.

Most of today's mixed criticality platforms feature Systems on Chip (SoC) where a multi-core CPU complex (the host) competes with an integrated Graphic Processor Unit (iGPU, the device) for accessing central memory. The multi-core host and the iGPU share the same memory controller, which has to arbitrate data access to both clients through often undisclosed or non-priority driven mechanisms. Such aspect becomes critical when the iGPU is a high performance massively parallel computing complex potentially able to saturate the available DRAM bandwidth of the considered SoC. The contribution of this paper is to qualitatively analyze and characterize the conflicts due to parallel accesses to main memory by both CPU cores and iGPU, so to motivate the need of novel paradigms for memory centric scheduling mechanisms. We analyzed different well known and commercially available platforms in order to estimate variations in throughput and latencies within various memory access patterns, both at host and device side.

As autonomous cars are entering mainstream, new research directions are opening involving several domains, from hardware design to control systems, from energy efficiency to computer vision. An exciting direction of research is represented by the coordination of the different vehicles, moving the focus from the single one to a collective system. In this paper we propose some challenging examples thatshow the motivations for a coordination approach in autonomous driving. Moreover, we present some techniques borrowed from distributed artificial intelligence that can be exploited to tackle the previously mentioned challenges.

Neuroevolution in robot controllers through objective-based genetic and evolutionary algorithms is a well-known methodology for studying the dynamics of evolution in swarms of simple robots. A robot within a swarm is able to evolve the simple neural network embedded as its controller by also taking into account how other robots are performing the task at hand. In online scenarios, this is obtained through inter-robot communications of the best performing genomes (i.e. representation of the weights of their embedded neural network). While many experiments from previous work have shown the soundness of this approach, we aim to extend this methodology using a novelty-based metric, so to be able to analyze different genome exchange strategies within a simulated swarm of robots in deceptive tasks or scenarios in which it is difficult to model a proper objective function to drive evolution. In particular, we want to study how different information sharing approaches affect the evolution. To do so we developed and tested three different ways to exchange genomes and information between robots using novelty driven evolution and we compared them using a recent variation of the mEDEA (minimal Environment-driven Distributed Evolutionary Algorithm) algorithm. As the deceptiveness and the complexity of the task increases, our proposed novelty-driven strategies display better performance in foraging scenarios.

Distributed autonomous systems consisting of large numbers of components with no central control point need to be able to dynamically adapt their control mechanisms to deal with an unpredictable and changing environment. Existing frameworks for engineering self-adaptive systems fail to account for the need to incorporate self-expression—that is, the capability of a system to dynamically adapt its coordination pattern during runtime. Although the benefits of incorporating self-expression are well known, currently there is no principled means of enabling this during system design. We propose a conceptual framework for principled design of systems that exhibit self-expression, based on inspiration from the natural immune system. The framework is described as a set of design principles and customizable algorithms and then is instantiated in three case studies, including two from robotics and one from artificial chemistry. We show that it enables self-expression in each case, resulting in systems that are able to adapt their choice of coordination pattern during runtime to optimize functional and nonfunctional goals, as well as to discover novel patterns and architectures.

Summary Multiagent systems are commonly used for simulation of new paradigms of energy distribution. Especially when considering Smart Grids, the autonomicity deployed by goal‐driven agents implies the need for being aware of multiple aspects connected to the energy distribution context. With ‘context’, we refer to the outside world variables such as weather, stock market trends, location of the users, government actions, and so on; therefore, an architecture highly context‐aware is needed. We propose a model in which every important factor concerning the electric energy distribution is presented by modeling context‐aware agents able to identify the impact of these factors. Moreover, some tests have been performed regarding the web service integration in which agents contracting energy will automatically retrieve data to be used in adaptive and collaborative aspects; an explicative example is represented by the retrieval of weather forecasting that provides input on ongoing demand and data for the predicted availability (in case of photovoltaic or wind powered environments). Copyright © 2013 John Wiley & Sons, Ltd.

We propose a novel method for evolving adaptive locomotive strategies for virtual limbless creatures that addresses both functional and non-functional requirements, respectively the ability to avoid obstacles and to minimise spent energy. We describe an approach inspired by artificial immune systems, based on a dual-layer idiotypic network that results in a completely decentralised controller. Starting from a system initialised with five non-adaptive locomotion strategies, we show that an adaptive controller can evolve that both min- imises energy requirements and maximises distance covered when compared to the initial strategies.

Given the complexity of modelling actors and interactions of the deregulated electric energy market, the Multi-Agent System approach can be used for both simulation and applications of critical aspects in the Smart Grid. In particular, balancing demand and offer and handling negotiation among peers: now, even a domestic environment that features photovoltaic and/or wind turbines modules can decide to enter the deregulated market as a small-scale seller, thus making the requirement of having such an architecture to be autonomous by deploying Self-* properties such as Self-Organization, Self-Repairing, Self-Adaptation. To be more specific about the presented case study, we propose a model in which small-scale seller agents dynamically decide from to time to time, either to address the market as lone operators or by aggregating into Virtual Power Plants. This iterated decisional process depends on highly variable market related factors, thus the goal to design a net of agents able to autonomously react to this dynamic environment.

Private houses are more and more enabled with devices that can produce renewable energy, and the not so remote chance of selling the surplus energy makes them new players in the energy market. This market is likely to become deregulated since each energy home-producer can negotiate the energy price with consumers, typically by means of an auction; on the other hand, consumers can always rely on energy companies, even if their energy is more expensive. This scenario could lead to advantages for users, but it is certainly complex and dynamic, and needs an appropriate management. To this purpose, in this paper we propose an agent-based application to deal with the negotiation among different parties producing and consuming energy. Software agents, thanks to their autonomy in taking decisions, well suit the requirements of the proposed scenario. For our application, we adopt a strategy derived from game theory, in order to optimize energy production and supply costs by means of negotiation and learning. The effectiveness of our approach is proved by simulation results of a situation involving energy buyers, energy producers using renewable micro-generation facilities and large-scale traditional electricity companies.