Real-world applications requiring real-time responsiveness frequently rely on energy-intensive and compute-heavy neural network algorithms. Strategies include deploying distributed and optimized Deep Neural Networks on mobile devices, which can lead to considerable energy consumption and degraded performance, or offloading larger models to edge servers, which requires low-latency wireless channels. Here we present Furcifer, a novel middleware that autonomously adjusts the computing strategy (i.e., local computing, edge computing, or split computing) based on context conditions. Utilizing container-based services and low-complexity predictors that generalize across environments, Furcifer supports supervised compression as a viable alternative to pure local or remote processing in real-time environments. An extensive set of experiments coversdiverse scenarios, including both stable and highly dynamic channel environments with unpredictable changes in connection quality and load. In moderate-varying scenarios, Furcifer demonstrates significant benefits: achieving a 2x reduction in energy consumption, a 30% higher mean Average Precision score compared to local computing, and a three-fold FPS increase over static offloading. In highly dynamic environments with unreliable connectivity and rapid increases in concurrent clients, Furcifer’s predictive capabilities preserves up to 30% energy, achieving a 16% higher accuracy rate, and completing 80% more frame inferences compared to pure local computing and approaches without trend forecasting, respectively. • Adaptive Split Computing: an efficient strategy for real-time vision applications. • A Middleware for automated, dynamic, and resilient flexible computing. • A Low-Complexity Manager for efficient power, higher FPS rate, and enhanced accuracy.

Next-generation mobile systems will support multiple AI-based applications, each leveraging heterogeneous sensors and data sources through deep neural network (DNN) architectures collaboratively executed within the network. In this context, to minimize the cost of the AI inference task subject to requirements on latency, quality, and – crucially –reliabilityof the inference process, it is vital to optimize (i) the set of sensors/data sources and (ii) the DNN architecture, (iii) the network nodes executing sections of the DNN, and (iv) the resources to use. To achieve these goals, we leverage dynamic gated neural networks with branches, and propose a novel algorithmic strategy called Quantile-constrained Inference (QIC), based upon quantile-Constrained policy optimization. QIC makes joint, high-quality, swift decisions on all the above aspects of the system, with the aim to minimize inference energy cost. We remark that this is the first contribution connecting gated dynamic DNNs with infrastructure-level decision making. We evaluate QIC using a dynamic gated DNN with stems and branches for optimal sensor fusion and inference, trained on the RADIATE dataset offering Radar, LiDAR, and Camera data, and real-world wireless measurements. Our results confirm that QIC closely matches the optimum and outperforms existing approaches in reducing energy consumption (compute, communication, and total) and application requirements failure by over 70%.

Modern real-time applications widely embed compute intense neural algorithms at their core. Current solutions to support such algorithms either deploy highly-optimized Deep Neural Networks at mobile devices or offload the execution of possibly larger higher-performance neural models to edge servers. While the former solution typically maps to higher energy consumption and lower performance, the latter necessitates the low-latency wireless transfer of high volumes of data. Time-varying variables describing the state of these systems, such as connection quality and system load, determine the optimality of the different computing configurations in terms of energy consumption, task performance, and latency. Herein, we propose Furcifer, a framework capable of dynamically adapting the cloud continuum computing configuration in response to the perceived state of the system. Our container-based approach incorporates low-complexity predictors that generalize well across operating environments. In addition, we develop a highly optimized split Deep Neural Network model, which achieves in-model supervised compression and enhances task offloading. Experimental results for object detection across diverse conditions, environments, and wireless technologies, show Furcifer's remarkable outcomes, including a 2x energy reduction, 30% higher mean Average Precision score than pure local computing, and a notable three-fold increase in frame per second rate compared to static offloading.

Machine learning is becoming an increasingly integral component of mobile applications. However, the execution of compute-heavy neural models (e.g., for computer vision tasks) on resource-constrained devices is challenging due to their limited computing power, memory, and energy reservoir. While edge computing mitigates these issues, the transfer of information-rich signals over capacity-limited and time-varying wireless channels may result in large latency and latency variations. Herein, we propose a methodology to route heterogeneous tasks across the resources and layers of systems composed of mobile devices and edge servers. Different from prior work, we consider aspects of real-world systems, such as context switching, task accumulation, and the interplay between communications and computing components of the overall pipeline, that are rarely captured in abstract models. To optimize the task flow, we use a deep reinforcement learning agent trained on real-world data collected using a system we developed. The agent uses an articulate definition of state drawing features from several logical blocks of the system. Results indicate that the agent adapts the routing of tasks to parameters controlling their heterogeneity, as well as the hardware setup and the state of the wireless channel.

LiDAR is rapidly emerging as a central sensor in many applications involving autonomous navigation. However, the execution of state-of-the-art neural models for the analysis of point clouds produced by LiDARs necessitates considerable computing power, energy and memory. As a consequence, real-time analysis – e.g., 3D object detection – on resource-constrained mobile platforms such as Unmanned Aerial Vehicles (UAV) is often impractical. In this paper, we evaluate the feasibility of real-time LiDAR-based object detection for UAVs using measures and data obtained from a real-world deployment. First, we demonstrate that state-of-the-art neural models for 3D object detection cannot be executed even in relatively powerful embedded computers suitable for airborne drones, such as the NVIDIA Jetson Nano. Then, we focus our attention on edge computing, where the UAV offloads the execution of the analysis model to a compute-capable device (an edge server) positioned at the network edge. The key challenge is that point clouds generated by LiDARs have a large size (1.2MB per point cloud frame). We evaluate the overall capture-to-output delay of a remote analysis loop experimentally for WiFi and using expected data rate for cellular LTE environments. Finally, we evaluate the performance of 3D object detection on available datasets for autonomous vehicles and emphasize the challenges posed by the ability of UAVs to move in the 3D space. With our LiDAR-UAV system, we achieved detections with averages of 86% accuracy, 71.6% precision, and 55.14% recall outputted with an average end-to-end delay of 1920ms.