This paper demonstrates ELENNA, an end-to-end latency-aware energy optimization technique to dynamically partition deep neural networks (DNNs) between an IoT device and an edge server. ELENNA employs ML predictors of (i) per-layer compute latency on device/server, (ii) transmission time as a function of intermediate tensor size and network state, and (iii) on-device energy, to determine the partition point that satisfies an application service-quality level (SQL) while minimizing energy consumption. ELENNA is tested in a latency-critical Edge-AI application in the automotive domain. The results show that ELENNA adaptively partitions under time-varying edge server load and network conditions, reducing on-device power draw while keeping end-to-end latency within SQL in edge AI applications.

Mobile devices' processing power and battery life are being strained by the increasing need for real-time computing applications. These tasks can be offloaded to nearby servers with greater processing capability thanks to mobile edge computing. However, choosing the optimal number of tasks to be offloaded to save energy while optimizing the end-to-end latency becomes challenging. This is because we have to make a compromise between minimising energy consumption and optimizing the end-to-end latency. This paper proposes an energy-efficient Deep Neural Network Dividing Technique (EDNNDiT) for latency optimisation in dynamic mobile edge networks. EDNNDiT utilizes external machine-learning models to predict the latency and energy consumption of each DNN layer. These external machine learning models gather real data metrics from the IoT device, edge server, and network components to predict the end-to-end latency and energy consumption for each layer of the DNN model. Next, we use a straightforward heuristic to identify the candidate dividing points of the DNN model that meet the required Quality Service Level (QSL) of the end-to-end latency. Finally, EDNNDiT selects the candidate dividing point with the minimum energy consumption to save power on the IoT device while satisfying the QSL of the end-to-end latency. We conducted real experiments using the NVIDIA JetRacer Robot AI car and Se-QaM platform with both WiFi and 5G that were deployed in our lab under various load conditions on the edge server device. The experimental results for both sequential and non-sequential DNN architectures demonstrate that our EDNNDiT approach reduces energy consumption on the NVIDIA JetRacer while maintaining an acceptable QSL for end-to-end latency. EDNNDiT dynamically adapts to various network conditions and server requirements, showing a powerful and energy-efficient solution for deploying DNNs on limited-resource devices with enhanced overall performance.

Effective end-to-end service quality management is critical for successfully adopting edge computing. However, existing solutions lack the necessary integration of key characteristics to identify and provide actionable insights for resolving the root causes of service quality issues. To address this gap, this paper introduces a service quality manager (SeQaM) to improve service quality in edge computing. SeQaM includes distributed observability, adaptive data collection, real-time analytics, rapid feedback mechanisms, and the ability to create controlled events and experimental scenarios. These features are of utmost importance for infrastructure and service providers, application developers, and researchers to implement, test, validate, and benchmark solutions focused on service quality. To accomplish this, SeQaM is composed of distributed and central components. The distributed components are responsible for collecting service quality metrics and implementing feedback mechanisms in edge applications, user devices, network devices, and edge servers. The central components aggregate the collected metrics, perform holistic analysis, and plan corrective actions to enhance service quality. Moreover, SeQaM can be seamlessly deployed across diverse environments, including emulated testbeds, laboratory settings, and real-world infrastructures. Finally, the effectiveness of SeQaM is demonstrated through three use cases, highlighting its capability to provide detailed insights on the causes of service quality issues, generate data for model training, and support data-driven decision-making.

Dynamic batching in machine learning (ML) serving systems can significantly improve throughput, yet it also introduces a non-trivial trade-off between inference latency and system energy consumption. This paper presents Energy-Latency Trade-off Optimization (ELTO) for ML Inference with Dynamic Batching. ELTO empirically profiles the latency and energy characteristics of any newly deployed model under varying batch sizes and request rates using NVIDIA Triton Server. It leverages supervised regression to predict per-batch latency and energy based on profiled data. ELTO formulates and solves a cost-based optimization to select the batch size that minimizes a weighted sum of normalized predicted latency and energy for deployed ML models. Experimental evaluations using ML vision models (ResNet18, ResNet50) on the NVIDIA GeForce RTX 3060Ti demonstrate the effectiveness of the proposed ELTO. ELTO is compared to heuristic baselines like fixed small or large batch sizes, ELTO significantly reduces the average scaled operational cost-a balanced measure of both latency and energy. For instance, evaluations show cost reductions of $\mathbf{7 5. 5 \%}$ for ResNet18 and $48.2 \%$ for ResNet50 relative to a no-batching strategy, thereby ensuring a more consistently near-optimal operational balance under diverse load conditions

Effective service quality management is essential for leveraging the advantages of edge computing, which prioritizes localized data processing to minimize latency and improve efficiency. This paper presents an End-to-End Edge Service Quality Manager, a platform designed to ensure efficient monitoring, analysis, planning, and timely execution of data-driven decision-making to guarantee service quality in edge computing. Furthermore, the platform may be seamlessly deployed and utilized in various deployment scenarios, including emulated, laboratory, and real-world environments, supporting the entire life cycle of an edge application, from development through experimentation, testing, and runtime.

The paper analyzes the use of vanishing points for steering a robot. Parallel lines in the environment of the robot are used to compute vanishing points which serve as a reference for guiding the robot. To accomplish the steering task, three subtasks are performed: detection of straight lines, computation of vanishing points, and robot steering using vanishing points. Straight lines are detected by employing a high precision edge detector and a line-fitting algorithm. The cross product method introduced by Magee and Aggarwal (1984) is modified to make the detection of vanishing points appropriate for an indoor environment. Properties of vanishing points under camera rotation and translation are derived. Using these properties, the location of the vanishing points can serve as a reference for steering the robot. A model of the robot environment is defined, summarizing the minimum number of constraints necessary for the method to work. Finally, the limitations as well as the advantages of using vanishing points in robot navigation are discussed.>