On-device machine learning applications are increasingly deployed in dynamic and open system environments, where resource availability fluctuates unpredictably. This variability, coupled with limited computing resources, poses significant challenges in achieving high responsiveness. Existing on-device machine learning frameworks typically rely on static and coarse-grained resource allocation, leading to performance degradation under resource contention. To address this, we propose FlexOn, a novel framework that combines fine-grained model segmentation and dynamic resource selection to rapidly adapt to highly dynamic runtime conditions and effectively mitigate unpredictable resource contention. A prototype built on LiteRT demonstrates significant improvements in both average and tail latencies of up to 54% and 58%, respectively, across three different embedded platforms under dynamically varying resource availability. To the best of our knowledge, this is the first work that addresses the resource contention in open embedded systems for better machine learning inference responsiveness.

The Robot Operating System 2 (ROS 2) is a popular middleware for distributed robotic applications. However, achieving real-time guarantees in ROS 2 is challenging due to unpredictable delays and priority inversions. We reveal that these issues arise from the lack of consistent priority propagation across ROS 2's multi-layered communication architecture, particularly down to the kernel layer. To address this, we present CROS-RT, the first cross-layer scheduler explicitly designed to tackle the unpredictability in ROS 2 inter-process communication caused by multi-layer priority misalignment. CROS-RT ensures consistent, priority-based scheduling across the application, middleware, and kernel layers, introducing mechanisms for priority propagation, kernel-level message prioritization, and dynamic kernel thread adjustment. We have implemented and evaluated CROS-RT on the current stable release of ROS 2. Experiments demonstrate that CROS-RT enhances communication predictability, reducing the worst-case response time by up to 89.3 % over a baseline (vanilla ROS 2). Additionally, we provide an analytical model to derive upper bounds on response times, ensuring reliable realtime performance for safety-critical applications.

Even with advances in single-cell batteries, mobile users still experience low battery anxiety. By analyzing 19,855 hours of user behavior, we proposeMixMax, a heterogeneous battery system consisting of three complementary battery types tailored to minimizing low battery time. While the heterogeneous battery system offers an opportunity to simultaneously improve capacity and charging speed, one must face non-trivial challenges to design charge/discharge policies during runtime and determine the ratio of enclosed batteries. They are highly dependent on each other, which entails almost infinite candidates for the choice.MixMaxsimplifies this by reformulating the problem as an optimization problem, breaking it down into manageable sub-problems. However,MixMaxstill faces the challenge of catering to all users due to their diverse battery usage patterns. To address this, we introduce a customizedMixMaxthat groups users based on their usage patterns and provides tailored battery solutions. In evaluatingMixMax, we fabricate coin-cell batteries, develop a precise battery emulator using the fabricated batteries, and prototypeMixMaxon a real-world smartphone. Our evaluation shows thatMixMaxreduces low battery time by up to 24.6% without compromising capacity, volume, weight, or user behavior, and its customized version can further reduce it by up to 46.2%.

The increasing complexity and memory demands of Deep Neural Networks (DNNs) for real-time systems pose new significant challenges, one of which is the GPU memory capacity bottleneck, where the limited physical memory inside GPUs impedes the deployment of sophisticated DNN models. This paper presents, to the best of our knowledge, the first study of addressing the GPU memory bottleneck issues, while simultaneously ensuring the timely inference of multiple DNN tasks. We propose RT-Swap, a real-time memory management framework, that enables transparent and efficient swap scheduling of memory objects, employing the relatively larger CPU memory to extend the available GPU memory capacity, without compromising timing guarantees. We have implemented RT-Swap on top of representative machine-learning frameworks, demonstrating its effectiveness in making significantly more DNN task sets schedulable at least 72% over existing approaches even when the task sets demand up to 96.2% more memory than the GPU's physical capacity.

The growing trend of multi-device ownerships creates opportunities to use applications across devices. However, the current methods of app development/usage remain in the single-device paradigm, which is far below user expectations. For example, it is currently impossible for users to dynamically partition an existing app across different devices to utilize multiple surfaces. We introduce FLUID, a novel multi-device platform that supports simultaneous operation of multiple devices. FLUID aims toi)distribute the user interfaces (UIs) of a single app across multiple devices,ii)support unmodified legacy apps without extra engineering, andiii)support numerous apps with customized UIs. Previous approaches, like screen mirroring and app migration, do not satisfy those goals altogether. However, FLUID is designed to satisfy the goals. It can efficiently deploy UI objects to different devices by identifying only UI states necessary for accurate rendering. And FLUID can execute the distributed UI objects by supporting cross-device method invocations transparently and synchronizing the replicated UIs across devices. Furthermore, FLUID automatically handles unexpected events that may degrade its usability by efficiently maintaining the distributed UIs up to date. Our evaluation using 20 legacy apps shows that FLUID can transparently support numerous apps and is fast enough for interactive use.

Deep neural networks (DNNs) have been deployed in many safety-critical real-time embedded systems. To support DNN tasks in real-time, most previous studies focused on GPU or CPU. However, Edge TPU has not yet been studied for real-time guarantees. This paper presents a real-time DNNs framework for Edge TPU to satisfy multiple DNN inference tasks’ timing requirements. The proposed framework provides 1) SRAM allocation and model partitioning techniques and 2) a MIP-based algorithm that determines the amount of SRAM and the number of segments for each task. The experiment result shows that our framework provides 79% higher schedulability than the existing Edge TPU system.

Despite the physical advance of an existing single-cell battery system, mobile users are still suffering from low battery anxiety. With a careful analysis of users' battery usage behavior collected for 19,855 hours, we propose a heterogeneous battery system, MixMax, consisting of three complementary battery types tailored to minimizing the low battery time. While composing a heterogeneous battery system opens up a chance to simultaneously improve the capacity and the charging speed, one must face non-trivial challenges to determine the ratio of enclosed batteries and charge/discharge policies during the run-time. They are highly dependent on each other, which entails almost infinite candidates for the choice. MixMax gracefully unwinds the dependencies as it formulates the decision-making problem into an optimization problem and decomposes it into multiple sub-problems instead. To evaluate MixMax, we fabricate coin-cell batteries and experiment with them to model an accurate battery emulator which sophisticatedly reproduces the dynamics of battery systems. Our experimental results demonstrate that MixMax can reduce the low battery time by up to 24.6% without compromising capacity, volume, weight, and more importantly, users' battery usage behavior. In addition, we prototype MixMax on a smartphone, presenting the practicality of MixMax on mobile systems.

Multi-access edge computing (MEC) providing server capabilities at near end-users is introduced to enable Ultra Reliable Low Latency Communication (URLLC) for mission-critical and time-sensitive networked services. However, the current MEC simply shortens the physical travel distance of traffic but does not include any architectural approach for supporting URLLC. As a result, MEC implicates resource contention issues, and important packets can be easily delayed or lost, resulting in critical flaws for those services. To address these problems, we introduce MECaNIC, which extends the data plane of MEC to SmartNIC and assists URLLC of MEC. It provides i) precise packet scheduling that handles traffic priorities into two dimensions of reliability and latency, and ii) task offloading that accelerates MEC applications, including payload matching and response caching. The prototype implemented using NetFPGA shows that MECaNIC reduces the average latency of the high-priority traffic from$2,883\ \mu s$to$397\ \mu s$while ensuring packet delivery, even when the traffic competes with other lower priority traffic. Also, task offloading improves a MEC's payload processing 4-fold and reduces file downloading time and video random access time by 44% and 17%, respectively.

As real-time object detection systems, such as autonomous cars, need to process input images acquired from multiple cameras, they face significant challenges in delivering accurate and timely inferences often based on machine learning (ML). To meet these challenges, we want to provide different levels of object detection accuracy and timeliness to different portions within each input image with different criticality levels. Specifically, we develop DNN-SAM, a dynamic Split-And-Merge Deep Neural Network (DNN) execution and scheduling framework, that enables seamless split-and-merge DNN execution for unmodified DNN models. Instead of processing an entire input image once in a full DNN model, DNN-SAM first splits a DNN inference task into two smaller sub-tasks-a mandatory sub-task dedicated for a safety-critical (cropped) portion of each image and an optional sub-task for processing a down-scaled image–then executes them independently, and finally merges their results into a complete inference. To achieve DNN-SAM’s timely and accurate detection of objects in each image, we also develop two scheduling algorithms that prioritize sub-tasks according to their criticality levels and adaptively adjust the scale of the input image to meet the timing constraints while minimizing the response time of mandatory sub-tasks or maximizing the accuracy of optional sub-tasks. We have implemented and evaluated DNN-SAM on a representative ML framework. Our evaluation shows DNN-SAM to improve detection accuracy in the safety-critical region by $2.0-3.7\times$ and lower average inference latency by $4.8-9.7\times$ over existing approaches without violating any timing constraints.

Deep neural networks (DNNs) have shown remarkable success in various machine-learning (ML) tasks useful for many safety-critical, real-time embedded systems. The foremost design goal for enabling DNN execution on real-time embedded systems is to provide worst-case timing guarantees with limited computing resources. Yet, the state-of-the-art ML frameworks hardly leverage heterogeneous computing resources (i.e., CPU, GPU) to improve the schedulability of real-time DNN tasks due to several factors, which include a coarse-grained resource allocation model (one-resource-per-task), the asymmetric nature of DNN execution on CPU and GPU, and lack of schedulability-aware CPU/GPU allocation scheme. This paper presents, to the best of our knowledge, the first study of addressing the above three major barriers and examining their cooperative effect on schedulability improvement. In this paper, we propose LaLaRAND, a real-time layer-level DNN scheduling framework, that enables flexible CPU/GPU scheduling of individual DNN layers by tightly coupling CPU-friendly quantization with fine-grained CPU/GPU allocation schemes (one-resource-per-layer) while mitigating accuracy loss without compromising timing guarantees. We have implemented and evaluated LaLaRAND on top of the state-of-the-art ML framework to demonstrate its effectiveness in making more DNN task sets schedulable by 56% and 80% over an existing approach and a baseline (vanilla PyTorch), respectively, with only up to -0.4% of performance (inference accuracy) difference.

The growing trend of multi-device ownerships creates a need and an opportunity to use applications across multiple devices. However, in general, the current app development and usage still remain within the single-device paradigm, falling far short of user expectations. For example, it is currently not possible for a user to dynamically partition an existing live streaming app with chatting capabilities across different devices, such that she watches her favorite broadcast on her smart TV while real-time chatting on her smartphone. In this paper, we present FLUID, a new Android-based multi-device platform that enables innovative ways of using multiple devices. FLUID aims to i) allow users to migrate or replicate individual user interfaces (UIs) of a single app on multiple devices (high flexibility), ii) require no additional development effort to support unmodified, legacy applications (ease of development), and iii) support a wide range of apps that follow the trend of using custom-made UIs (wide applicability). Previous approaches, such as screen mirroring, app migration, and customized apps utilizing multiple devices, do not satisfy those goals altogether. FLUID, on the other hand, meets the goals by carefully analyzing which UI states are necessary to correctly render UI objects, deploying only those states on different devices, supporting cross-device function calls transparently, and synchronizing the UI states of replicated UI objects across multiple devices. Our evaluation with 20 unmodified, real-world Android apps shows that FLUID can transparently support a wide range of apps and is fast enough for interactive use.

The growing trend of multi-device ownerships creates a need and an opportunity to use applications across multiple devices. However, in general, the current app development and usage still remain within the single-device paradigm, falling far short of user expectations. We present FLUID, a new multi-device platform that allows users to migrate or replicate individual user interfaces (UIs) of a single app on multiple devices. In addition, FLUID aims to require no extra development effort to support a wide range of legacy apps that follow the trend of using custom-made UIs. To this end, FLUID analyzes which UI states are necessary to correctly render UI objects, deploys only those states on different devices, and supports cross-device function calls transparently. In this demo, we demonstrate several interesting use cases supported by our Android-based FLUID prototype.

The development of complex cyber-physical systems necessitates real-time networking with timing guarantees even in the presence of a link fault. Targeting firm real-time flows with the maximum allowable number of continuous deadline misses, this paper introduces FR-SDN, a fault-resilient SDN (Software-Defined Networking) framework that satisfies the timing requirements of firm real-time flows. To this end, we first investigate individual steps for path restoration: fault recognition, path recalculation, and path reassignment. We then design novel system architecture that reduces the delay of the fault recognition and path reassignment steps to potentially assign more time budget to the path recalculation step. Based on the calculation of tight upper-bounds on the delays in individual steps under the proposed system design, we derive a necessary feasibility condition that guarantees the timing requirements of firm real-time flows, and we calculate a time budget for the path recalculation step. Finally, we develop a multi-constrained path finding algorithm that can dynamically adjust the scope of flows to reroute according to the time budget. To the best of our knowledge, FR-SDN is the first study on adaptive path restoration for real-time flows, taking into account path restoration delay and fault tolerance constraints in case of link fault. We have implemented and evaluated FR-SDN on top of Open vSwitch to demonstrate its effectiveness, achieving an order of magnitude reduction in path restoration delay. In addition, we have deployed FR-SDN into a 1/10 scale autonomous vehicle and have shown, via an in-depth case study of adaptive cruise control, that FR-SDN is able to meet all fault tolerance requirements so that it can behave similarly as if there were no link failure.

Battery aging is one of the critical issues in battery-powered electric systems. However, this issue has not received much attention in the real-time systems community. In this paper, we present the first attempt to translate the problem of minimizing battery aging subject to timing requirements into a real-time scheduling problem, addressing the following issues. (i) Can scheduling make a systematic impact on battery aging? If so, which scheduling principles are favorable to minimizing battery aging? (ii) If there exists any, how can we build upon the scheduling principle to guarantee real-time requirements? For (i), we first illuminate the connection between task scheduling and battery aging minimization and then derive a principle for task scheduling from abstracting the complicated dynamics of battery aging, which is to minimize the variance of total power consumption over time. In addition, we implement a battery aging simulator and use it to verify the effectiveness of the proposed principle in minimizing battery aging and its impact on quantitative improvement. For (ii), we propose a scheduling framework that separates control for timing guarantees from that for battery aging minimization. Such a separation allows reducing the complexity significantly such that we can employ existing scheduling algorithm and schedulability analysis for real-time guarantee and tailor the proposed scheduling principle to decelerate battery aging without taking real-time guarantees into accounts. Our simulation results show that the proposed framework can extend the battery lifespan by up to 144.4%.

These days, the term of Internet of Things (IoT) becomes popular to interact and cooperate with individual smart objects, and one of the most critical challenges for IoT is to achieve efficient resource sharing as well as ensure safety-stringent timing constraints. To design such reliable real-time IoT, this paper focuses on the concept of mixed-criticality (MC) introduced to address the low processor utilization on traditional real-time systems. Although different worst-case execution time estimates depending on criticality are proven effective on processor scheduling, the MC concept is not yet mature on distributed systems (such as IoT), especially with end-to-end deadline guarantee. To the best of our knowledge, this paper presents the first attempt to apply the MC concept into interference (or jitter), which is a complicated source of pessimism when analyzing the schedulability of distributed systems. Our goal is to guarantee the end-to-end deadlines of high-criticality flows and minimize the deadline miss ratio of low-criticality flows in distributed systems. To achieve this goal, we introduce a jitter-based MC (JMC) scheduling framework, which supports node-level mode changes in distributed systems. We present an optimal feasibility condition (subject to given schedulability analysis) and two policies to determine jitter-threshold values to achieve the goal in different conditions. Via simulation results for randomly generated workloads, JMC outperforms an existing criticality-monotonic scheme in terms of achieving higher schedulability and fewer deadline misses.

Despite recent advances, there still remain many problems to design reliable cyber-physical systems. One of the typical problems is to achieve a seemingly conflicting goal, which is to support timely delivery of real-time flows while improving resource efficiency. Recently, the concept of mixed-criticality (MC) has been widely accepted as useful in addressing the goal for real-time resource management. However, it has not been yet studied well for real-time communication. In this paper, we present the first approach to support MC flow scheduling on switched Ethernet networks leveraging an emerging network architecture, software-defined networking (SDN). Though SDN provides flexible and programmatic ways to control packet forwarding and scheduling, it yet raises several challenges to enable real-time MC flow scheduling on SDN, including: 1) how to handle (i.e., drop or re-prioritize) out-of-mode packets in the middle of the network when the criticality mode changes and 2) how the mode change affects end-to-end transmission delays. Addressing such challenges, we develop MC-SDN that supports real-time MC flow scheduling by extending SDN-enabled switches and OpenFlow protocols. It manages and schedules MC packets in different ways depending on the system criticality mode. To this end, we carefully design the mode change protocol that provides analytic mode change delay bound, and then resolve implementation issues for system architecture. For evaluation, we implement a prototype of MC-SDN on top of Open vSwitch, and integrate it into a real world network testbed as well as a 1/10 autonomous vehicle. Our extensive evaluations with the network testbed and vehicle deployment show that MC-SDN supports MC flow scheduling with minimal delays on forwarding rule updates and it brings a significant improvement in safety in a real-world application scenario.

In this paper, we present the first approach to support mixed-criticality (MC) flow scheduling on switched Ethernet networks leveraging an emerging network architecture, Software-Defined Networking (SDN). Though SDN provides flexible and programmatic ways to control packet forwarding and scheduling, it yet raises several challenges to enable real-time MC flow scheduling on SDN, including i) how to handle (i.e., drop or reprioritize) out-of-mode packets in the middle of the network when the criticality mode changes, and ii) how the mode change affects end-to-end transmission delays. Addressing such challenges, we develop MC-SDN that supports real-time MC flow scheduling by extending SDN-enabled switches and OpenFlow protocols. It manages and schedules MC packets in different ways depending on the system criticality mode. To this end, we carefully design the mode change protocol that provides analytic mode change delay bound, and then resolve implementation issues for system architecture. For evaluation, we implement a prototype of MC-SDN on top of Open vSwitch, and integrate it into a real world network testbed as well as a 1/10 autonomous vehicle. Our extensive evaluations with the network testbed and vehicle deployment show that MC-SDN supports MC flow scheduling with minimal delays on forwarding rule updates and it brings a significant improvement in safety in a real-world application scenario.

In this paper, we propose an accurate cycle estimation framework which allows to use multiple instruction set simulators to simulate not only processors but also diverse peripheral devices. An instruction set simulator runs on a host machine to mimic functional behaviors of instructions running on a target hardware. It allows to estimate the execution time of software in a fast and accurate way and validate a system even when its target hardware does not yet exist or is not available.

Modern mobile devices are equipped with multiple network interfaces, including 3G/LTE and WiFi. Bandwidth aggregation over LTE and WiFi links offers an attractive opportunity of supporting bandwidth-intensive services, such as high-quality video streaming, on mobile devices. However, achieving effective bandwidth aggregation in mobile environments raises several challenges related to deployment, link heterogeneity, network fluctuation, and energy consumption. We present GreenBag, an energy-efficient bandwidth aggregation middleware that supports real-time data-streaming services over asymmetric wireless links, requiring no modifications to the existing Internet infrastructure and servers. GreenBag employs several techniques, including medium load balancing, efficient segment management, and energy-aware mode control, to resolve such challenges. We implement a prototype of GreenBag on Android-based mobile devices which hosts, to the best knowledge of the authors, the first LTE-enabled bandwidth aggregation prototype for energy-efficient real-time video streaming. Our experiment results in both emulated and real-world environments show that GreenBag not only achieves good bandwidth aggregation to provide QoS in bandwidth-scarce environments but also efficiently saves energy on mobile devices. Moreover, energy-aware GreenBag can minimize video interruption while consuming 14-25% less energy than the non-energy-aware counterpart in real-world experiments.

The last several years have seen a rapid increase in smart phone use. Android offers an open-source software platform on smart phones, that includes a Linux-based kernel, Java applications, and middleware. The Android middleware provides system libraries and services to facilitate the development of performance-sensitive or device-specific functionalities, such as screen display, multimedia, and web browsing. Android keeps track of which applications make use of which system services for some pre-defined functionalities, and which application is running in the foreground attracting the user's attention. Such information is valuable in capturing application characteristics and can be useful for resource management tailored to application requirements. However, the Linux-based Android kernel does not utilize such information for I/O resource management. This paper is the first work, to the best of our knowledge, to attempt to understand application characteristics through Android architecture and to incorporate those characteristics into disk and network I/O management. Our proposed approach, Aciom (Application Characteristics-aware I/O Management), requires no modification to applications and characterizes application I/O requests as time-sensitive, bursty, or plain, depending on which system services are involved and which application receives the user's focus. Aciom then provides differentiated I/O management services for different types of I/O requests, supporting minimum bandwidth reservations for time-sensitive requests and placing maximum bandwidth limits on bursty requests. We present the design of Aciom and a prototype implementation on Android. Our experimental results show that Aciom is quite effective in handling disk and network I/O requests in support of time-sensitive applications in the presence of bursty I/O requests.