Internet of Things (IoT) paradigm raises challenges for devising efficient strategies that offload applications to the fog or the cloud layer while ensuring the optimal response time for a service. Traditional computation offloading policies assume the response time is only dominated by the execution time. However, the response time is a function of many factors including contextual parameters and application characteristics that can change over time. For the computation offloading problem, the majority of existing literature presents efficient solutions considering a limited number of parameters (e.g., computation capacity and network bandwidth) neglecting the effect of the application characteristics and dataflow configuration. In this paper, we explore the impact of the computation offloading on total application response time in three-layer IoT systems considering more realistic parameters, e.g., application characteristics, system complexity, communication cost, and dataflow configuration. This paper also highlights the impact of a new application characteristic parameter defined as Output–Input Data Generation (OIDG) ratio and dataflow configuration on the system behavior. In addition, we present a proof-of-concept end-to-end dynamic computation offloading technique, implemented in a real hardware setup, that observes the aforementioned parameters to perform real-time decision-making.

Deep-learning-based intelligent services have become prevalent in cyber-physical applications, including smart cities and health-care. Deploying deep-learning-based intelligence near the end-user enhances privacy protection, responsiveness, and reliability. Resource-constrained end-devices must be carefully managed to meet the latency and energy requirements of computationally intensive deep learning services. Collaborative end-edge-cloud computing for deep learning provides a range of performance and efficiency that can address application requirements through computation offloading. The decision to offload computation is a communication-computation co-optimization problem that varies with both system parameters (e.g., network condition) and workload characteristics (e.g., inputs). However, deep learning model optimization provides another source of tradeoff between latency and model accuracy. An end-to-end decision-making solution that considers such computation-communication problem is required to synergistically find the optimal offloading policy and model for deep learning services. To this end, we propose a reinforcement-learning-based computation offloading solution that learns optimal offloading policy considering deep learning model selection techniques to minimize response time while providing sufficient accuracy. We demonstrate the effectiveness of our solution for edge devices in an end-edge-cloud system and evaluate with a real-setup implementation using multiple AWS and ARM core configurations. Our solution provides 35% speedup in the average response time compared to the state-of-the-art with less than 0.9% accuracy reduction, demonstrating the promise of our online learning framework for orchestrating DL inference in end-edge-cloud systems.

Self-driving systems execute an ensemble of different self-driving workloads on embedded systems in an end-to-end manner, subject to functional and performance requirements. To enable exploration, optimization, and end-to-end evaluation on different embedded platforms, system designers critically need a benchmark suite that enables flexible and seamless configuration of self-driving scenarios, which realistically reflects real-world self-driving workloads’ unique characteristics. Existing CPU and GPU embedded benchmark suites typically (1) consider isolated applications, (2) are not sensor-driven, and (3) are unable to support emerging self-driving applications that simultaneously utilize CPUs and GPUs with stringent timing requirements. On the other hand, full-system self-driving simulators (e.g., AUTOWARE, APOLLO) focus on functional simulation, but lack the ability to evaluate the self-driving software stack on various embedded platforms. To address design needs, we present Chauffeur, the first open-source end-to-end benchmark suite for self-driving vehicles with configurable representative workloads. Chauffeur is easy to configure and run, enabling researchers to evaluate different platform configurations and explore alternative instantiations of the self-driving software pipeline. Chauffeur runs on diverse emerging platforms and exploits heterogeneous onboard resources. Our initial characterization of Chauffeur on different embedded platforms – NVIDIA Jetson TX2 and Drive PX2 – enables comparative evaluation of these GPU platforms in executing an end-to-end self-driving computational pipeline to assess the end-to-end response times on these emerging embedded platforms while also creating opportunities to create application gangs for better response times. Chauffeur enables researchers to benchmark representative self-driving workloads and flexibly compose them for different self-driving scenarios to explore end-to-end tradeoffs between design constraints, power budget, real-time performance requirements, and accuracy of applications.

Modern heterogeneous CPU-GPU-based mobile architectures, which execute intensive mobile gaming/graphics applications, use software governors to achieve high performance with energy-efficiency. However, existing governors typically utilize simple statistical or heuristic models, assuming linear relationships using a small unbalanced dataset of mobile games; and the limitations result in high prediction errors for dynamic and diverse gaming workloads on heterogeneous platforms. To overcome these limitations, we propose an interpretable machine learning (ML) model enhanced integrated CPU-GPU governor: (1) It builds tree-based piecewise linear models (i.e., model trees) offline considering both high accuracy (low error) and interpretable ML models based on mathematical formulas using a simulatability operation counts quantitative metric. And then (2) it deploys the selected models for online estimation into an integrated CPU-GPU Dynamic Voltage Frequency Scaling governor. Our experiments on a test set of 20 mobile games exhibiting diverse characteristics show that our governor achieved significant energy efficiency gains of over 10% (up to 38%) improvements on average in energy-per-frame with a surprising-but-modest 3% improvement in Frames-per-Second performance, compared to a typical state-of-the-art governor that employs simple linear regression models.

Mobile platforms are increasingly using Heterogeneous Multiprocessor Systems-on-Chip (HMPSoCs) with differentiated processing cores and GPUs to achieve high performance for graphics-intensive applications such as mobile games. Traditionally, separate CPU and GPU governors are deployed in order to achieve energy efficiency through Dynamic Voltage Frequency Scaling (DVFS) but miss opportunities for further energy savings through coordinated system-level application of DVFS. We present a cooperative CPU-GPU DVFS strategy (called Co-Cap) that orchestrates energy-efficient CPU and GPU DVFS through synergistic CPU and GPU frequency capping to avoid frequency overprovisioning while maintaining desired performance. Unlike traditional approaches that target a narrow set of mobile games, our Co-Cap approach is applicable across a wide range of microbenchmarks and mobile games. Our methodology employs a systematic training phase using fine-grained refinement steps with evaluations of frequency capping tables followed by a deployment phase, allowing deployment across a wide range of microbenchmarks and mobile games with varying graphics workloads. Our experimental results across multiple sets of over 200 microbenchmarks and 40 mobile games show that Co-Cap improves energy per frame by on average 8.9% (up to 18.3%) and 7.8% (up to 27.6%) (16.6% and 15.7% in CPU-dominant applications) and achieves minimal frames-per-second (FPS) loss by 0.9% and 0.85% (1.3% and 1.5% in CPU-dominant applications) on average in training and deployment sets, respectively, compared to the default CPU and GPU governors, with negligible overhead in execution time and power consumption on the ODROID-XU3 platform.

Contemporary mobile platforms use software governors to achieve high performance with energy-efficiency for heterogeneous CPU-GPU based architectures that execute mobile games and other graphics-intensive applications. Mobile games typically exhibit inherent behavioral dynamism, which existing governor policies are unable to exploit effectively to manage CPU/GPU DVFS policies. To overcome this problem, we present HiCAP: a Hierarchical Finite State Machine (HFSM) based CPU-GPU governor that models the dynamic behavior of mobile gaming workloads, and applies a cooperative, dynamic CPU-GPU frequency-capping policy to yield energy efficiency adapting to the game's inherent dynamism. Our experiments on a large set of 37 mobile games exhibiting dynamic behavior show that our CAP dynamic governor policy achieved substantial energy efficiency gains of up to 18% improvement in energy-per-frame over existing governor policies, with minimal degradation in quality.

We briefly introduce our ongoing work on the development of ARM-based hypervisor for real-time embedded systems. Our open source hypervisor fully utilize hardware-assisted virtualization of ARM and significantly improves performance predictability.

In the early design stage of processors, Dynamic Thermal Management (DTM) schemes should be evaluated to avoid excessively high temperature, while minimizing performance overhead. In this paper, we show that conventional thermal simulations using the fixed ambient temperature may lead to the wrong conclusions in terms of temperature, performance, reliability, and leakage power. Though ambient temperature converges to a steady-state value after hundreds of seconds when we run SPEC CPU2000 benchmark suite, the steady-state ambient temperature is significantly different depending on applications and system configuration. To overcome inaccuracy of conventional thermal simulations, we propose that microarchitectural thermal simulations should exploit application/system-dependent ambient temperature. Our evaluation results reveal that performance, thermal behavior, reliability, and leakage power of the same DTM scheme are different when we use the application/system-dependent ambient temperature instead of the fixed ambient temperature. For accurate simulation results, future microarchitectural thermal researchers are expected to evaluate their proposed DTM schemes based on application/system-dependent ambient temperature.

In the early stage of processor design, Dynamic Thermal Management (DTM) schemes should be evaluated to avoid excessively high temperature, while minimizing performance overhead as small as possible. In this paper, we show that conventional thermal simulations using fixed ambient temperature may lead to wrong conclusion in terms of performance and temperature; though ambient temperature converges to a steady state after hundreds of seconds, the steady state ambient temperature is significantly different depending on applications. To overcome the inaccuracy of conventional thermal simulations, we propose that architectural thermal simulation should exploit application-dependent ambient temperature. Our evaluation results show that the performance of the same DTM scheme is different, when application-dependent ambient temperature (compared to fixed temperature) is used. For accurate simulation, future architectural thermal researchers are expected to evaluate their proposed DTM schemes, reflecting application- dependent ambient temperature.

Application launching times in embedded systems are more crucial than in general-purpose systems since the response times of embedded applications are significantly affected by the launching times. As general-purpose operating systems are increasingly used in embedded systems, reducing appli-cation launching times are one of the most influential factors for performance improvement. In order to reduce the application launching times, three factors should be considered at the same time: relocation time, symbol resolution time, and binary loading time. In this paper, we propose a new application execution model using a combination of prelinking and preloading to reduce the relocation, symbol resolution, and binary load overheads at the same time. Such application execution model is realized using fork and dlopen execution model instead of traditional fork and exec execution model. We evaluate the performance effect of the proposed fork and dlopen application execution model on a Linux-based embedded system using XScale processor. By applying the proposed application execution model using both prelinking and preloading, the application launching times are reduced up to 71% and relocation counts are reduced up to 91% in the benchmark programs we used.

We propose a worst case timing analysis technique for in-order multiple-issue machines. In the proposed technique, timing information for each program construct is represented by a directed acyclic graph (DAG) that shows dependences among instructions in the program construct. From this information, we derive for each pair of instructions the distance bounds between their issue times. Using these distance bounds, we identify the sets of instructions that can be issued at the same time. Deciding such instructions is an essential task in reasoning about the timing behavior of multiple-issue machines. In order to reduce the complexity of analysis, the distance bounds are progressively refined through a hierarchical analysis over the program syntax tree in a bottom-up fashion. Our experimental results show that the proposed technique can predict the worst case execution times for in-order multiple-issue machines as accurately as ones for simpler RISC processors.

This paper presents a case study of worst case timing analysis for a RISC processor. The target machine consists of the R3000 CPU and R3010 FPA (Floating Point Accelerator). This target machine is typical of a RISC system with pipelined execution units and cache memories. Our methodology is an extension of the existing timing schema. The extended timing schema provides means to reason about the execution time variation of a program construct by surrounding program constructs due to pipelined execution and cache memories of RISC processors. The main focus of this paper is on explaining the necessary steps for performing timing analysis of a given target machine within the extended timing schema framework. This paper also gives results from experiments using a timing tool for the target machine that is built based on the extended timing schema approach.

An accurate and safe estimation of a task's worst case execution time (WCET) is crucial for reasoning about the timing properties of real time systems. In RISC processors, the execution time of a program construct (e.g., a statement) is affected by various factors such as cache hits/misses and pipeline hazards, and these factors impose serious problems in analyzing the WCETs of tasks. To analyze the timing effects of RISC's pipelined execution and cache memory, we propose extensions to the original timing schema where the timing information associated with each program construct is a simple time bound. In our approach, associated with each program construct is worst case timing abstraction, (WCTA), which contains detailed timing information of every execution path that might be the worst case execution path of the program construct. This extension leads to a revised timing schema that is similar to the original timing schema except that concatenation and pruning operations on WCTAs are newly defined to replace the add and max operations on time bounds in the original timing schema. Our revised timing schema accurately accounts for the timing effects of pipelined execution and cache memory not only within but also across program constructs. The paper also reports on preliminary results of WCET analysis for a RISC processor. Our results show that tight WCET bounds (within a maximum of about 30% overestimation) can be obtained by using the revised timing schema approach.>

An accurate and safe estimation of a task's worst case execution time (WCET) is crucial for reasoning about the timing properties of real-time systems. In RISC processors, the execution time of a program construct (e.g., a statement) is affected by various factors such as cache hits/misses and pipeline hazards, and these factors impose serious problems in analyzing the WCETs of tasks. To analyze the timing effects of RISC's pipelined execution and cache memory, this paper proposes extensions of the original timing schema (Shaw, 1989) where the timing information associated with each program construct is a simple time-bound. We associate with each program construct what we call a WCTA (Worst Case Timing Abstraction), which contains detailed timing information of every execution path that might be the worst case execution path of the program construct. This extension leads to a revised timing schema that is similar to the original timing schema except that concatenation and pruning operations on WCTAs are newly defined to replace the add and max operations on time-bounds in the original timing schema. Our revised timing schema accurately accounts for the timing effects of pipelined execution and cache memory not only within but also across program constructs. This paper also reports on preliminary results of WCET analyses for a pipelined processor. Our results show that up to 50% tighter WCET bounds can be obtained by using the revised timing schema.>