Tiny machine learning (TinyML) enables low-power microcontrollers to leverage the power of artificial intelligence without relying on remote computing resources. Typically, developing a TinyML application relies primarily on existing TinyML frameworks, which provide runtime APIs for loading and executing machine learning models. However, such framework-based TinyML development has limitations in terms of portability, programmability, and resource efficiency, motivating the need for a new approach to the TinyML development and deployment process.

As the memory wall problem worsens, Near-Data Processing (NDP) has emerged to reduce data movement by computing close to data. Recently proposed memory-mapped NDP (M2NDP) enables general-purpose NDP with low hardware overhead by extending RISC-V ISA and maximizing data parallelism with lightweight µthreads. However, a high-level programming model for this architecture—particularly one that naturally expresses control flow across kernels—has not yet been established.

Achieving ultra low-power consumption is essential for embedded systems deployed in harsh environments, such as space and deep sea locations, where energy resources are scarce and physical accessibility is limited. Typically, these systems employ ultra low-power microcontrollers that operate on narrow data widths of 8 or 16 bits at the microarchitecture level. If software developers do not carefully consider the data widths during programming, the resulting programs may be suboptimally optimized for these ultra low-power systems. To address this issue and enable more efficient low-power computing, this work proposes a novel optimizing compiler that supports bit-level analyses and transformations. The proposed compiler analyzes how each individual bit of a data item is utilized within a program to determine its optimal width. Consequently, the proposed compiler reduces unnecessary data movements and computational overhead on ultra low-power processors. This work implements the prototype compiler on top of the LLVM compiler framework and evaluates the performance impact of the optimized embedded applications with a processor simulator.

Thanks to its massive parallel resources, a GPU is a promising platform for graph processing. However, the increasing size and skewed characteristics of the real-world graphs limit the performance improvement. Prior work proposes locality-enhancing graph transformations and load balancing techniques to improve performance, but they still suffer from excessive memory usage and inefficient parallel resource utilization because their graph representations are not fully tailored for a GPU. To efficiently utilize the GPU resource with less memory, this work proposes a new graph representation, called CR2. First, CR2 extracts community-aware subgraphs from a graph by clustering densely-connected vertices together. For the community-aware subgraphs, CR2 decomposes a vertex ID into a cluster ID and a local ID and represents each vertex only with the local ID, thus reducing memory usage. Second, CR2 additionally partitions the graph into multiple degree-ordered subgraphs in which all the vertices have the same regularized number of edges, thus making parallel workload balanced across GPU warps. This work evaluates CR2 with four commonly used graph algorithms and shows that CR2 achieves 1.53 times performance speedup while using 32.1% less memory on the geomean average compared to the state-of-the-art techniques.

Image segmentation is one of the major computer vision tasks, which is applicable in a variety of domains, such as autonomous navigation of an unmanned aerial vehicle. However, image segmentation cannot easily materialize on tiny embedded systems because image segmentation models generally have high peak memory usage due to their architectural characteristics. This work finds that image segmentation models unnecessarily require large memory space with an existing tiny machine learning framework. That is, the existing framework cannot effectively manage the memory space for the image segmentation models. This work proposes TinySeg, a new model optimizing framework that enables memory-efficient image segmentation for tiny embedded systems. TinySeg analyzes the lifetimes of tensors in the target model and identifies long-living tensors. Then, TinySeg optimizes the memory usage of the target model mainly with two methods: (i) tensor spilling into local or remote storage and (ii) fused fetching of spilled tensors. This work implements TinySeg on top of the existing tiny machine learning framework and demonstrates that TinySeg can reduce the peak memory usage of an image segmentation model by 39.3% for tiny embedded systems.

Employing high-performance neural network models is challenging for resource-constrained devices as the models require strong computing power and large memory space. One-shot neural architecture search (NAS) helps to find a suitable neural architecture more efficiently by using a single one-shot network that shares weights with sub-networks. However, the existing one-shot NAS methods pay little attention to the sparsity achievable from the one-shot network. Therefore, there still may be room to reduce the resource requirement of the network. This work presents a new one-shot sparse NAS method, which tries to find an optimal sparsity for the network using soft channel masking during the architecture search. The experimental results show that the proposed method can find a more sparse architecture with little accuracy drop.

Despite the benefit of Fully Homomorphic Encryption (FHE) that supports encrypted computation, writing an efficient FHE application is challenging due to magnitude scale management. Each FHE operation increases scales of ciphertext and leaving the scales high harms performance of the following FHE operations. Thus, rescaling ciphertext is inevitable to optimize an FHE application, but since FHE requires programmers to match the rescaling levels of operands of each FHE operation, programmers should rescale ciphertext reflecting the entire FHE application. Although recently proposed FHE compilers reduce the programming burden by automatically manipulating ciphertext scales, they fail to fully optimize the FHE application because they greedily rescale the ciphertext without considering their performance impacts throughout the entire application. This work proposes HECATE, a new FHE compiler framework that optimizes scales of ciphertext reflecting their rescaling levels and performance impact. With a new type system that embeds the scale and rescaling level, and a new rescaling operation called downscale, HECATE makes various scale management plans, analyzes their expected performance, and finds the optimal rescaling points throughout the entire FHE application. This work implements HECATE on top of the MLIR framework with a Python frontend and shows that HECATE achieves 27% speedup over the state-of-the-art approach for various FHE applications.

Serverless computing like Function-as-a-Service (FaaS) is attractive for IoT service providers, liberating the providers from server maintenance. Since a data processing function is executed on the cloud instead of a dedicated server in the FaaS platform, the service users send their private data in their IoT devices to the third-party cloud, taking privacy leakage risks. Homomorphic encryption (HE) can preserve the privacy by enabling encrypted data processing on the cloud, but using HE for every data item incurs large computation and communication overheads. This work proposes SelectiveCrypt, a compiler-assisted semantic-aware encryption scheme that applies different cryptographic primitives depending on the operations on each data item. SelectiveCrypt homomorphically encrypts data items if arithmetic operations are applied to the data, while SelectiveCrypt encrypts data items with a symmetric key if the data are stored in the cloud without any arithmetic operation. The SelectiveCrypt framework consists of a compiler and its runtime system. The SelectiveCrypt compiler statically analyzes the data processing, determines an appropriate cryptographic primitive for each data item, and automatically transforms arithmetic operations into the homomorphic computation. The SelectiveCrypt runtime encrypts and decrypts the data items according to the static analysis result. This work evaluates the prototype SelectiveCrypt framework with five benchmarks that reflect real-world IoT scenarios. The evaluation results show that the SelectiveCrypt framework successfully reduces response time and communication overhead by 1.59 times and 9.61 times, respectively, compared with a HE scheme.

The Internet of Things (IoT) enables new services, such as smart home and smart healthcare integrating various resources of networked devices. However, managing IoT resources is very challenging because a device may have multiple different resources and the service accesses subsets of the resources in multiple devices. To simplify the resource management, the existing IoT frameworks provide resource models and protocols for manufacturers and programmers, but their access granularity is too coarse grained causing overprivilege problems, or too fine grained causing management overheads. To avoid the overprivilege problems without the management overheads, this work proposes a new composable and flexible resource management scheme and implements its prototype compiler-runtime framework called ComFlex. The ComFlex compiler allows manufacturers to register their devices as composition of fine-grained resources by inheriting the existing interfaces of the resources, and programmers to define their own access granularity as a virtual resource that consists of the fine-grained resources of multiple devices. The ComFlex runtime supports fine-grained access control without additional overheads by mapping the fine-grained resources of the virtual resource into physical resources of different devices. To evaluate the ComFlex framework, this work implements 52 resources on 14 devices and six IoT services with ComFlex and an existing fine-grained resource management scheme. Compared to the existing scheme, the ComFlex framework supports the IoT services with 40.2% fewer lines of code and 80.9% less discovery time, without any overprivilege problem and response time delay.

Thanks to the recent advances in Deep Neural Networks (DNNs), DNN-based object detection systems become highly accurate and widely used in real-time environments such as autonomous vehicles, drones and security robots. Although the systems should detect objects within a certain time limit that can vary depending on their execution environments such as vehicle speeds, existing systems blindly execute the entire long-latency DNNs without reflecting the time-varying time limits, and thus they cannot guarantee real-time constraints. This work proposes a novel real-time object detection system that employs multipath neural networks based on a new worst-case execution time (WCET) model for DNNs on a GPU. This work designs the WCET model for a single DNN layer analyzing processor and memory contention on GPUs, and extends the WCET model to the end-to-end networks. This work also designs the multipath networks with three new operators such as skip, switch, and dynamic generate proposals that dynamically change their execution paths and the number of target objects. Finally, this work proposes a path decision model that chooses the optimal execution path at run-time reflecting dynamically changing environments and time constraints. Our detailed evaluation using widely-used driving datasets shows that the proposed real-time object detection system performs as good as a baseline object detection system without violating the time-varying time limits. Moreover, the WCET model predicts the worst-case execution latency of convolutional and group normalization layers with only 27% and 81% errors on average, respectively.

In the Internet-of-Things (IoT) environments, users define event-condition-action (ECA) rules, and expect IoT frameworks to evaluate conditions and take appropriate actions within a certain time limit after an event occurs. To evaluate the conditions with fresh data items, the frameworks acquire required data from IoT sensors. Since the data acquisition causes battery consumption of sensors, the frameworks should minimize the number of the data acquisition while keeping the sensor data fresh until finishing the condition evaluation. However, existing data acquisition schedulers inefficiently acquire sensor data because the schedulers assume each ECA rule in a program is independent of each other although different rules may share some sensing data from the same sensors. This work proposes an efficient sharing-aware data acquisition scheduling algorithm that reduces unnecessary data acquisition by sharing sensor data commonly used in different rules while satisfying time constraints. To evaluate the proposed scheduling algorithm, this work deploys 19 devices in an office, collects values of 26 different sensors for 144 hours, and simulates the proposed algorithm and a baseline algorithm. Compared to the baseline algorithm, the proposed algorithm reduces communication count and deadline miss ratio by 31.9% and 50.2% respectively.

In the Internet of Things (IoT) environments, cloud servers integrate various IoT devices including sensors and actuators, and provide new services that assist daily lives of users interacting with the physical world. While response time is a crucial factor of quality of the services, supporting short response time is challenging for the cloud servers due to a growing number and amount of connected devices and their communication. To reduce the burden of the cloud servers, fog computing is a promising alternative to offload computation and communication overheads from the cloud servers to fog nodes. However, since existing fog computing frameworks do not extract codes for fog nodes fully automatically, programmers should manually write and analyze their applications for fog computing. This work proposes Spinal Code, a new compiler-runtime framework for near-user computation that automatically partitions an original cloud-centric program into distributed sub-programs running over the cloud and fog nodes. Moreover, to reduce response time in the physical world, Spinal Code allows programmers to annotate latency sensitive actuators in a program, and optimizes the critical paths from required sensors to the actuators when it generates the sub-programs. This work implements 9 IoT programs across 4 service domains: healthcare, smart home, smart building and smart factory, and demonstrates that Spinal Code successfully reduces 44.3% of response time and 79.9% of communication on the cloud compared with a cloud-centric model.

The explosion of networked devices has driven a new computing environment called the Internet of Things (IoT), enabling various services such as home automation and health monitoring. Despite the promising applicability of the IoT, developing an IoT service is challenging for programmers, because the programmers should integrate multiple programmable devices and heterogeneous third-party devices. Recent works have proposed integrated programming platforms, but they either require device-specific implementation for third-party devices without any device abstraction, or abstract all the devices to the standard interfaces requiring unnecessary abstraction of programmable devices. To integrate IoT devices with selective abstraction, this work revisits the object oriented programming (OOP) model, and proposes a new language extension and its compiler-runtime framework, called Esperanto. With three annotations that map each object to its corresponding IoT device, the Esperanto language allows programmers to integrate multiple programmable devices into one OOP program and to abstract similar third-party devices into their common ancestor classes. Given the annotations, the Esperanto compiler automatically partitions the integrated program into multiple sub-programs for each programmable IoT device, and inserts communication and synchronization code. Moreover, for the ancestor classes, the Esperanto runtime dynamically identifies connected third-party devices, and links their corresponding descendent objects. Compared to an existing approach on the integrated IoT programming, Esperanto requires 33.3% fewer lines of code to implement 5 IoT services, and reduces their response time by 44.8% on average.

Integrating various networked devices, the Internet of Things (IoT) enables various new services like home automation, making its market larger and more competitive. Although rapid development of an IoT application is crucial to keep up with the highly competitive IoT market, developing an IoT application is challenging for programmers because the programmers should integrate multiple programmable devices and heterogeneous third-party devices. Some IoT frameworks integrate programming environments of multiple devices, but they either require device-specific implementation for third-party devices without any device abstraction, or abstract all the devices to the standard interfaces requiring unnecessary abstraction of programmable devices. This work introduces the Esperanto framework that integrates IoT devices with selective abstraction, allowing rapid prototyping of an IoT application. Exploiting the correspondence between an object and a thing in the object oriented programming (OOP) model, the Esperanto framework allows programmers to write only one OOP program instead of multiple programs for each device, and to manipulate third-party devices with their common ancestor classes. Compared to an existing approach on the integrated IoT programming, Esperanto requires 33.3% fewer lines of code to implement 5 IoT services, and reduces their response time by 44.8% on average. Moreover, with an empirical study, this work shows that the Esperanto framework reduces the development time by 52.7%.

With a simple “If This Then That” syntax, IoT frameworks such as IFTTT and Microsoft Flow allow users to easily create custom applets integrating sensors and actuators. Users expect appropriate actions to be taken within a certain latency in response to sensor value changes while the sensors usually have limited battery power. Therefore, reading the sensor values at the right time point is crucial for the IoT frameworks to support real-time responses of the applets while saving battery lives of sensors. However, existing IoT frameworks periodically read the sensor data with fixed intervals without reflecting current sensor values and trigger conditions of applets, so the intervals are either too long to meet the real-time constraints, or too short wasting batteries of sensors. This work extends the existing IFTTT syntax for users to describe real-time constraints, and proposes the first real-time IoT framework with trigger condition-aware flexible polling intervals, called RT-IFTTT. RT-IFTTT analyzes current sensor values, trigger conditions and constraints of all the applets in the framework, and dynamically calculates the efficient polling intervals for each sensor. This work collects real-world sensing data from 10 physical sensors for 10 days, and shows that the RT-IFTTT framework with the proposed scheduling algorithm executes 100 to 400 applets according to user-defined real-time constraints with up to 64.12% less sensor polling counts compared to the framework with the fixed intervals.

Although mobile devices have been evolved enough to support complex mobile programs, performance of the mobile devices is lagging behind performance of servers. To bridge the performance gap, computation offloading allows a mobile device to remotely execute heavy tasks at servers. However, due to architectural differences between mobile devices and servers, most existing computation offloading systems rely on virtual machines, so they cannot offload native applications. Some offloading systems can offload native mobile applications, but their applicability is limited to well-analyzable simple applications. This work presents automatic cross-architecture computation offloading for general-purpose native applications with a prototype framework that is called Native Offloader. At compile-time, Native Offloader automatically finds heavy tasks without any annotation, and generates offloading-enabled native binaries with memory unification for a mobile device and a server. At run-time, Native Offloader efficiently supports seamless migration between the mobile device and the server with a unified virtual address space and communication optimization. Native Offloader automatically offloads 17 native C applications from SPEC CPU2000 and CPU2006 benchmark suites without a virtual machine, and achieves a geomean program speedup of 6.42× and battery saving of 82.0%.