ABSTRACT Pipeline parallelization is an effective technique that enables the efficient execution of deep neural network (DNN) inference on resource-constrained IoT devices. To enable pipeline parallelization across computing nodes with asymmetric performance profiles, interconnected via low-latency, high-bandwidth networks, we propose DNNPipe, a DNN partitioning algorithm that constructs a pipeline plan for a given DNN. The primary objective of DNNPipe is to maximize the throughput of DNN inference while minimizing the runtime overhead of DNN partitioning, which is repeatedly executed online in dynamically changing IoT environments. To achieve this, DNNPipe uses dynamic programming (DP) with pruning techniques that preserve optimality to explore the search space and find the optimal pipeline plan whose maximum stage time is no greater than that of any other possible pipeline plan. Specifically, it aggressively prunes suboptimal pipeline plans using two pruning techniques: upper-bound-based pruning and under-utilized-stage pruning . Our experimental results demonstrate that pipelined inference using an obtained optimal pipeline plan improves DNN throughput by up to 1.78 times compared to the highest performing single device and DNNPipe achieves up to 98.26% lower runtime overhead compared to PipeEdge, the fastest known optimal DNN partitioning algorithm.

The emergence of software-defined vehicles (SDVs) introduces significant challenges in dynamically deploying services with diverse criticality semantics. To address this issue, we present a framework for the dynamic mapping of mixed-criticality applications (MCAs) onto a mixed-criticality runtime system (MCR) with probabilistic guarantees. We model an SDV service, such as a Docker container, as an MCA and provide an MCR based on a finite-state machine. We present an approach that maps the criticality levels of an MCA to those of the MCR, tracks available resources in the MCR, converts the resource demands of an MCA, and performs admission control to ensure the MCR remains schedulable. This framework enables the reliable and prioritized execution of critical SDV functions while appropriately managing less critical tasks.

Pipeline parallelization is an effective technique that enables the efficient execution of deep neural network (DNN) inference on resource-constrained IoT devices. To support pipeline parallelization on heterogeneous computing nodes with low-latency networks, we propose DNNPipe, a DNN partitioning algorithm that constructs a pipeline plan for a given DNN. The primary objective of DNNPipe is to maximize the throughput of DNN inference while minimizing the runtime overhead of DNN partitioning, which is repeatedly executed in dynamically changing IoT environments. To achieve this, DNNPipe uses dynamic programming for an exhaustive exploration to find the optimal pipeline plan whose maximum stage execution time is no greater than that of any other possible pipeline plan. Additionally, it aggressively prunes suboptimal pipeline plans using an upper bound on the minimum value among all possible pipeline plans' maximum stage execution times. Experimental results demonstrate that DNNPipe significantly reduces its execution time and iteration counts compared to PipeEdge, the fastest known optimal DNN partitioning algorithm.

One of the major difficulties in developing embedded systems with object-oriented modeling is to translate a designed model into code that satisfies required real-time performance. This paper proposes scenario-based implementation synthesis architecture with timing guarantee (SISAtime) that addresses these difficulties. The problems that SISAtime must solve are: how to synthesize multithreaded-code from a real-time object-oriented model; and how to design supporting development tools and runtime system architecture while ensuring that the scenarios in the system have minimal response times and the code satisfies the given timing constraints with a minimal number of threads. SISAtime provides a new scheduling algorithm which minimizes scenario response times. SISAtime also provides a new thread derivation method that derives tasks and maps tasks to threads while automatically assigning task scheduling attributes. We have fully implemented SISAtime by extending the RoseRT development tool that uses UML 2.0 as a modeling language, and we applied it to an existing industrial private branch exchange system. The performance evaluation results show that the response times, context switches, and the number of threads of the system with SISAtime were reduced by 21.6, 33.2, and 65.2 percent, respectively, compared to the system with the best known existing thread derivation method.

Despite a rapid decrease in the price of solid state memory devices, system memory is still a very precious resource in embedded systems. The use of shared libraries and execution-in-place (XIP) is known to be effective in significantly reducing memory usage. Unfortunately, many resource-constrained embedded systems lack an MMU, making it extremely difficult to support these techniques. To address this problem, we propose a novel shared library technique called a quasi-static shared library and an XIP, both based on our enhanced position independent code technique. In our quasistatic shared libraries, global symbols are bound to pseudoaddresses at linking time and actual physical addresses are bound at loading time. Unlike conventional shared libraries, they do not require symbol tables that take up valuable memory space and, therefore, allow for expedited address translation at runtime. Our XIP technique is facilitated by our enhanced position independent code where a data section can be arbitrarily located. Both the shared library and XIP techniques are made possible by emulating an MMU's memory mapping feature with a data section base register (DSBR) and a data section base table (DSBT). We have implemented these proposed techniques in a commercial ADSL (Asymmetric Digital Subscriber Line) home network gateway equipped with an MMU-less ARM7TDMI processor core, 2MB flash memory, and 16MB RAM. We measured its memory usage and evaluated its performance overhead by conducting a series of experiments. These experiments clearly demonstrate the effectiveness of our techniques in reducing memory usage. The results are impressive: 35% reduction in flash memory usage when using only the shared library and 30% reduction in RAM usage when using the shared library and XIP together. These results were achieved with only a negligible performance penalty of less than 4%. Even though these techniques were applied to uClinux-based embedded systems, they can be used for any MMU-less real-time operating system.

The software communications architecture (SCA), which has been adopted as an SDR (software defined radio) Forum standard, provides a framework that successfully exploits common design patterns of distributed, real-time, and object-oriented embedded systems software. We have fully implemented the SCA v2.2 in C++. During this implementation process, we have encountered the lack of a suitable design pattern for releasing the SCA applications. Unfortunately, design patterns for releasing objects have been neither extensively addressed nor well investigated as opposed to creational design patterns. This is largely due to the fact that such releasing design patterns are highly dependent on programming languages. In this paper, we investigate three viable design patterns for releasing the SCA applications in C++ and discuss their pros and cons. In addition, we select the most portable and thus most reusable pattern, which we name Vulture design pattern, among those alternatives and detail our specific implementation.