Augmented Reality (AR) System-on-Chips (SoCs) have strict power budgets and form-factor limitations for wearable, all-day use AR glasses running high-performance applications. Limited compute and memory resources that can fit within the strict industrial design area footprint of an AR SoC, however, create performance bottlenecks for demanding workloads such as Pixel Codec Avatars (PiCA) group-calling which connects multiple users with their photorealistic representations. To alleviate this unique wearables challenge, 3D integration with hybrid-bonding technology offers energy-efficient 3D stacking of more silicon resources within the same SoC footprint. Implementing such 3D architectures, however, is another challenge as current EDA tools and flows offer limited 3D design control. In this work, we present a 3D design methodology for robust 3D clock network and datapath design using current EDA tools. To validate the proposed methodology, we implemented a 3D integrated prototype AR SoC housing a 3D-stacked Machine Learning (ML) accelerator utilizing TSMC SoIC™bonding technology. Silicon measurements demonstrate that the 3D ML accelerator enables running PiCA AR group call at 30 frames-per-second (fps) by 3D-expanding its memory resources by 4× to achieve 2× better energy-efficiency when compared to a 2D baseline accelerator at iso-footprint.

This paper proposes an extension to dataflow models with timing specifications to facilitate the construction of deterministic, platform independent, precisely timed models of software in cyber-physical systems (CPS). Dataflow models are often used to describe the software/cyber part of a CPS, owing to their succinct and analyzable representation of computation and concurrency. To capture the interaction of the cyber with the physical part, it is common practice to augment the dataflow model with nodes to represent physical sensors and actuators and handle the timing outside the dataflow model. However, the precise timing of these interactions is critical to the overall application behavior, and conventional dataflow models do not capture these timing requirements. In this work, we introduce timing configurations in dataflow models to specify when this communication between cyber and physical parts takes place. Timing specifications are derived from application requirements which are independent of the platform execution behavior. A correct implementation must fulfill the dataflow and timing requirements. This paper discusses the extension of the well-studied Synchronous Dataflow (SDF) model with timing configurations, shows how traditional SDF analysis for consistency and deadlock freedom is adapted for this model, and discusses hierarchical composition and analysis of composite SDF nodes with timing configurations. We believe that a model for the cyber part of a CPS must allow for the specification of application timing behavior as an integral part of the model. Timing extensions for dataflow models accomplish this in a natural and comprehensible manner. By illustrating timing configurations for SDF, we lay the groundwork for their application to a variety of dataflow models.

Signal processing and multimedia applications are commonly modeled using Static/Cyclo-Static Dataflow (SDF/CSDF) models. SDF/CSDF explicitly specifies how much data is produced and consumed per firing during computation. This results in strong compile-time analyzability of many useful execution properties such as deadlock absence, channel boundedness, and throughput. However, SDF/CSDF is limited in its ability to capture how data is accessed in time. Hence, using these models often leads to implementations that are sub-optimal (i.e., use more resources than necessary) or even incorrect (i.e., use insufficient resources). In this work, we advance a new model called Static Dataflow with Access Patterns (SDF-AP) that captures the timing of data accesses (for both production and consumption). This paper formalizes the semantics of SDF-AP, defines key properties governing model execution, and discusses algorithms to check these properties under correctness and resource constraints. Results are presented to evaluate these analysis algorithms on practical applications modeled by SDF-AP.

FPGA devices provide flexible, fast, and low-cost prototyping and production solutions for system design. However, as the design complexity continues to rise, the design and synthesis iterations become a labor intensive and time consuming ordeal. Consequently, it becomes imperative to raise the level of abstraction for FPGA designs, while providing insight into performance metrics early in the design process. In particular, an important design time problem is to determine the maximum clock frequency that a circuit can achieve on a specific FPGA target before full synthesis and implementation. This early quantification can greatly help evaluate key design characteristics without reverting to tedious runs of the full implementation flow. In this work, we focus on the predictability of timing delay of circuits composed of high-level blocks on an FPGA. We are well aware of difficulties in tackling uncertainties in early timing estimation, e.g., an inherent gap between a high-level representation and gates/wires; extremely difficult delay estimation due to the randomness in physical design tools, etc. We show that the estimation uncertainties can be mitigated through a carefully characterized timing database of primitive building blocks and refined timing analysis models. We primarily focus on applications composed of data-intensive word-level arithmetic computations from the DSP domain and specified using static dataflow models. Our experiments indicate that for these applications, timing estimates can be obtained reliably within a good error margin on average and in the worst case. As future work, we plan to fine tune the timing database by modeling resource utilization effects and inter-primitive/actor routing delay via variants of Rent's rule and related efforts. We are also interested in exploring dynamic sub-cycle timing characterization.

We present a statistical optimization approach for scheduling a task dependence graph with variable task execution times onto a heterogeneous multiprocessor system. Scheduling methods in the presence of variations typically rely on worst-case timing estimates for hard real-time applications, or average-case analysis for other applications. However, a large class of soft real-time applications require only statistical guarantees on latency and throughput. We present a general statistical model that captures the probability distributions of task execution times as well as the correlations of execution times of different tasks. We use a Monte Carlo based technique to perform makespan analysis of different schedules based on this model. This approach can be used to analyze the variability present in a variety of soft real-time applications, including a H.264 video processing application.

The paper presents a decomposition strategy to speed up constraint optimization for a representative multiprocessor scheduling problem. In the manner of Benders decomposition, our technique solves relaxed versions of the problem and iteratively learns constraints to prune the solution space. Typical formulations suffer prohibitive run times even on medium-sized problems with less than 30 tasks. Our decomposition strategy enhances constraint optimization to robustly handle instances with over 100 tasks. Moreover, the extensibility of constraint formulations permits realistic application and resource constraints, which is a limitation of common heuristic methods for scheduling. The inherent extensibility, coupled with improved run times from a decomposition strategy, posit constraint optimization as a powerful tool for resource constrained scheduling and multiprocessor design space exploration

The H.264 decoder has a sequential, control intensive front end that makes it difficult to leverage the potential performance of emerging manycore processors. Preparsing is a functional parallelization technique to resolve this front end bottleneck. However, the resulting parallel macro block (MB) rendering tasks have highly input-dependent execution times and precedence constraints, which make them difficult to schedule efficiently on manycore processors. To address these issues, we propose a two step approach: (i) a custom preparsing technique to resolve control dependencies in the input stream and expose MB level data parallelism, (ii) an MB level scheduling technique to allocate and load balance MB rendering tasks. The run time MB level scheduling increases the efficiency of parallel execution in the rest of the H.264 decoder, providing 60% speedup over greedy dynamic scheduling and 9-15% speedup over static compile time scheduling for more than four processors. The preparsing technique coupled with run time MB level scheduling enables a potential 7times speedup for H.264 decoding.

Modern network applications require devices that provide high-performance at gigabit line rates with the flexibility to support diverse application standards and services. However, prohibitive product design costs and shrinking market windows restrict the number of ASIC/ASSP design starts to only selective application niches. The FPGA is an alternate cost-effective medium for many applications. However, creating a system solution through programming an FPGA with an HDL is only attractive to few application developers. Designers would prefer a software solution if it can meet their design constraints. An alterative is an FPGA-based soft multiprocessor system: a programmable multiprocessor on the FPGA composed of a network of hard and soft processing cores. Soft multiprocessors provide a software-level abstraction for programming an FPGA. In this study, we evaluate the feasibility and effectiveness of soft multiprocessor systems. We compare a soft multiprocessor implementation against FPGA hardware and an ASSP implementation for two network applications: IPv4 packet forwarding and Network Address Translation (NAT). Our study indicates that soft multiprocessors lose only a factor of 2X in performance compared to a custom ASSP, while providing great savings in terms of silicon development costs. Moreover, the presence of architectures and development environments for soft multiprocessor systems could open the FPGA market to the much larger world of embedded system software designers.

To realize high performance, embedded applications are deployed on multiprocessor platforms tailored for an application domain. However, when a suitable platform is not available, only few application niches can justify the increasing costs of an IC product design. An alternative is to design the multiprocessor on an FPGA. This retains the programmability advantage, while obviating the risks in producing silicon. This also opens FPGAs to the world of software designers. In this paper, we demonstrate the feasibility of FPGA-based multiprocessors for high performance applications. We deploy IPv4 packet forwarding on a multiprocessor on the Xilinx Virtex-II Pro FPGA. The design achieves a 1.8 Gbps throughput and loses only 2.6X in performance (normalized to area) compared to an implementation on the Intel IXP-28OO network processor. We also develop a design space exploration framework using integer linear programming to explore multiprocessor configurations for an application. Using this framework, we achieve a more efficient multiprocessor design surpassing the performance of our hand-tuned solution for packet forwarding.