Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings [1, 5]. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long data acquisition times and computationally demanding data processing. While deep learning has reduced post-processing times, time-resolved data acquisition remains a bottleneck for real-time applications. To address this,we propose a method to achieve near realtime FLI using an FPGA-based hardware accelerator. Specifically,we implemented a GRU-based sequence-to-sequence (Seq2Seq) model [3] on an FPGA board compatible with time-resolved cameras. The GRU model balances accurate processing with the resource constraints of FPGAs, which have limited DSP units and BRAM. The limited memory and computational resources on the FPGA require efficient scheduling of operations and memory allocation to deploy deep learning models for low-latency applications.We address these challenges by using STOMP, a queue-based discrete-event simulator that automates and optimizes task scheduling and memory management on hardware [4]. By integrating a GRU-based Seq2Seq model and its compressed version, called Seq2SeqLite [2], generated through knowledge distillation (KD), we were able to process multiple pixels in parallel, reducing latency compared to sequential processing. We explore various levels of parallelism to achieve an optimal balance between performance and resource utilization. Our results indicate that the proposed techniques achieved a 17.7x and 52.0x speedup over manual scheduling for the Seq2Seq model and the Seq2SeqLite model, respectively. Furthermore, the Seq2SeqLite model achieved a 98% reduction in parameters compared to the Seq2Seq model.

Heterogeneous systems-on-chips (SoCs) are pivotal for real-time applications like autonomous driving, as they blend the versatility of CPUs with the efficiency of accelerator IPs. However, evolving application demands necessitate domain-specific SoCs to meet real-time deadlines within strict power and area constraints. While prior research focused on microarchitectural optimizations, overlooking broader system-level considerations can lead to suboptimal design decisions. Thus, there is a need to elevate the abstraction level of design space exploration (DSE) to the SoC level. However, SoC-level DSE is challenging due to the vast design space, encompassing microarchitectural parameters and dynamic task-to-hardware mapping choices based on runtime characteristics and real-time constraints. This paper proposes a systematic and agile methodology, called ARTEMIS, for efficient DSE of real-time, domain-specific SoCs that are constrained by task deadlines, power, and area. The core concept involves integrating a dynamic SoC scheduler to reduce the design space by eliminating the mapping dimension. Enhanced scheduling policies, incorporating techniques like task procrastination and memory-traffic/energy awareness, expedite navigation through the pruned design space. Additionally, DSE heuristics are optimized with real-time deadline and power/areaaware ranking mechanisms. ARTEMIS is evaluated on autonomous vehicle (AV) and augmented/virtual reality (AR/VR) applications, and additionally validated on an FPGA. Compared to the state-of-the-art, DSE using ARTEMIS converges 5.1$12.8 \times$ faster, while yielding SoCs that meet $100 \%$ real-time deadlines with $1.2-3 \times$ better throughput at iso-area or up to $2.4 \times$ lower area for at iso-input-rate. ARTEMIS thus enables DSE of large designs with tractable simulation resources, without compromising on the power-performance-area metrics of the explored SoC design.

Fully homomorphic encryption (FHE) is a powerful cryptographic technique that enables computation on encrypted data without needing to decrypt it. It has broad applications in scenarios where sensitive data needs to be processed in the cloud or in other untrusted environments. FHE applications are both compute- and memory-intensive, owing to expensive operations on large data. While prior works address the challenges of efficient compute using dedicated hardware, expensive memory transfers still remain a major limiting factor. In this work, we propose a hierarchical near-DRAM processing (NDP) solution for FHE applications, called FHENDI, that harnesses the massive DRAM bank bandwidth. We observe various data access patterns in FHE that reveal distinct levels of parallelism: element-wise, limb-wise, coefficient-wise, and ciphertext-wise. FHENDI exploits these levels of parallelism to map FHE operations and data onto different hierarchies of our design, while addressing three major challenges with NDP for FHE: (i) the lack of bank-to-bank communication support, (ii) limited die-to-die bandwidth, and (iii) large memory access latencies. We resolve the first problem through a novel, conflict-free mapping algorithm built atop localized permutation networks that enables efficient element-wise and butterfly operations in FHE. The second problem is addressed by pipelining the execution of parallel bootstrap operations observed in compiled FHE workloads. Finally, we hide the memory access latency behind computation latency by exploiting a dual-banking scheme and subarray-level parallelism (SLP) of the DRAM banks. We evaluate FHENDI using representative workloads in the domains of privacy-preserving machine learning inference on CNNs and Transformers, database range query, and sorting, that are obtained using a compiler framework called HElayers. We compare FHENDI with a server-class CPU and GPU running the state-of-the-art HEaaN library, and an FHE accelerator ASIC, and report mean speedups of $2145.8 \times, 118.29 \times$, and $2.45 \times$, respectively.

Monolithic 3D technology is emerging as a promising solution that can bring massive opportunities, but the gains can be hindered due to the reliability issues exaggerated by high temperature. Conventional reliability solutions focus on one specific feature and assume that the other required features would be provided by different solutions. Hence, this assumption has resulted in solutions that are proposed in isolation of each other and fail to consider the overall compatibility and the implied overheads of multiple isolated solutions for one system. This article proposes a holistic reliability management engine, R2D3, for post-Moore’s M3D parallel systems that have low yield and high failure rate. The proposed engine, comprising a controller, reconfigurable crossbars, and detection circuitry, provides concurrent single-replay detection and diagnosis, fault-mitigating repair, and aging-aware lifetime management at runtime. This holistic view enables us to create a solution that is highly effective while achieving a low overhead. Our solution achieves 96% coverage of defect; reduces V th degradation by 53%, leading to a 78% performance improvement on average over 8 years for an eight-core system; and ultimately yields a 2.16× longer mean-time-to-failure (MTTF) while incurring an overhead of 7.4% in area, 6.5% in power, and an 8.2% decrease in frequency.

Dynamic adaptation is a post-silicon optimization technique that adapts the hardware to workload phases. However, current adaptive approaches are oblivious to implicit phases that arise from operating on irregular data, such as sparse linear algebra operations. Implicit phases are short-lived and do not exhibit consistent behavior throughout execution. This calls for a high-accuracy, low overhead runtime mechanism for adaptation at a fine granularity. Moreover, adopting such techniques for reconfigurable manycore hardware, such as coarse-grained reconfigurable architectures (CGRAs), adds complexity due to synchronization and resource contention.

The advancement of Silicon CMOS technology has led information technology innovation for decades. However, scaling transistors down according to Moore’s law is almost reaching its limitations. To improve system performance, cost, and energy efficiency, vertical-optimization in multiple layers of the computing stack is required. Technological awareness in terms of devices and circuits could enable informed system-level decisions. For example, graphene is a promising material for extremely scaled high-speed transistors because of its remarkably high mobility, but it can not be used in integrated circuits as a result of the high leakage current from its zero bandgap. In this article, we discuss the fundamental physics of transistors and their ramifications on system design to assist device-level technology consideration during system design. Additionally, various emerging devices and their utilization on a vertically-optimized computing stack are introduced. This article serves as a survey of emerging device technologies that may be relevant in these areas, with an emphasis on making the descriptions approachable by system and software designers to understand the potential solutions. A basic vocabulary will be built to understand how to digest technical content, followed by a survey of devices, and finally a discussion of the implications for future processing systems.

With the end of Dennard scaling and Moore's law, it is becoming increasingly difficult to build hardware for emerging applications that meet power and performance targets, while remaining flexible and programmable for end users. This is particularly true for domains that have frequently changing algorithms and applications involving mixed sparse/dense data structures, such as those in machine learning and graph analytics. To overcome this, we present a flexible accelerator called Transmuter, in a novel effort to bridge the gap between General-Purpose Processors (GPPs) and Application-Specific Integrated Circuits (ASICs). Transmuter adapts to changing kernel characteristics, such as data reuse and control divergence, through the ability to reconfigure the on-chip memory type, resource sharing and dataflow at run-time within a short latency. This is facilitated by a fabric of light-weight cores connected to a network of reconfigurable caches and crossbars. Transmuter addresses a rapidly growing set of algorithms exhibiting dynamic data movement patterns, irregularity, and sparsity, while delivering GPU-like efficiencies for traditional dense applications. Finally, in order to support programmability and ease-of-adoption, we prototype a software stack composed of low-level runtime routines, and a high-level language library called TransPy, that cater to expert programmers and end-users, respectively.

While systolic arrays are widely used for dense-matrix operations, they are seldom used for sparse-matrix operations. In this paper, we show how a systolic array of Multiply-and-Accumulate (MAC) units, similar to Google's Tensor Processing Unit (TPU), can be adapted to efficiently handle sparse matrices. TPU-like accelerators are built upon a 2D array of MAC units and have demonstrated high throughput and efficiency for dense matrix multiplication, which is a key kernel in machine learning algorithms and is the target of the TPU. In this work, we employ a co-designed approach of first developing a packing technique to condense a sparse matrix and then propose a systolic array based system, Sparse-TPU, abbreviated to STPU, to accommodate the matrix computations for the packed denser matrix counterparts. To demonstrate the efficacy of our co-designed approach, we evaluate sparse matrix-vector multiplication on a broad set of synthetic and real-world sparse matrices. Experimental results show that STPU delivers 16.08X higher performance while consuming 4.39X and 19.79X lower energy for integer (int8) and floating point (float32) implementations, respectively, over a TPU baseline. Meanwhile, STPU has 12.93% area overhead and an average of 4.14% increase in dynamic energy over the TPU baseline for the float32 implementation.

Various emerging technologies have shown great potential of supplementing silicon transistors as Moore's law slows down. One such disruptive technology, carbon nanotube field-effect transistors (CNFETs), is one of the most promising competing technologies available, offering exceptional electrostatic properties. Furthermore, their low-temperature manufacturing process and low power consumption make these devices perfect candidates for 3D integration. However, due to the infancy of their manufacturing process, high defect densities, and variation issues, chip designers are not encouraged to consider these emerging technologies as a stand-alone replacement for Si-based transistors. Hence, to commercialize these new technologies, new architectural and circuit modifications that can work around high-fault rates are required, improving performance comparable to Silicon, while the manufacturing process is perfected.This paper proposes a design flow framework that can be used for large-scale chip production while mitigating yield and variation failures to bring up CNT-based technology, using a reliable reconfigurable architecture. The proposed framework can efficiently support high-variation technologies by providing protection against manufacturing defects at multiple granulari-ties: module and pipeline-stage levels.To incorporate different CNT-based transistor manufacturing processes, this work builds a flexible variation model and a CMOS-based CNT design library that can be used to synthesize physical CNFET-based processor designs over a range of 0.4 to 0.7 V. Based on the variation observed in the synthesized design, a reliable CNT-based 3D multi-granular reconfigurable architecture, 3DTUBE, is presented to overcome the manufacturing difficulties in the technology. For 0.4 V to 0.7 V, 3DTUBE provides up to 6.0× higher throughput and up to 3.1× lower Energy-Delay Product compared to a silicon-based multi-core design evaluated at 1 ppb transistor failure rate, which is 10,000× lower in comparison to CNFETs failure rate.

Sparse matrices are widely used in graph and data analytics, machine learning, engineering and scientific applications. This paper describes and analyzes OuterSPACE, an accelerator targeted at applications that involve large sparse matrices. OuterSPACE is a highly-scalable, energy-efficient, reconfigurable design, consisting of massively parallel Single Program, Multiple Data (SPMD)-style processing units, distributed memories, high-speed crossbars and High Bandwidth Memory (HBM). We identify redundant memory accesses to non-zeros as a key bottleneck in traditional sparse matrix-matrix multiplication algorithms. To ameliorate this, we implement an outer product based matrix multiplication technique that eliminates redundant accesses by decoupling multiplication from accumulation. We demonstrate that traditional architectures, due to limitations in their memory hierarchies and ability to harness parallelism in the algorithm, are unable to take advantage of this reduction without incurring significant overheads. OuterSPACE is designed to specifically overcome these challenges. We simulate the key components of our architecture using gem5 on a diverse set of matrices from the University of Florida's SuiteSparse collection and the Stanford Network Analysis Project and show a mean speedup of 7.9× over Intel Math Kernel Library on a Xeon CPU, 13.0× against cuSPARSE and 14.0× against CUSP when run on an NVIDIA K40 GPU, while achieving an average throughput of 2.9 GFLOPS within a 24 W power budget in an area of 87 mm2.

With silicon-based transistors approaching their scaling limits, multiple successor technologies are competing for silicon's place. Due to recent fabrication breakthroughs, one promising alternative is the carbon nanotube field-effect transistor (CNTFET), which uses carbon nanotubes as the channel medium instead of silicon. Although logic gates using CNTFETs have been demonstrated to provide up to an order of magnitude better energy-delay product (EDP) over silicon-based counterparts, system-level design using CNTFETs show significantly smaller EDP improvement because of the critical path of the design, output load capacitance and corresponding drive strengths of gates. In this paper, we address this challenge by exploring various architectural design choices using CNTFET-based pass transistor logic (PTL) and create an energy-efficient RISC-V processor. While silicon-based design traditionally prefers complementary logic over PTL, CNTFETs are ideal candidates for PTL due to their low threshold voltage, low power dissipation, and equal strength p-type and n-type transistors. By utilizing PTL to design modules that lie on the processor's critical path, systems can efficiently exploit CNTFET's potential benefits. Our results show that while a CNTFET RISC-V processor using complementary logic achieves a 2.9× EDP improvement over a silicon design, using PTL along the critical path components in the ALU can boost EDP improvement 5× as well as reduce area by 17% over 16 nm silicon CMOS.