We propose an efficient and scalable LLM decoding framework optimized for AMD Ryzen AI NPUs, leveraging two novel techniques: FusedDQP and FlowKV. FusedDQP fuses dequantization with projection to minimize memory operations and latency, while FlowKV introduces a pipelined, bandwidth-optimized approach for KV cache access across compute tiles (CT). Together, these methods deliver substantial improvements in both speed and energy efficiency without altering model accuracy. Our solution achieves up to 14.2× speedup and 2.66× power efficiency gains compared to existing state-of-the-art (SOTA) NPU baselines, demonstrating linear scalability with CT count and robustness across LLaMA-3.1/3.2 model variants (1B, 3B, and 8B parameters). We also benchmark against CPU and iGPU on the same platform, our performance surpasses CPU and iGPU (up to 1.8x and 16.2x speedup), while delivering substantially improved energy efficiency (up to 3.63x and 11.38x for CPU and iGPU, respectively).

We present a general and efficient dataflow framework for mapping Diffusion Transformer (DiT)-based models onto tiled mesh accelerators. The framework explicitly orchestrates tiling, streaming, and Direct Memory Access (DMA)-driven data movement to efficiently execute attention and large matrix multiplication (MM). We use the AMD Ryzen AI Neural Processing Unit (NPU) [1] as a representative edge-class tiled-mesh dataflow platform. In our implementation, the DiT denoising stage runs on the NPU, while the text encoder and Variational Autoencoder (VAE) decoder remain on the CPU.The iterative denoising loop dominates end-to-end latency. We therefore focus our acceleration and optimization efforts on the denoising module, while keeping the text-embedding and VAE-decoder stages on the CPU. Each denoising step reuses the same transformer backbone, and the dominant operations within a step are MM, self-attention, and MLP submodules that combine MM with elementwise nonlinear operations. The NPU is organized as a two-dimensional array of compute tiles (CTs), also referred to as AIEs. Our design partitions input matrices into tiles and maps independent tile computations across multiple AIE cores to exploit spatial parallelism.We evaluate the framework on two representative DiT-based text-to-image models, FLUX.1-schnell and Z-Image-Turbo [2], [3], and achieve end-to-end generation of a high-quality image in just over one minute. Compared with the integrated GPU (iGPU), the NPU delivers comparable generation latency while being approximately 4.4× more power efficient (package). For FLUX.1-schnell, the iGPU achieves 10.5 s/step, while the NPU achieves 13.3 s/step. For Z-Image-Turbo, the iGPU achieves 10.1 s/step, while the NPU achieves 9.8 s/step, showing that the NPU can match and slightly surpass iGPU denoising latency. These results translate into substantially improved energy efficiency for on-device image generation. Although demonstrated on a specific NPU and two representative models, the proposed dataflow framework is neither hardware-specific nor model-specific. It generalizes naturally to other diffusion-family models and mesh-based dataflow accelerators. Our results highlight the potential of spatial NPUs as energy-efficient platforms for next-generation on-device generative AI.Code Availability: The source code and setup instructions are available at: https://github.com/jia1217/vigenflow.

Hardware-in-the-Loop (HIL) simulation is a critical technique for validating embedded controllers in power-electronic systems such as electric vehicles and data centers, where switching frequencies can exceed 100kHz. Achieving real-time performance at these frequencies requires sub-microsecond simulation steps while maintaining sufficient numerical accuracy. Existing commercial HIL solutions predominantly rely on FPGA platforms due to their fine-grained timing control and deterministic execution. However, the emergence of spatial, dataflow-oriented accelerators raises the question of whether alternative architectures can meet these stringent requirements.

Stencil computation is an essential method, particularly useful for numerical simulations in areas like acoustics, heat transfer, and electromagnetism. Recent studies have utilized AMD AI Engines (AIEs) for stencil computations by configuring multiple AIE tiles within a Compute Unit to exploit task-level parallelism, achieving notable speedup through concurrent task execution. However, this setup suffers from suboptimal performance due to high memory bandwidth demands, resulting in underutilization of the available AIE tiles. This work introduces a Tile-Level Pipeline architecture designed for stencil computation that operates with constant memory bandwidth. This approach achieves linear scalability, where performance scales linearly with the number of AIE tiles, and ensures full utilization of all AIE tiles on the chip. We empirically demonstrate these benefits using AIEs.

Recurrent Neural Networks (RNNs) face challenges with the Back Propagation Through Time (BPTT) algorithm, leading to substantial computational and memory demands in training, especially on GPUs. Inspired by biological neural systems, we introduce the TwinStep Network (TwNet) via algorithm/architecture co-design, achieving online training via a neuron-centric design. At its core, TwinStep signifies that both the forward pass (inference) and back propagation (training) steps for each neuron happen concurrently. This approach, which more closely resembles biological neural processes, eliminates the necessity of storing the intermediate state of each neuron at each time steps, as required in BPTT. Consequently, it overcomes the limitation on the number of time steps that can be included in the BPTT training process. Uniquely, TwNet's “pipeline parallelism” facilitates serial processing and concurrent handling of multiple time steps. We implemented TwNet on FPGAs with a fully pipelined architecture. It achieves up to 885x speedup in training several popular RNN testbenches in comparison with other state-of-the-art approaches while maintaining accuracy, marking an advancement in online RNN training and potential applications.

This paper introduces an FPGA-enabled framework to accelerate the automated design process for Photonic Integrated Circuit (PIC) devices. PICs are foreseen as a foundation for the next-generation semiconductors. However, the complexity of PIC design presents considerable challenges. Machine Learning (ML) techniques have shown promise in the realm of PIC design. The primary hurdle, however, is the extended training duration, solely constrained by the slow electromagnetic (EM) Finite-Difference Time-Domain (FDTD) solver. We propose a fast framework with a dedicated FPGA FDTD accelerator tailor-designed to speed up the PIC simulation. Benchmarking was carried out against commercial tools, with the single-FPGA accelerator outperforming both a multicore CPU and a GPU cluster. We taped out and evaluated the PIC devices designed through the proposed framework, and the experimental outcomes aligned. This demonstrates the full design circle, showcasing that the proposed framework enabled by FPGA breaks the current bottleneck in this domain.

High-efficiency energy conversion systems have become increasingly important due to their wide use in all electronic systems such as data centers, smart mobile devices, E-vehicles, medical instruments, and so forth. Complex and interdependent parameters make optimal designs of power converters challenging to get. Recent research has shown that machine learning (ML) algorithms, such as reinforcement learning (RL), show great promise in design of such converter circuits. A trained RL agent can search for optimal design parameters for power conversion circuit topologies under targeted application requirements. Training an RL agent requires numerous circuit simulations. It requires significantly more training iterations when the tolerance of circuit components due to manufacturing inconsistency, aging, and temperature variation is considered. As a result, they may take days to complete, primarily because of the slow time-domain circuit simulation. This paper proposes a new FPGA architecture that accelerates the circuit simulation and hence substantially speeds up the RL-based design method for power converters. Our new architecture supports all power electronic circuit converters and their variations. It substantially improves the training speed of RL-based design methods. High-level synthesis (HLS) was used to build the accelerator on Amazon Web Service (AWS) F1 instance. An AWS virtual PC hosts the training algorithm. The host interacts with the FPGA accelerator by updating the circuit parameters, initiating simulation, and collecting the simulation results during training iterations. A script was created on the host side to facilitate this design method to convert a netlist containing circuit topology and parameters into core matrices in the FPGA accelerator. Experimental results showed$\mathbf{60}\times$overall speedup of our RL-based design method in comparison with using a popular commercial simulator, PowerSim.

This paper proposes a framework to substantially accelerate Reinforcement Learning (RL)-based design method for Photonic Integrated Circuit (PIC) devices. PICs are widely anticipated to underpin the forthcoming generation of semiconductor chips. However, the complexity of PIC design, which includes hundreds of degrees of freedom (DOF), presents considerable challenges. Machine Learning (ML) techniques, inclusive of RL, have demonstrated their effectiveness in the domain of PIC design. The primary hurdle, however, is the extended training duration, primarily constrained by the sluggish electromagnetic (EM) solver, specifically, the Finite-Difference Time-Domain (FDTD) solver. We have engineered a novel computational architecture that can be deployed on cloud-based systems using a cluster of Central Processing Units (CPUs), Field-Programmable Gate Arrays (FPGAs), and Graphics Processing Units (GPUs). An FPGA-FDTD accelerator, which capitalizes on the high memory bandwidth of on-chip memory (OCM), has been specifically designed to simulate planar PIC devices. Each FPGA-FDTD accelerator, also denoted as an FPGA kernel, functions as an autonomous RL environment. The host machine, in conjunction with the FPGA kernel, is designated as a worker node within the cluster. A functional prototype has been successfully implemented on the Amazon Web Services (AWS) cloud. The framework has effectively designed numerous PIC devices, and experimental results show the architecture outperforms existing methods significantly in design speed while maintaining or exceeding their design quality. Notably, the framework's versatility requires minimal adjustments for a broad range of devices, significantly reducing design time and promising to expedite PIC innovation.

This paper presents an FPGA cluster-based Finite-Difference Time-Domain (FDTD) accelerator that offers a linear speedup with the number of FPGAs participating in computation within the cluster. FDTD is a numeric method for simulating electromagnetic wave propagation and interactions with diverse materials and structures. Recent advancements in machine learning-based design and optimization techniques for photonic integrated circuits and microwave circuits, known as inverse design, have demonstrated remarkable success. Inverse design necessitates numerous FDTD simulations, and the high-performance FDTD accelerator enables rapid design automation, which is crucial for accelerating innovation. Our proposed accelerator comprises deeply pipelined FDTD cell update kernels that can traverse multiple FPGAs via high-speed optical links, effectively utilizing available resources across all FPGAs in a cluster. The architecture includes a head node and a flexible number of cascaded server nodes, together with custom cross-FPGA data routing kernels integrated into the "Open Cloud Testbed" (OCT) FPGA infrastructure to facilitate seamless data transfer. The proposed accelerator is developed on an existing platform, OCT FPGA. Our experiments reveal that, for a 4096×4096 2.5D FDTD simulation, each server node (Xilinx Alveo U280) can achieve 86.4 Giga-cells updates per second (GCUPS), and the head node can achieve 38.4 GCUPS. The overall speed with 4 server nodes is 38.4 + 4×86.4 = 384 GCUPS.

High-efficiency energy conversion systems have become increasingly important due to their wide use in all electronic systems such as data centers, smart mobile devices, E-vehicles, medical instruments, and so forth. Complex and interdependent parameters make optimal designs of power converters challenging to get. Recent research has shown that reinforcement learning (RL) shows great promise in the design of such converter circuits. A trained RL agent can search for optimal design parameters for power conversion circuit topologies under targeted application requirements. Training an RL agent requires numerous circuit simulations. As a result, they may take days to complete, primarily because of the slow time-domain circuit simulation.

The rapid growth of high-speed transceiver technology has met the demand for higher data rates in modern society. The field of optical communication has taken advantage of this growth by using these transceivers with small form-factor pluggable modules. This work demonstrates that these readily available, widely deployed, and commercialized modules can be converted into a state-of-the art software defined optical time-domain reflectometer (SD-OTDR). Enabled by the reconfigurable computing resource, the SD-OTDR can realize in-situ diagnostics of optical fiber without adding any overhead to existing systems. This software defined reflectometer can obtain sub-cm spatial resolutions with a measurement sensitivity on the order of -65dB. This is made possible by using the advantages offered by the reconfigurable fabric in modern System-on-Chip platforms often found in communication networks.

The emerging System-in-Package (SiP) technology has enabled multiple dies fabricated on a single chip for high performance and energy efficiency. Die-to-die (D2D) communication in SiP is typically unencrypted, exposing sensitive data to possible microprobing attacks. In this paper, we propose an on-chip microprobe detection circuit together with a noise canceling technique to protect D2D buses for future SiP security. The proposed method utilizes the metastable state of a flip-flop to detect the small timing variation caused by the inevitable loading effect of a microprobe. This design requires minimum digital resources with high scalability. Uniquely, the proposed design protects D2D buses at runtime without interfering with normal data transfers. In addition, it introduces zero latency to the communication channel. We built the detection circuit in a Xilinx ZYNQ Ultrascale+ SoC to prove its feasibility. Dynamic partial reconfiguration function is employed to create the test platform and emulate D2D interconnections as well as microprobing attacks on them. To demonstrate its potential to be used in standard communication protocols, we integrated the detection circuit with a fully functional Advanced eXtensible Interface (AXI) bus. Experimental results show that the proposed runtime detection method is effective, resource-efficient, and reliable under temperature-varying environments.

Currently available automotive radars are designed to stream real-time 2D image data over high-speed links to a central ADAS (Advance Driver-Assistance System) computer for object recognition, which considerably contributes to the system's power consumption and complexity. This paper presents a preliminary work for the implementation of a new in-sensor computer architecture to extract representative features from raw sensor data to detect and identify objects with radar signals. Such new architecture makes it possible to reduce the data transferred between sensors and the central ADAS computer significantly, giving rise to significant energy savings and latency reductions, while still maintaining sufficient accuracy and preserving image details. An experimental prototype has been built using the Texas Instruments AWR1243 Frequency-Modulated Continuous Wave (FMCW) radar board. We carried out experiments using the prototype to collect radar images, to preprocess raw data, and to transfer feature vectors to the central ADAS computer for classification and object detection. Two different approaches will be presented in this paper: First, a vanilla autoencoder will demonstrate the possibility of data reduction on radar signals. Second, a convolutional neural network based cross-domain deep learning architecture is presented by using a sample dataset to show the feasibility of computing Range-Angle Heatmaps directly on the sensor board eliminating the need for the raw data preprocessing on the central ADAS computer. We show that the reconstruction of Range-Angle Heatmaps can be predicted with a very high accuracy by leveraging deep learning architectures. Implementation of such a deep learning architecture on the sensor board can reduce the amount of data transferred from sensors to the central ADAS computer implying great potential for an energy efficient deep learning architecture in such environments.

The emerging System-in-Package (SiP) technology has enabled multiple dies fabricated on a single chip for high performance and energy efficiency. Die-to-die (D2D) communication in SiP is typically unencrypted, exposing sensitive data to possible microprobing attacks. This paper presents a new architecture design that protects D2D interconnection network from such microprobing attacks. Our new design provides instant detection of a microprobe at run time with no interference with normal data transfers and easily scalable to hundreds of D2D interconnection buses. The uniqueness of the new architecture is extremely simple and readily applicable to any system on a chip architecture. The trick is exploiting metastable state of a flip-flop (FF) to detect the small timing variation caused by the inevitable loading effect of a microprobe. To demonstrate its effectiveness and performance, a working prototype has been built on an emulation testbench in a Xilinx ZYNQ Ultrascale+ SoC to prove its feasibility. Dynamic partial reconfiguration (DPR) function is an enabler to create the test platform. DPR emulates both unprobed and probed scenarios on a D2D bus, and allow us to switch in between as required. To show our new design can be easily integrated to standard communication protocols, we implemented our prototype on a fully functional Advanced eXtensible Interface (AXI) bus. Experimental results show that the proposed runtime detection method is effective, resource-efficient, and reliable under temperature-varying environments.

It has been reported that physical probing on an off-chip bus can reveal confidential information in an electronic system. An attacker can use non-invasive and inexpensive electric probes (or interposers) to measure signals from circuit traces, such as the memory bus between the memory controller and a memory module. This paper describes a method to detect any bus probing/tampering by tracking the phase shift of output digital waveforms, induced by input impedance change at the bus transmitter (Tx). A low-overhead digital circuit based on flip-flop's metastability is built around the Tx using a field-programmable logic gate array (FPGA) to precisely measure the phase shift of output signals. Uniquely, the output data launched by the Tx is used as a stimulus signal, thus, the proposed method holds the advantage of detecting probing attacks at run-time. That is, the detection action operates in parallel with the normal data transfer on a bus without any interference, imposing zero latency to the communication channel. In order to show its feasibility in a real-world communication protocol, we implemented the proposed method in the DDR memory controller on an FPGA board (Xilinx ZCU104). The working prototype is able to protect a memory bus between the FPGA board and a DDR4 DIMM with a data rate of 2400MT/s. Experimental results show that the proposed method can be used to countermeasure interposer attacks, probing attacks, and cold boot attacks. We believe that the proposed method can be implemented in a variety of communication channels.

Analog to digital converters (ADCs) are indispensable nowadays. Analog signals are digitized earlier and earlier in the processing chain to reduce the need for complex analog signal processing. For this reason, ADCs are often integrated directly into field-programmable gate arrays (FPGA) or microprocessors. However, such ADCs are designed for a specific set of requirements with limited flexibility. In this paper, a new structure of an FPGA-based ADC is proposed. The ADC is based on the slope ADC, where a time-to-digital converter (TDC) measures the time from the beginning of a reference slope until the slope reaches the voltage-to-be-measured. Only FPGA-internal elements are used to build the ADC. It is fully reconfigurable and does not require any external components. This innovation offers the flexibility to convert almost any digital input/output (I/O) into an ADC. Considering the very high number of digital I/O ports available in today's FPGA systems, this enables the construction of a massive and powerful ADC array directly on a standard FPGA. The proposed ADC has a resolution of 9.3 bit and achieves an effective number of bits (ENOB) of 7 at a sample rate of 600 MSample/s. The differential nonlinearity (DNL) ranges from -0.9 to 0.9 bit, and the integral nonlinearity (INL) is in the range between -1.1 and 0.9 bit. An alternative version of the ADC operates at 1.2 GSample/s and achieves an ENOB of 5.3.

Tamper-proof hardware designs present a great challenge to computer architects. Most existing research limits hardware trusted computing base (TCB) to a CPU chip and anything off the CPU chip is vulnerable to probing and tampering. This paper introduces a new hardware design that provides strong defenses against physical attacks on interconnecting buses between chips in a computer system thereby extending the hardware TCB beyond CPU chips. The new approach is referred to as DIVOT: Detecting Impedance Variations Of Transmission-lines (Tx-lines). Every Tx-line in a computer system, such as a bus and interconnection wire has a unique, intrinsic, and fingerprint-like property: Impedance Inhomogeneity Pattern (IIP), i.e. the impedance distribution over distance. Such unpredictable, uncontrollable, and non-reproducible IIP fingerprints can be used to authenticate a Tx-line to ensure the confidentiality and integrity of data being transmitted. In addition, physical probes perturb the electromagnetic (EM) field around a Tx-line, leading to an altered IIP. As a result, runtime monitoring of IIPs can also be used to actively detect physical probing, snooping, and wire-tapping on buses. While the physics behind the IIP is known, the major technical breakthrough of DIVOT is the new integrated time domain reflectometer, iTDR, that is capable of carrying out in-situ and runtime monitoring of a Tx-line without interfering with normal data transfers. The iTDR is based on two innovations: analog-to-probability conversion (APC) and probability density modulation (PDM). The iTDR performs runtime IIP measurements noninvasively and is CMOS-compatible allowing it to be integrated with any interface logic connected to a bus. DIVOT is a generic, scalable, cost-effective, and low-overhead security solution for any computer system from servers to embedded computers in smart mobile devices and IoTs. To demonstrate the proposed architecture, a working prototype of DIVOT has been built on an FPGA as a proof of concept. Experimental results clearly showed the feasibility and performance of DIVOT for both hardware authentication and tamperproof applications. More specifically, the probability of correctly identifying a bus is close to 1 with an equal error rate (EER) of less than 0.06% at room temperature. We present an example design that incorporates DIVOT into an off-chip memory bus to protect against physical attacks including probing/snooping, tampering, and cold boot attacks.

Data intensive analysis is the major challenge in smart cities because of the ubiquitous deployment of various kinds of sensors. The natural characteristic of geodistribution requires a new computing paradigm to offer location-awareness and latency-sensitive monitoring and intelligent control. Fog Computing that extends the computing to the edge of network, fits this need. In this paper, we introduce a hierarchical distributed Fog Computing architecture to support the integration of massive number of infrastructure components and services in future smart cities. To secure future communities, it is necessary to integrate intelligence in our Fog Computing architecture, e.g., to perform data representation and feature extraction, to identify anomalous and hazardous events, and to offer optimal responses and controls. We analyze case studies using a smart pipeline monitoring system based on fiber optic sensors and sequential learning algorithms to detect events threatening pipeline safety. A working prototype was constructed to experimentally evaluate event detection performance of the recognition of 12 distinct events. These experimental results demonstrate the feasibility of the system's city-wide implementation in the future.

We introduce a new parallel computing and communication architecture, Reflex-Tree, with massive sensing, data processing, and control functions suitable for future smart cities. The central feature of the proposed Reflex-Tree architecture is inspired by a fundamental element of the human nervous system: reflex arcs, the neuromuscular reactions and instinctive motions of a part of the body in response to urgent situations. At the bottom level of the Reflex-Tree (layer 4), novel sensing devices are proposed that are controlled by low power processing elements. These "leaf" nodes are then connected to new classification engines based on machine learning techniques, including support vector machines (SVM), to form the third layer. The next layer up consists of servers that provide accurate control decisions via multi-layer adaptive learning and spatial-temporal association, before they are connected to the top level cloud where complex system behavior analysis is performed. Our multi-layered architecture mimics human neural circuits to achieve the high levels of parallelization and scalability required for efficient city-wide monitoring and feedback. To demonstrate the utility of our architecture, we present the design, implementation, and experimental evaluation of a prototype Reflex-Tree. City power supply network and gas pipeline management scenarios are used to drive our prototype as case studies. We show the effectiveness for several levels of the architecture and discuss the feasibility of implementation.