Hardware accelerators such as GPUs, NPUs, and FPGAs are essential to meeting AI’s computational demands. With the proliferation of heterogeneous devices across cloud and edge, various model optimization techniques adapt to diverse hardware characteristics through operator transformations and structural modifications. Accurate, efficient latency prediction enables rapid selection of optimal strategies across hardware backends. Many existing methods treat hardware as a black-box executor, directly regressing latency without explicitly modeling the intricate interactions between neural network (NN) structures and device-specific execution behaviors. To address these challenges, we introduce a new modeling perspective that captures the interaction between neural architectures and hardware execution. To capture device-specific characteristics, we propose two complementary modeling strategies. The Device Behavior Signature Selector (DBSel) characterizes hardware execution behavior by selectively probing a small set of representative architectures, forming a compact, workload-driven profile. In parallel, we construct capability vectors that capture the hierarchical memory of each device and compute characteristics, providing a structured abstraction of its architectural capacity. To unify both behavioral and structural views, we introduce the Hardware–Operation Dialogue Module (HODM), which models fine-grained interactions between neural operators and hardware properties. Together, these components empower CloserToMe to deliver accurate and transferable latency predictions across unseen and diverse platforms.

Low-bitwidth multiply-accumulate (MAC) operations are fundamental to efficient hardware acceleration of recent neural networks. Existing approaches often fail to fully exploit the potential of digital signal processing (DSP) blocks in FPGAs. They struggle to balance the high-bitwidth computational capabilities of DSPs with the low-precision quantization requirements of neural networks. DSP packing consolidates multiple low-precision operations into a single DSP unit, significantly enhancing MAC efficiency. However, current DSP packing solutions suffer from limited support for continuous operations, poor adaptation to different neural network computation patterns, and complex software-hardware deployment workflows. These issues result in insufficient utilization of DSP resources, limiting the overall effectiveness of acceleration.

Scoring models such as click-through rate (CTR) prediction underpin recommendation and advertising systems, but their features are highly sensitive, making plaintext cloud inference risky. Fully homomorphic encryption (FHE) enables inference directly on ciphertexts, yet homomorphic computation is expensive and bootstrapping often dominates end-to-end latency. We present HE-DeepFM, a FHE inference system for CTR prediction. We first design HE-FM, a homomorphic-friendly Factorization Machine (FM) operator that exploits CKKS SIMD packing to compute second-order interactions efficiently, thereby avoiding the naive O(F2) cost of pairwise feature interactions. Building on HE-FM, HE-DeepFM reduces bootstrapping under the same FHE budget, and can eliminate it for small model configurations. We implement HE-DeepFM with Orion and Lattigo and evaluate it on real-world datasets. On Criteo, HE-DeepFM reduces bootstrapping from 12 to 4 and achieves up to 2.85× end-to-end speedup while maintaining prediction quality comparable to the baseline.

Emerging FPGA devices are widely used in graph processing and achieve high performance by exploiting fine-grained parallelism and high-bandwidth memory (HBM) subsystems with dozens of channels. However, the graph size increases rapidly and often exceeds the capacity of the FPGA device or one of the memory channels, incurring out-of-memory (OoM) faults and performance degradation. Existing approaches focus on improving the performance of on-device full graph processing, ignoring the support for hyper-scale graphs and cross-iteration acceleration. In this work, we propose SoGraph, a graph processing architecture for large graph acceleration on HBM-equipped FPGAs, cooperating with the host. To minimize PCIe traffic during the processing, we adopt three hardware-oriented optimizations: 1) activeness-driven subgraph scheduler, which minimizes on-device graph resident set; 2) lightweight state-aware processing extension in edge-centric paradigm; 3) bit-level update analyzer for to-be-transfer subgraph reduction. The results show that the prototype on Alveo U280 FPGA achieves an average of 22.3x performance speedup over the modified state-of-the-art FPGA design and 1.3x device energy efficiency over the GPU solution.

Tensorized programs leverage hardware intrinsics on accelerators to boost tensor computation performance. With the rise of hardware customization, massive accelerators and intrinsics have emerged, posing engineering challenges for manually coding efficient tensorized programs. Tensorized program tuning in deep learning compilers (DLCs) is considered an effective approach. At the core of program tuning relies the design of the cost model to predict performance. However, there is currently a lack of cost models specifically for tensorized programs, which severely hinders the co-optimization of deep learning compilers and hardware accelerators.In this paper, we propose UniCoX, a unified cost model specifically for tensorized program tuning across ubiquitous accelerators. We systematically analyze the design challenges introduced by tensorized programs from the perspectives of feature representation and transfer prediction. For feature representation, we leverage attention mechanisms to mine key software schedule features and design corresponding aligned hardware features, resulting in a unified cross-accelerator feature representation. For transfer prediction, by integrating lifelong learning and transfer learning with data sampling strategies, we propose a unified transfer prediction strategy to keep pace with the rapid development of accelerators. To meet training and testing demands, we construct TensorizeSetX, a dataset dedicated to tensorized program tuning. Results show that UniCoX achieves the state-of-the-art accuracy while supporting low-cost and flexible transfer prediction. It can accelerate search time by 11.3’ and improve inference speed by 1.9’ within the state-of-the-art tensorized program tuning framework, TVM MetaSchedule.

Sparse tensor computation is widely used in deep learning and scientific computing. However, diverse sparse data and algorithmic characteristics at the application level, combined with the diversity of hardware platforms, pose significant challenges for efficient sparse tensor program optimization. Manually crafted operator libraries are time-consuming to develop and lack portability. To address this, we propose UniSparTa, a unified sparse tensor program tuning framework that automatically generates high-performance programs. First, we extract unified optimization principles for high-performance sparse tensor programs and propose a domain-specific language (DSL) to automatically generate a high-quality design space without manual intervention. Second, by analyzing the general distribution of the design space, we introduce an adaptive search strategy combining Deep Q-Networks (DQN) and Simulated Annealing (SA). Finally, to avoid the unacceptable time cost of real measurement during tuning, we propose a unified cost model based on multimodal fusion to accurately predict program performance. Furthermore, by leveraging data augmentation and transfer learning, we enable low-cost transfer prediction across different sparse data patterns, algorithms, and hardware platforms. Results show that, compared to the state-of-the-art operator library MKL, the manually optimized scheme ASpT, the tensor compiler TVM, and the sparse tensor tuning framework WACO, UniSparTa achieves average speedups of 1.98×, 2.75×, 6.13×, and 1.75×, respectively. Moreover, UniSparTa significantly accelerates the tuning process.

The remarkable performance of Large Language Models (LLMs) has driven their widespread deployment in data centers to support diverse user-facing applications. However, the rapidly growing computational and storage demands of these models have made single-device deployment increasingly impractical. Prior research on LLM inference has primarily addressed this challenge through algorithmic optimizations such as quantization or by integrating customized hardware acceleration frameworks. As model parameters continue to scale, multi-device deployment has become a necessary approach for enabling efficient LLM inference. Nevertheless, constructing low-latency multi-device platforms for LLMs inference using available FPGA or GPU accelerators remains constrained by inefficient synchronization schemes or limited compute intensity in current architectures. Furthermore, existing solutions often lack co-optimized designs that effectively integrate communication with computation. To address these limitations, this paper proposes LORA, a low-latency end-to-end LLMs acceleration platform utilizing multiple FPGAs. Firstly, we optimize the synchronization timing within the LLMs to minimize storage, computation, and BRAM overhead. Secondly, we tightly couple communication and computation through techniques such as pipeline overlapping and input data packing. Next, we deploy homogeneous accelerators on each FPGA device, leveraging a recurrent architecture to further reduce inference latency. Finally, we apply FPGA-specific optimizations and conduct performance modeling and analysis of the acceleration framework to select optimal deployment parameters for various computational tasks. Implemented on Xilinx Alveo U280 FPGAs, LORA-F and LORA-Q achieve average speedups of 14.4× and 32.6×, respectively, compared to NVIDIA V100 GPUs when running modern LLMs. Compared with existing multi-FPGA accelerator platforms, LORA-F and LORA-Q demonstrate average performance improvements of up to 2.6× and 4.3×, respectively.

Field-Programmable Gate Arrays (FPGAs) are increasingly used in cloud platforms to accelerate diverse workloads, thanks to their reconfigurability and high performance. However, in multi-tenant cloud environments, existing FPGA virtualization mechanisms fail to align with dynamic application demands due to their static partitioning methods, leading to significant internal fragmentation. To address these issues, we present Tetris , an FPGA virtualization framework that supports flexible and dynamic reconfigurable resource allocation to improve cloud platform deployment efficiency. Specifically, Tetris deploys nested dynamic reconfigurable regions on FPGAs and adopts a tree structure to manage reconfigurable resources, enabling fine-grained resource reallocation at runtime. Enabled by the proposed fat-tree-based data transmission architecture between reconfigurable regions, Tetris can map dataflow-based applications onto these regions effectively. Additionally, Tetris offers dual-level resource optimization strategies to help system to balance the resource utilization and run-time compilation overhead. We evaluate Tetris with dataflow HLS benchmarks. Experimental results show that Tetris achieves a 1.16 \(\times\) improvement in resource utilization compared to advanced FPGA virtualization frameworks and delivers a 1.3 \(\times\) increase in run-time throughput, incurring less than 30% additional compilation latency. Tetris enables scalable, on-demand cloud FPGA acceleration, improving adaptability across different workloads.

Compared to traditional Vision Transformers (ViT), Mixture-of-Experts ViTs (MoE-ViTs) are introduced to scale model size without a proportional increase in computational complexity, making them a new research focus. Given the high performance and reconfigurability, field-programmable gate array (FPGA)-based accelerators for MoE-ViTs emerge, delivering substantial gains over general-purpose processors. However, existing accelerators often fall short of efficiently managing the highly dynamic and sparse computation patterns, resulting in suboptimal tradeoffs between resource utilization and performance. To address the inefficiencies in deploying MoE-ViTs on FPGAs, we present MoE-Sched, a novel end-to-end accelerator that embraces a scheduling-centric design philosophy. Rather than optimizing isolated kernels, MoE-Sched coordinates multilevel scheduling, from fine-grained intrakernel streaming to module reuse and multidie mapping, to holistically balance latency, bandwidth (BW), and resource usage. We further integrate a hardware-aware quantization scheme tailored for streaming attention and sparse expert execution, preserving accuracy while minimizing overhead. Experimental results demonstrate that our accelerator achieves nearly 100 frames/s on M3ViT-tiny, a$3.13\times $improvement in throughput, and over 75% energy reduction compared to state-of-the-art (SOTA) FPGA MoE accelerators, while maintaining less than 1% accuracy loss across vision benchmarks. Our implementation will be open-sourced.

Hybrid Homomorphic Encryption (HHE) scheme integrates two types of Fully Homomorphic Encryption (FHE), arithmetic FHE and logic FHE to enhance the performance and scalability of privacy-preserving computations. However, the performance of HHE mainly depends on the efficiency of the Number Theoretic Transform (NTT). Accordingly, this paper introduces Hermes, an FPGA-based NTT accelerator for HHE. We have designed a cross-scheme-friendly NTT architecture that supports NTT of varying lengths through the reuse of NTT units. Experimental results demonstrate that our proposed architecture achieves high hardware utilization and increases throughput by 1.3× compared to existing state-of-the-art approaches across various NTT lengths.

Dynamic workloads running on multiple hosts will bring changing access patterns on CXL-enabled shared disaggregated memory. Existing works often un-traceably cache multi-source accesses, making it hard to exploit each host’s access behavior and assure service quality. Our solution Alchemy jointly optimizes cache replacement and bypassing and runs as an online reinforcement learning agent with source-aware adaptivity. It gives rewards derived from sampling-based action effectiveness and per-host macro performance. The multi-host prototype-based results on FPGAs show $8.71 \%-14.56 \%$ reduction in average access latency over LRU policy and 44x faster than the hardware-efficient ICGMM method in decision-making with comparable overhead.

Current algorithm-hardware co-search works often suffer from lengthy training times and inadequate exploration of hardware design spaces, leading to suboptimal performance. This work introduces UniCoS, a unified framework for co-optimizing neural networks and accelerators for CNNs and Vision Transformers (ViTs). By introducing a novel training-free proxy that evaluates accuracy within seconds and a clustering-based algorithm for exploring heterogeneous dataflows, UniCoS efficiently navigates the design spaces of both architectures. Experimental results demonstrate that the solutions generated by UniCoS consistently surpass state-of-the-art (SOTA) methods (e.g., $3.54 \times$ energy-delay product (EDP) improvement with a $1.76 \%$ higher accuracy on ImageNet) while requiring notably reduced search time (up to $48 \times, \sim 3$ hours). The code is available at https://github.com/mine7777/Unicos.git.

This paper presents FastNN, a novel accelerator architecture for efficient K-Nearest Neighbors (KNN) search in point clouds. FastNN leverages a locality-sensitive E2LSH partitioning method and a precomparator module to significantly reduce the candidate search space and minimize the number of Euclidean distance calculations. Compared to octree-based partitioning methods, our approach reduces candidate points by 58.57% to 86.17% and achieves a $10.04 \times$ acceleration in processing throughput relative to the BitNN comparator subsystem. The proposed design effectively enhances search throughput, resource utilization, and precision, highlighting its potential for accelerating KNN search on FPGA platforms.

Exploring bit-level sparsity in the MAC process has been proven to be an important method for improving the efficiency of neural network feedforward processing. The reconfigurable platform offers possibilities for identifying the bitlevel unstructured redundancy during inference with different DNN models. Researchers noticed significant progress in valueaware accelerators on ASICs, yet we are concerned about the few studies on FPGAs. This paper observed the limitations of implementing bit-level sparsity optimizations using FPGA and proposed a software/architecture co-design solution. Specifically, by introducing LUT-friendly encoding with adaptable granularity and hardware structure supporting multiplication time uncertainty, we achieved a better trade-off between potential redundancy and accuracy with compatibility and scalability. Experiments show that under accurate calculation, PEs are up to $2.2 \times$ smaller than bit-parallel ones, and our design boosts performance by $1.04 \times$ to $1.74 \times$ and $1.40 \times$ to $2.79 \times$ over bitparallel and Booth-based designs, respectively.

Sparse tensor computing is widely used in deep learning and scientific computing, but its irregularity challenges GPU/NPU acceleration. FPGAs, with their reconfigurability, are well-suited for sparse workloads. However, current point-wise, manual design methodologies significantly limit the performance potential of reconfigurable hardware across diverse sparse scenarios. This paper proposes a compiler framework that automatically generates high-performance sparse accelerators on FPGA. It includes: (1) a schedule-primitive-based DSL serving as sparse accelerator design specification, covering a wide range of design concerns; (2) a two-stage mapping mechanism based on sparse meta-operations, enabling flexible mapping from arbitrary sparse dataflow to hardware microarchitectures; and (3) a heuristic design space exploration strategy guided by workload characteristics.

Transformers have revolutionized natural language processing (NLP) and computer vision (CV) tasks, yet their deployment remains constrained by the quadratic complexity of self-attention. While FPGA-based accelerators offer promising solutions, existing designs struggle with fixed dataflow patterns and limited hardware specialization across diverse model architectures and sequence lengths. This paper presents AutoTrans, an automated framework for generating optimized FPGA accelerators tailored to Transformers. To improve dataflow flexibility, we propose configurable fusion granularities at the tensor, row, and block levels for self-attention layers, enabling fine-grained trade-offs between computational efficiency and resource utilization. For hardware specialization, we develop a parameterized heterogeneous multi-core architecture featuring dedicated compute engines for attention and linear layers, guided by a genetic algorithm-based design space exploration (DSE) strategy. Experimental results demonstrate the effectiveness of AutoTrans across varied application scenarios. On the ZCU102 board, AutoTrans achieves up to 621 GOPS when accelerating BERT-base with 4 K-token inputs, yielding a 1.27 × to 1.91 × improvement in energy efficiency over previous designs. For ViT-Base with 256-token inputs on the Alveo U50, AutoTrans attains up to 1548 GOPS, achieving up to 1.96 × higher compute density, highlighting its scalability and adaptability across NLP and vision domains.

This paper presents a novel DC-DC loop control framework, ResDDQN-FPGA, which integrates reinforcement learning with field-programmable gate arrays (FPGAs) to enhance the dynamic response and stability of LLC resonant DC-DC converters. Traditional control strategies for LLC converters face difficulties in adapting to variations in operating conditions, especially with aging circuit components. To address these issues, we introduce the Double DQN algorithm, modified with a residual structure to form the ResDDQN control algorithm, which provides accurate estimation of control objectives with reduced computational demand. The ResDDQN algorithm is deployed on an FPGA platform, leveraging its parallel processing capabilities to improve execution speed and efficiency, while also reducing resource usage and energy consumption. Experimental validation on a 1kW full-bridge LLC circuit demonstrates the framework’s effectiveness, achieving superior dynamic performance, lower energy consumption, and greater stability. This highly efficient and responsive control solution is well-suited for real-time applications, maintaining scalability without compromising computational demand.

Compared to traditional Vision Transformers (ViT), Mixture-of-Experts Vision Transformers (MoE-ViT) are introduced to scale model size without a proportional increase in computational complexity, making them a new research focus. Given the high performance and reconfigurability, FPGA-based accelerators for MoE-ViT emerge, delivering substantial gains over general-purpose processors. However, existing accelerators often fall short of fully exploring the design space, leading to suboptimal trade-offs between resource utilization and performance. To overcome this problem, we introduce UbiMoE, a novel end-to-end FPGA accelerator tailored for MoE-ViT. Leveraging the unique computational and memory access patterns of MoE-ViTs, we develop a latency-optimized streaming attention kernel and a resource-efficient reusable linear kernel, effectively balancing performance and resource consumption. To further enhance design efficiency, we propose a two-stage heuristic search algorithm that optimally tunes hardware parameters for various FPGA resource constraints. Compared to state-of-the-art (SOTA) FPGA designs, UbiMoE achieves 1.34× and 3.35× throughput improvements for MoE-ViT on Xilinx ZCU102 and Alveo U280 platforms, respectively, while enhancing energy efficiency by 1.75× and 1.54×. Our implementation is available at https://github.com/DJ000011/UbiMoE.

Real-Time Embedded Systems (RTES) are widely used in safety-critical devices, where deterministic data flow is essential to system verification and reliable execution. It requires that each consumer task instance reads data from the deterministic producer task instance. In software based on general-purpose computing instruction sets, communication-related instruction execution order couples data flow among tasks, necessitating a deterministic execution order of these instructions to preserve data-flow determinism. However, enforcing this order complicates software, and suffers from priority inversion and variable execution overheads, which significantly increases task worst-case response times (WCRT) and response time variability. This paper identifies the cause of above issues as the semantics of general-purpose instruction sets, under which dataflow determinism relies on the deterministic execution order of communication-related instructions. To address this, we make the following contributions. First, we propose Time-Semantic Instruction set (TSI), which supports memory access using both addresses and timestamps. TSI enables data-flow determinism without strict instruction ordering. Second, we design a TSI-enabled implementation compatible with conventional memory systems. Third, we provide two TSI-based deterministic data-flow programming paradigms, along with correctness proofs. Finally, we evaluate TSI hardware cost and implement a cycle-accurate simulator based on a TSI-extended RISC-V. Experiments demonstrate that, under reasonable memory overhead, our approach reduces programming complexity and achieves up to$21.6 \times$reduction in WCRT and up to$89.6 \times$reduction in response time variability compared to existing methods.

Heterogeneous systems commonly adopt dynamic scheduling algorithms to improve resource utilization and enhance scheduling flexibility. However, it may introduce timing anomalies, wherein locally reduced tasks' actual execution times can lead to an increase in the overall system execution time. This phenomenon significantly complicates the analysis of WorstCase Response Time (WCRT), rendering conventional analysis either overly pessimistic or unsafe, and often necessitating exhaustive state-space exploration to ensure correctness. To address this challenge, this paper presents the first timing-anomalyfree dynamic scheduling algorithm for heterogeneous systems, referred to as Deterministic Dynamic Execution. The core idea is to apply deterministic execution constraints, which partially restrict the resource allocation and execution order of tasks at runtime. It achieves a safe and tight WCRT through a single offline simulation execution. In this paper, we provide preliminary experimental validation of the timing-anomaly-free property of our algorithm and outline the basic idea of a formal proof.

Vision transformer (ViT) models have demonstrated remarkable advantages in visual tasks. However, the ViT model contains various types of operators, and its sophisticated model structure imposes substantial computational complexity and storage burden. Existing hardware solutions still fail to fully unleash the ViT acceleration potential due to the mismatch between operators and hardware architectures, suffering from inefficient dataflow mapping. This work proposes HDViT, a full-fledged heterogeneous hardware accelerator on FPGA, to enhance the ViT acceleration by comprehensively analyzing and addressing the challenges of heterogeneous architecture design. Specifically, HDViT first develops a heterogeneous architecture design that is composed of multiple processing engines (PEs) to accelerate various operators in the ViT model. Then, HDViT devises a hybrid-oriented dataflow mapping strategy to reduce data transmission granularity and alleviate storage resource pressure. Lastly, to achieve the latency balancing among multiple PEs, we formulate the HDViT architecture and implement an automated exploration process to identify optimized parallelism parameters that satisfy computation and storage demands while enhancing the heterogeneous architectural performance. Experimental results indicate that HDViT achieves significant performance speedups of 2.16$\times$and 3.51$\times$compared to previous heterogeneous and unified accelerators, respectively. HDViT also achieves a maximum of 98.46% hardware utilization.

Spiking neural networks (SNNs) replace the multiply-and-accumulate operations in traditional artificial neural networks (ANNs) with lightweight mask-and-accumulate operations, achieving greater performance. Existing SNN architectures are primarily designed based on fully-connected or convolutional SNN topologies and still struggle with low task accuracy, limiting their practical applications. Recently, transformer SNN (TSNN) models have shown promise in matching the accuracy of nonspiking ANNs and demonstrated potential application prospects. However, their diverse computation pattern and sophisticated network structure with high computation and memory footprints impede their efficient deployment. Thus, in this work, we move our attention to heterogeneous architecture design and propose SpikeTA, the first neuromorphic hardware accelerator explicitly designed for the TSNN model on FPGA. First, SpikeTA enables parameterizable hardware engines (HEs) designed for the network layers in TSNN, enhancing compatibility between HEs and network layers. Second, SpikeTA optimizes arithmetic operations between binary spikes and synaptic weights by presenting a DSP-efficient addition tree. By analyzing the inherent data characteristics, SpikeTA further introduces a depth-aware buffer management strategy to provide sufficient access ports. Third, SpikeTA employs a streaming dataflow mapping to optimize data transmission granularity and leverages a split-engine dataflow mapping to facilitate pipelined latency balancing. Experimental results demonstrate that SpikeTA achieves significant performance speedups of$140.73\times $–$1023.53\times $and$2.97\times $–$7.29\times $over architectures running on the AMD EPYC 7542 CPU and NVIDIA A100 GPU, respectively. SpikeTA also outperforms state-of-the-art SNN and Transformer accelerators by$2.79\times $and$2.66\times $in architecture performance while achieving a peak performance of 28.99 TOPs.

Self-knowledge distillation has emerged as a powerful method, notably boosting the prediction accuracy of deep neural networks while being resource-efficient, setting it apart from traditional teacher-student knowledge distillation approaches. However, in safety-critical applications, high accuracy alone is not adequate; conveying uncertainty effectively holds equal importance. Regrettably, existing self-knowledge distillation methods have not met the need to improve both prediction accuracy and uncertainty quantification simultaneously. In response to this gap, we present an uncertainty-aware self-knowledge distillation method named UASKD. UASKD introduces an uncertainty-aware contrastive loss and a prediction synthesis technique within the self-knowledge distillation process, aiming to fully harness the potential of self-knowledge distillation for improving both prediction accuracy and uncertainty quantification. Extensive assessments illustrate that UASKD consistently surpasses other self-knowledge distillation techniques and numerous uncertainty calibration methods in both prediction accuracy and uncertainty quantification metrics across various classification and object detection tasks, highlighting its efficacy and adaptability.

Emerging FPGA devices are widely used in graph processing and achieve high performance by exploiting fine-grained parallelism and high-bandwidth memory (HBM) sub-systems with dozens of channels. However, the graph size increases rapidly and often exceeds the capacity of the FPGA device or one of the memory channels, incurring out-of-memory (OoM) faults and performance degradation. Existing approaches focus on improving the performance of on-device full graph processing, ignoring the support for hyper-scale graphs and cross-iteration acceleration. In this work, we propose SoGraph, a graph processing architecture for large graph acceleration on HBM-equipped FPGAs, cooperating with the host. To minimize PCIe traffic during the processing, we adopt three hardware-oriented optimizations: 1) activeness-driven subgraph scheduler, which minimizes on-device graph resident set; 2) lightweight state-aware processing extension in edge-centric paradigm; 3) bit-level update analyzer for to-be-transfer subgraph reduction. The results show that the prototype on Alveo U280 FPGA achieves an average of 3.18x performance speedup over the modified state-of-the-art FPGA design and 1.3x energy efficiency over the GPU solution.

Large Language Models (LLMs) have been widely deployed in data centers to provide various services, among which the most representative is the Generative Pre-trained Transformer (GPT). The GPT model has heavy memory and computing overhead, and its inference process has two stages with distinct computing characteristics: Prefill and Decode. Utilizing existing GPUs and FPGA accelerators to construct a platform for deploying GPT in data centers faces the challenges of needing more effective synchronization schemes or structures with higher computational intensity. This paper proposes LORA, a low latency end-to-end GPT acceleration platform utilizing multiple FPGAs. Firstly, we optimize the synchronization timing of the GPT model to reduce the computation and communication overhead. Secondly, we devise some efficient synchronization steps for specific layers of the GPT model that overlap part of the computation and communication delay to improve the latency of our platform. Finally, we deploy recurrent structures on each FPGA to accelerate the different stages of the GPT model. Implemented on the Xilinx Alveo U280 FPGAs, LORA achieves an average $11.1 \times$ speedup over NVIDIA V100 GPUs on the modern GPT-2 model. Compared to the existing multi-FPGA accelerator appliance, LORA shows performance improvements of up to $4 \times$ and $2.7 \times$ in the Prefill and Decode stages.

The HLS toolchain effectively reduces the design complexity of FPGA hardware accelerators. However, in scenarios involving the multi-objective optimization of large-scale HLS designs, determining the knob configurations of Pareto design points remains a challenging task for designers. Our work re-evaluates the key factors affecting the efficiency of multiobjective design space exploration in HLS design and proposes an efficient framework named FlexWalker. It utilizes the upper confidence bound algorithm to organize various heterogeneous regression models for predicting the quality of HLS designs with different knob configurations in the design space and introduces a probability sampling algorithm and an elastic Pareto frontier to counteract the negative impact of regression model errors. Experimental results show that our work can stably eliminate over 90% of non-Pareto frontier design points in the tested HLS design space, effectively enhancing the efficiency of multiobjective design space exploration.

Attention-based transformers have achieved significant performance breakthroughs in natural language processing (NLP) and computer vision (CV) tasks. Meanwhile, the ever-increasing length of today’s input sequences puts much pressure on computing devices. FPGAs are widely used to accelerate Transformer inference due to their high energy efficiency and flexibility. However, most of the existing FPGA-based Transformer accelerators are oriented to small input lengths, making it hard to accelerate long input sequences. To this end, we design an efficient Transformer accelerator for FPGA and long-sequence input scenarios. We use the tiling softmax algorithm to fuse attention computation, eliminating the memory and bandwidth bottleneck in the attention layer and allowing our accelerator to support arbitrary input sequence lengths. We use BERT-Base on the Alveo U50 board for evaluation, and our implementation achieves computational efficiency improvements of 1.09 ∼ 2.48 × over prior FPGA accelerators. Besides, our accelerator can support up to 175K input sequence length when running BERT-like structures, far more than previous designs.

Sparse tensor computation plays a crucial role in modern deep learning workloads, and its expensive computational cost leads to a strong demand for high-performance oper-ators. However, developing high-performance sparse operators is exceptionally challenging and tedious. Existing vendor operator libraries fail to keep pace with the evolving trends in new algorithms. Sparse tensor compilers simplify the development and optimization of operator, but existing work either requires significant engineering effort for tuning or suffers from limitations in search space and search strategies, which creates unavoidable cost and efficiency issues. In this paper, we propose AutoSparse, a source-to-source auto-tuning framework that targets sparse for-mat and schedule for sparse tensor program. Firstly, AutoSparse designs a sparse tensor DSL based on dynamic computational graph at the front-end, and proposes a sparse tensor program computational pattern extraction and automatic design space generation scheme based on it. Second, AutoSparse's back-end designs an adaptive exploration strategy based on reinforcement learning and heuristic algorithm to find the optimal format and schedule configuration in a large-scale design space. Compared to prior work, developers using AutoSparse do not need to specify tuning design space relied on any compilation or hardware knowledge. We use the SuiteS parse dataset to compare with four state-of-the-art baselines, namely, the high-performance operator library MKL, the manually-based optimisation scheme ASpT, the auto-tuning-based framework TVM-S and WACO. The results demonstrate that AutoSparse achieves average speedups of 1.92-$2.48 \times. 1.19-6.34 \times$. and$1.47-2.23\times$for the SpMV, SpMM, and SDDMM operators, respectively. We will open-source AutoSparse at https://github.com/Qu-Xiangjun/AutoSparse.

Tensorized programs use hardware intrinsics on accelerators to significantly improve tensor computation performance. The trend of hardware customization has led to the emergence of massive hardware accelerators and intrinsics, which poses a significant engineering challenge for manually coding efficient tensorized programs. Tensorized program tuning in deep learning compilers (DLCs) is considered an effective approach to address the challenge. At the core of program tuning relies the design of the cost model, but currently there is still a lack of cost models specifically designed for tensorized programs, which hampers the co-optimization of DLCs and hardware accelerators. In this paper, we propose UniCoMo, a unified cost model for tensorized program tuning on various hardware platforms and hardware intrinsics. We first analyze the design challenges introduced by tensorized programs for cost models in terms of feature representation and transfer prediction. And then, for feature representation, we propose a unified feature representation for tensorized programs by using program behavior as a template, mining program features with the schedule attention matrix, and incorporating hardware intrinsic abstraction. For transfer prediction, we propose a unified transfer prediction strategy for tensorized program cost model based on lifelong learning and transfer learning. To meet training and testing requirements, we constructed a dataset dedicated to tensorized program tuning. Results show that UniCoMo maintains the state-of-the-art accuracy while significantly improving adaptability to diverse execution environments and enabling flexible transfer prediction. It can speed up search time by 9.8× and improve inference speed by 1.9× open-sourced at https://github.com/ZhW-loop/UniCoMo.

Graph random walk sampling is becoming increasingly important with the widespread popularity of graph applications. It aims to capture the desirable graph properties by launching multiple walkers to collect feature paths. However, previous research suffers long sampling latency and severe memory access bottlenecks due to intrinsic data dependency and skewed vertex distribution. Thus, in this paper, we propose FastRW, a dedicated accelerator to boost graph random walk operation on FPGAs. Specifically, FastRW first integrates multiple parallel processing engines to achieve data-level parallelism, where each processing engine also leverages dataflow scheduling to resolve data dependency and hide long sampling latency. Secondly, FastRW leverages a combination of multiple storage resources to implement a hybrid memory architecture adapted to skewed vertex distribution. By integrating the above optimizations, FastRW develops a performance model to take advantage of the balance between computation parallelism and bandwidth demand. We evaluate FastRW with two classic sampling algorithms on a wide range of real-world graph datasets. The experimental results show that FastRW achieves a speedup of 37.52$\boldsymbol{\times}$on average over the system running on two 8-core Intel CPUs. FastRW also achieves an average of 28.04$\boldsymbol{\times}$speedup over the architecture implemented on V100 GPU.

Recently, algorithm-hardware co-exploration for neural networks (NNs) has become the key to obtaining high-quality solutions. However, previous efforts for FPGAs focus on neural architecture search (NAS) while lacking hardware architecture search (HAS), thus limiting the full potential of co-design. Although expanding the scope of HAS offers performance potential, the exponentially increased joint search space presents a formidable challenge. To address this, we propose a deep and efficient framework, which jointly searches for Networks and Accelerators for FPGAs in a balanced co-search space. First, we adjust the NAS space and then introduce a block-level bitwidth search on the software side. Meanwhile, we design a hardware-friendly quantization algorithm to facilitate hardware efficiency and accuracy. Second, we design a dataflow-configurable hardware unit with computation and memory access optimizations for quantized multiplication. Based on this, we incorporate critical heterogeneous multicore architecture exploration on the hardware side. Third, to enable rapid hardware feedback in the enlarged HAS space, we perform resource and performance modeling and design a fast hardware generation algorithm based on the genetic algorithm. Specifically, we apply optimization techniques, like mapping space pruning, greedy bandwidth allocation, and coarse-grained search, to speed up this process. We validate in edge and cloud scenarios. Experimental results show that efficiently explores a significantly larger joint space and provides high-quality solutions. Compared with previous state-of-the-art co-design works, the searched CNN-accelerator pairs improve the throughput by 2.07× ~ 7.10× and energy efficiency by 1.41× ~ 2.27× under similar accuracy on the ImageNet dataset.

Block-circulant matrix (BCM) compression has garnered much attention in the hardware acceleration of convolutional neural networks (CNNs) due to its regularity and efficiency. However, constrained by the difficulty of exploring the compression parameter space, existing BCM-based methods often apply a uniform compression parameter to all CNN models’ layers, losing the compression’s flexibility. Additionally, independently optimizing models or accelerators makes achieving the optimal tradeoff between model accuracy and hardware efficiency challenging. To this end, we propose FlexBCM, a joint exploration framework that efficiently explores both the parameter compression and hardware parameter space to generate customized hybrid BCM-compressed CNN and field-programmable gate array (FPGA) accelerator solutions. On the algorithmic side, leveraging the idea of neural architecture search (NAS), we design an efficient differentiable sampling method to rapidly evaluate the accuracy of candidate subnets. Additionally, we devise a hardware-friendly frequency domain quantization scheme for BCM computation. On the hardware side, we develop the efficient and parameter-configurable convolutional core (ConvPU) alongside the BCM computing core (BCMPU). The BCMPU can flexibly accommodate different compression parameters at runtime, incorporate complex-number DSP packing and conjugate symmetry optimizations. For model-to-hardware evaluation, we construct accurate latency and resource consumption models. Moreover, we design a fast hardware generation algorithm based on the coarse-grained search to provide prompt feedback on the hardware evaluation of the current subnet. Finally, we validate FlexBCM on the Xilinx ZCU102 FPGA and compare its compressed CNN-accelerator solutions with previous state-of-the-art works. Experimental results demonstrate that FlexBCM achieves 1.21–3.02 times higher-computational efficiency for ResNet18 and ResNet34 models while maintaining an acceptable accuracy loss on the ImageNet dataset.

Graph random walk (GRW) sampling is becoming increasingly important with the widespread popularity of graph applications. It involves some walkers that wander through the graph to capture the desirable properties and reduce the size of the original graph. However, previous research suffers long sampling latency and severe memory access bottlenecks due to intrinsic data dependency and irregular vertex distribution. This paper proposes FastRW, a dedicated accelerator to release GRW acceleration on FPGAs. FastRW first schedules walkers' execution to address data dependency and mask long sampling latency. Then, FastRW leverages pipeline specialization and bit-level optimization to customize a processing engine with five modules and achieve a pipelining dataflow. Finally, to alleviate the differential accesses caused by irregular vertex distribution, FastRW implements a hybrid memory architecture to provide parallel access ports according to the vertex's degree. We evaluate FastRW with two classic GRW algorithms on a wide range of real-world graph datasets. The experimental results show that FastRW achieves a speedup of 14.13× on average over the system running on two 8-core Intel CPUs. FastRW also achieves 3.28×∼198.24× energy efficiency over the architecture implemented on V100 GPU.

Recently, algorithm and hardware co-design for neu-ral networks (NNs) has become the key to obtaining high-quality solutions. However, prior works lack consideration of the underlying hardware and thus suffer from a severely unbalanced neural architecture and hardware architecture search (NA-HAS) space on FPGAs, failing to unleash the performance potential. Nevertheless, a deeper joint search leads to a larger (multiplicative) search space, highly challenging the search. To this end, we propose an efficient differentiable search framework NAF, which jointly searches the networks (e.g., operations and bitwidths) and accelerators (e.g., heterogeneous multicores and mappings) under a balanced NA-HAS space. Concretely, we design a coarse-grained hardware-friendly quantization algorithm and integrate it at a block granularity into the co-search process. Meanwhile, we design a highly optimized block processing unit (BPU) with key dataflow configurable. Afterward, a dynamic hardware generation algorithm based on modeling and heuristic rules is designed to perform the critical HAS and fast generate hardware feedback. Experimental results show that compared with the previous state-of-the-art (SOTA) co-design works, NAF improves the throughput by$1.99\times\sim 6.84\times$on Xilinx ZCU102 and energy efficiency by 17%~88% under similar accuracy on the ImageNet dataset.

This paper proposes a comprehensive solution for managing resources in cloud FPGAs from application programming to accelerator deployment, which are compatible with existing FPGA virtualization frameworks. To achieve this, we leverage Continuous Passing Style-based application programming templates and dynamic task graph-based software IR and compute resource pool-based hardware IR and Access/Execute Decoupled-based computation execution patterns. A preliminary experiment shows that our method is able to make full use of idle resources in FPGAs for computation, resulting in an nearly 40% reduction in average turnaround time.

In the process of using FPGA to accelerate applications, we can obtain more efficient design by selecting appropriate data bit width and precision. However, the reasonable selection of data type needs to be determined by the designer, which often requires rich design experience and a lot of time. In this paper, we propose DataMaster, which is a GNN-based optimization framework for data types used in FPGA design. The purpose of DataMaster is to find the most suitable data bit width and precision for computing tasks at the operator level and finally obtain a hardware design with better performance and less resource consumption.

Regular expression (REGEX) matching tasks drive much research on automata processors (AP). Among them, the von Neumann AP can efficiently utilize on-chip memory to process the Deterministic Finite Automata (DFA), but it is limited to small REGEX sets due to the DFA's state explosion problem. For large REGEX sets, the spatial AP based on Nondeterministic Finite Automaton (NFA) is the mainstream choice. However, there are two problems with previous FPGA-based spatial AP. First, it cannot obtain a balanced FPGA resource usage (LUT and BRAM), which easily leads to resource shortage. Second, to compress the report output data of large REGEX sets, it uses dynamic report compression, which not only consumes a lot of FPGA resources but also limits performance.

Growing pattern matching applications are employing finite automata as their basic processing model. These applications match tens to thousands of patterns on a large amount of data, which brings a great challenge to conventional processors. Therefore hardware-based solutions have emerged frequently and achieved high throuphput automata processing. However, existing methods are generally difficult to achieve both processing speed and storage efficiency, and are often too heavy to be integrated into a small chip and have to rely on off-chip DRAMs or other high capacity memories even on some simple data sets, leading to the potential area and power consumption issues. In this paper, we focus on building a more lightweight automata processing engine, hoping to store the whole automata model into on-chip memory and run effectively and independently. We propose LAP, a lightweight automata processing engine. Powered with a novel automata model (A-DFA) and efficient packing algorithms, extremely high storage efficiency compared with traditional DFA is achieved in LAP. Meanwhile, we identify the key parallelization factors in the A-DFA model and then propose a specialized microarchitecture with novel instructions to further accelerate the state transition process. As a result, LAP can obtain more effective trade-off between processing speed and storage efficiency. Evaluation results show that LAP achieves extremely high storage efficiency on simple data sets, exceeding IBM's RegX by 8×, and achieves significant improvements in processing speed ranging from 1.32× to 1.91× compared with previous lightweight hardware implementations. Moreover, LAP has good scalability in hardware architecture. It is easy to build an acceleration system with higher throughput by increasing the number of cores. We prototype a 16-core system into Xilinx ZC702 FPGA and a 64-core system into Xilinx ZCU102 FPGA respectively. The prototype system on ZC702 on average achieves 3.5 GB/s throughput on simple data sets, and the prototype system on ZCU102 can obtain higher throughput and compute density values on part of large datasets in ANMLZoo compared with modern in-memory NFA-based solutions.

In recent years, graph neural networks (GNNs) have achieved impressive performance in various application fields by extracting information from graph-structured data. It contains extensive feature aggregation operations and has become a performance bottleneck, which can be abstracted as a specialized sparse-dense matrix multiplication (SpMM) operation. Previous works have leveraged the inner product or outer product to accelerate the feature aggregation process. However, inefficient execution leads to extremely unbalanced workloads and extensive intermediate data, hampering the performance of previous processors. So in this article, we demonstrate an algorithm/hardware co-optimization chance to enhance SpMM acceleration for GNNs. First, the algorithm part develops a dataflow-efficient SpMM algorithm that integrates three optimization methods to mitigate computation and memory access inefficiencies. Specifically, 1) the proposed equal-value partition method achieves fine-grained data partition and enables load balancing during data movement; 2) after observing the vertex aggregation phenomenon, a vertex-clustering optimization method is presented to enable significant data locality; and 3) the adaptive dataflow based on Gustavson’s algorithm is further implemented to enable the efficient distribution of sparse elements and improves computing resource utilization. Then, the hardware part features the proposed SpMM algorithm and customizes SDMA, a flexible and efficient accelerator to boost SpMM acceleration, which follows the adaptive dataflow to eliminate sparsity and explore the regular parallelism dimension. Finally, we prototype SDMA on the Xilinx Alveo U280 FPGA accelerator card. The results demonstrate that SDMA achieves$5.68\times $–$14.68\times $energy efficiency over the previous GPU implementations deployed on the Nvidia GTX 1080Ti and$1.32\times $higher throughput over the state-of-the-art FPGA prototype.

Random walk (RW) is a common graph analysis algorithm that consists of two phases: construction and sampling. The construction phase is responsible for generating the sampling table. The sampling phase contains many walkers which wander through the whole graph to sample. However, RW is notorious for its dynamic and sparse memory access pattern, which makes existing research suffer low throughput and memory bottleneck. In addition, the variety of RW algorithms in different scenarios also brings new design challenges.This paper proposes HeteroRW, a generalized framework to accelerate RWs on FPGAs. HeteroRW first identifies the two phases’ computation characteristics and presents corresponding hardware acceleration designs, respectively. Then, HeteroRW achieves the template-based design to support a variety of RW algorithms. Finally, HeteroRW integrates a novel scheduling layer to partition the input data and perform design space exploration (DSE). Experimental results show that HeteroRW achieves 4.3x speedup over the recent FPGA implementation while effectively simplifying the accelerator customization process.

Nowadays, the scale of deep neural networks is getting larger and larger. These large-scale deep neural networks are both compute and memory intensive. To overcome these problems, we use block-circulant weight matrices and Fast Fourier Transform (FFT) to compress model and optimize computation. Compared to weight pruning, this method does not suffer from irregular networks. The main contributions of this paper include the implementation of a convolution module and a fully-connected module with High-Level Synthesis (HLS), deployment and performance test on FPGA platform. We use AlexNet as a case study, which demonstrates our design is more efficient than the FPGA2016.

In recent years, graph neural networks (GNNs) as a deep learning model have emerged. Sparse-Dense Matrix Multiplication (SpMM) is the critical component of GNNs. However, SpMM involves many irregular calculations and random memory accesses, resulting in the inefficiency of general-purpose processors and dedicated accelerators. The highly sparse and uneven distribution of the graph further exacerbates the above problems. In this work, we propose SDMA, an efficient architecture to accelerate SpMM for GNNs. SDMA can collaboratively address the challenges of load imbalance and irregular memory accesses. We first present three hardware-oriented optimization methods: 1) The Equal-value partition method effectively divides the sparse matrix to achieve load balancing between tiles. 2) The vertex-clustering optimization method can explore more data locality. 3) An adaptive on-chip dataflow scheduling method is proposed to make full use of computing resources. Then, we combine and integrate the above optimization into SDMA to achieve a high-performance architecture. Finally, we prototype SDMA on the Xilinx Alveo U50 FPGA. The results demonstrate that SDMA achieves 2.19x-3.35x energy efficiency over the GPU implementation and 2.03x DSP efficiency over the FPGA implementation.

With the rapid development of convolutional neural networks (CNNs), FPGAs have become one of the most attractive candidates for deploying CNNs. However, previous FPGA solutions based on the traditional convolution are still limited by computational power. In this article, we introduce the octave convolution (OctConv) into the CNN accelerator design for the first time to improve the hardware acceleration efficiency and design a dedicated OctPU for mapping OctConv to FPGAs, which employs a parallel dataflow pattern to exploit the parallelism of OctConv. Then, we present a novel and scalable architecture that dynamically combines the inter-layer pipelined structure and multi-layer reuse structure. Meanwhile, to obtain the optimized solution, we build a multidimensional performance and resource analysis model and a two-stage search algorithm based on greedy and heuristic algorithms. We evaluate our proposal by implementing VGG16 and ResNet50 on the Xilinx VU9P FPGA. Experimental results show that our prototypes can achieve an average of 3321 GOP/s for the convolutional layers for VGG16 and 2873 GOP/s for the overall ResNet50 using OctConv. Compared to previous works based on the traditional convolution, our prototypes own a 1.72 to 2.33 speedup in throughput and a 2.01 to 5.18 improvement in computational density. Our design also presents an excellent compromise performance and generalization

Since Google proposed Transformer in 2017, it has made significant natural language processing (NLP) development. However, the increasing cost is a large amount of calculation and parameters. Previous researchers designed and proposed some accelerator structures for transformer models in field-programmable gate array (FPGA) to deal with NLP tasks efficiently. Now, the development of Transformer has also affected computer vision (CV) and has rapidly surpassed convolution neural networks (CNNs) in various image tasks. And there are apparent differences between the image data used in CV and the sequence data in NLP. The details in the models contained with transformer units in these two fields are also different. The difference in terms of data brings about the problem of the locality. The difference in the model structure brings about the problem of path dependence, which is not noticed in the existing related accelerator design. Therefore, in this work, we propose the ViA, a novel vision transformer (ViT) accelerator architecture based on FPGA, to execute the transformer application efficiently and avoid the cost of these challenges. By analyzing the data structure in the ViT, we design an appropriate partition strategy to reduce the impact of data locality in the image and improve the efficiency of computation and memory access. Meanwhile, by observing the computing flow of the ViT, we use the half-layer mapping and throughput analysis to reduce the impact of path dependence caused by the shortcut mechanism and fully utilize hardware resources to execute the Transformer efficiently. Based on optimization strategies, we design two reuse processing engines with the internal stream, different from the previous overlap or stream design patterns. In the stage of the experiment, we implement the ViA architecture in Xilinx Alveo U50 FPGA and finally achieved ~5.2 times improvement of energy efficiency compared with NVIDIA Tesla V100, and 4–10 times improvement of performance compared with related accelerators based on FPGA, that obtained nearly 309.6 GOP/s computing performance in the peek.

Growing applications are employing finite automata as their basic computational model. These applications match tens to thousands of patterns on a large amount of data, which brings great challenges to conventional processors. Hardware-based solutions have achieved high throughputs automata processing. However, they are too heavy to be integrated into small chips. Besides, they have to rely on DRAMs or other high capacity memories to store their underlying automata models. We focus on building a more lightweight automata processor, which can store the whole automata model into SRAMs with limited size and run independently. We propose LAP, a lightweight automata processor. Extremely high storage efficiency is achieved in LAP, leveraging a novel automata model (ADFA) and efficient packing algorithms. Besides, we exploit software-hardware co-design to achieve faster processing speed. We observe that ADFA's traversal algorithm is parallelizable. Thus, we propose novel hardware instructions to parallel the additional memory accesses in ADFA model and hide their access overhead. LAP is organized into a four-stage pipeline and prototyped into Xilinx Artix-7 FPGA at 263 MHz frequency. Evaluations show that LAP achieves extremely high storage efficiency, exceeding IBM's RegX and Micron's AP by 8×. Besides, LAP achieves significant improvements in processing speed ranging from 32% to 91% compared with previous lightweight implementations. As a result, a low-power CPU equipped with five LAP cores can achieve 9.5 Gbps processing throughput matching 400 patterns simultaneously.

The lossless image compression technique has a great application value in distortion-sensitive applications. JPEG-LS, as a mature lossless compression standard, is widely adopted for its excellent compression ratio. Many hardware JPEG-LS compressors are proposed on FPGAs and ASICs to achieve high energy efficiency and low cost. However, JPEG-LS has a contextual Read-After-Write (RAW) issue, making previous hardware either insufficiently explore its parallelism potential or induce other defects while parallelizing, such as compression ratio dropping and compatibility problems. In this paper, we propose a hardware/software co-design method for high-performance JPEG-LS compressor design. At the software level, we propose a pixel grouping scheduling scheme and the Pseudo-LS method to tap the parallelism aiming at the RAW issue. At the hardware level, we discuss the high-performance design methods of these software-level schemes and propose a design space exploration method to constrain the resource usage introduced by parallelization. To our knowledge, our architecture, UH-JLS, is the first pixel-level parallelization streaming image compressor based on the standard JPEG-LS. The experiments show that in the lossless manner and the Pseudo-LS manner, UH-JLS respectively achieves 5.6x and 7.1x speedup than the previous state-of-the-art FPGA-based JPEG-LS compressor.

Clustering algorithms are becoming popular and widely applied in many academic fields, such as machine learning, pattern recognition, and artificial intelligence. It has posed significant challenges to accelerate the algorithms due to the explosive data scale and wide variety of applications. However, previous studies mainly focus on the raw speedup with insufficient attention to the flexibility of the accelerator to support various applications. In order to accelerate different clustering algorithms in one accelerator, in this article, we design an accelerating framework based on FPGA for four state-of-the-art clustering methods, including K-means, PAM, SLINK, and DBSCAN algorithms. Moreover, we provide both euclidean and Manhattan distances as similarity metrics in the accelerator design paradigm. Moreover, we provide a custom instruction set to operate the accelerators within each application. In order to evaluate the performance and hardware cost of the accelerator, we constructed a hardware prototype on the state-of-the-art Xilinx FPGA platform. Experimental results demonstrate that the accelerator framework is able to achieve up to 23× speedup than Intel Xeon processor, and is 9.46× more energy efficient than NVIDIA GTX 750 GPU accelerators.

Currently, deep learning technology is widely used in various fields, such as face recognition, object recognition, and image classification. Multiple application instances sharing GPU resources can deploy more applications with limited GPU resources. However, this will lead to resource competition problems, resulting in application switching, timeouts, and other phenomena. Therefore, how to deploy these applications to a single machine with limited resources and properly schedule tasks while maximizing system performance is a new challenge. In this article, we propose Tinker, a middleware, to solve multiple CNN-based application deployment problems on a single machine. Tinker has two phases: 1) offline analysis and 2) runtime scheduling. Offline analysis generates the best application deployment configuration information of the current system. Runtime scheduling can properly schedule tasks to ensure that they are completed normally and efficiently utilize resources to improve system performance. Our experiment proved that Tinker boosts system performance and ensures that most tasks are completed efficiently.

Genome sequencing is one of the most challenging problems in computational biology and bioinformatics. As a traditional algorithm, the string match meets a challenge with the development of the massive volume of data because of gene sequencing. Surveys show that there will be a huge amount of short read segments during the process of gene sequencing and the need for a highly efficient is urgent. As a classic fast and exact single pattern matching algorithm, Knuth-Morris-Pratt (KMP) algorithm has been demonstrated in network security and computational biology. However, with the increasing amount of data in the modern society, it becomes increasingly important and essential to provide a High-performance implementation of KMP algorithm. In this article, we implement a scalable KMP accelerator based on FPGA, named GeneKMP. The accelerator is composed of different computing units to achieve a pipelined organization for higher throughput with satisfying scalability. A novel programming model is provided to alleviate the burden of the high-level programmers. We provide a greedy-based partitioning algorithm for the software/hardware design paradigms. Experimental results on the state-of-the-art Xilinx FPGA hardware prototype show that our accelerator can achieve up to a promising speedup with insignificant hardware cost and power consumption.

With the continuous evolution of Convolutional Neural Networks (CNNs) and the improvement of the computing capability of FPGAs, the deployment of CNN accelerator based on FPGA has become more and more popular in various computing scenarios. The key element of implementing these accelerators is to take full advantage of underlying hardware characteristics to adapt to the computational features of the software-level CNN model. To achieve this goal, however, previous designs mainly focus on the static hardware reconfiguration pattern, which is not flexible enough and can hardly make the accelerator architecture and the CNN features fully fit, resulting in inefficient computations and data communications. By leveraging the dynamic partial reconfiguration technology equipped in the modern FPGA devices, in this article, we propose a new accelerator architecture for implementing CNNs on FPGAs in which static and dynamic reconfigurabilities of the hardware are cooperatively utilized to maximize the acceleration efficiency. Based on this architecture, we further present a systematic design and optimization methodology for implementing the specific CNN model in the particular computing scenario, in which a static design space exploration method and a reinforcement learning-based decision method are proposed to obtain the optimal static hardware configuration and run-time reconfiguration strategy respectively. We evaluate our proposal by implementing three widely used CNN models, AlexNet, VGG16C, and ResNet34, on the Xilinx ZCU102 FPGA platform. Experimental results show that our implementations on average can achieve 683 GOPS under 16-bit fixed data type and 1.37 TOPS under 8-bit fixed data type for three targeted CNN models, and improve the computational density from 1.1× to 1.91× compared with previous implementations on the same type of FPGA platform.

Deep learning has been an emerging field of machine learning during past decades. However, the diversity and large scale data size have posed significant challenge to construct a flexible and high performance implementations of deep learning neural networks. In order to improve the performance as well to maintain the scalability, in this paper we present SOLAR, a services-oriented deep learning architecture using various accelerators like GPU and FPGA. SOLAR provides a uniform programming model to users so that the hardware implementation and the scheduling is invisible to the programmers. At runtime, the services can be executed either on the software processors or the hardware accelerators. To leverage the trade-offs between the metrics among performance, power, energy, and efficiency, we present a multitarget design space exploration. Experimental results on the real state-of-the-art FPGA board demonstrate that the SOLAR is able to provide a ubiquitous framework for diverse applications without increasing the burden of the programmers. Moreover, the speedup of the GPU and FPGA hardware accelerator in SOLAR can achieve significant speedup comparing to the conventional Intel i5 processors with great scalability.

Recently, embedded FPGAs have been explored as a potential platform for deploying machine learning on edge-devices due to their high energy efficiency and low cost. However, the lack of resources also makes the deployment of CNN on FPGAs more challenging. In this paper, we present OctCNN, which utilizes the octave convolution (OctConv) algorithm to optimize the FPGA-based CNN accelerator. We first propose a novel architecture for deploying OctConv on FPGAs and then present a resource and performance analysis model to guide a fast design space exploration. As a case study, we implement a classic CNN model, VGG16, on Xilinx ZC702. Results show, compared to the mobile-class CPU and GPU, OctCNN achieves$\mathbf{16.88}\times$and$\mathbf{2.43}\times$energy efficiency, respectively. Besides, it has a promising energy efficiency compared to previous FPGA accelerators.

Among all the neural network specialized hardware accelerators like the Application-Specific-Integrate-Circuit(ASIC), an FPGA accelerator stands out for its flexibility, short time-to-market, and energy efficiency. However, when it comes to multitasking and high-speed requirements or realtime and power-efficient scenarios (e.g., UAVs, self-driving cars, and IoT devices), a single-board FPGA accelerator has difficulties in achieving excellent performance. Therefore, Cloud FPGAs(Multi-FPGAs) will be a significant role in high-performance and energy-efficient computation of CNNs for both mobile and cloud computing domains. In this work, we propose an adaptive neural network accelerator on Cloud FPGAs, using multi-FPGA design to satisfy multitasking and high-speed requirements or realtime and power-efficient scenarios. We adopt the roofline model to figure out the optimal configuration of each CNN layer. And a layer clustering algorithm and a layer sequence detection method are proposed to transform CNN models into layer sequences for mapping the CNN model layers efficiently to different FPGA boards. Then, we built an adaptive CNN mapping method of Multi-FPGA chips for CNN models. Preliminary results on the Multi-FPGAs platform demonstrate that our accelerator can improve the performance significantly due to the adaptive mapping method.

Recommendation algorithms, such as Neighborhood-based Collaborative- Filtering (CF), have been widely applied in various emerging machine learning applications. However, under the circumstance of the explosive big data, it poses significant challenges to CF recommendation algorithms as it is becoming quite time and energy-consuming. It has to be optimized and accelerated by powerful engines to process on large data scale. To solve these problems, in this article, we propose WooKong, a ubiquitous accelerator architecture for the collaborative-filtering recommendation on FPGA. It is able to accommodate three types of CF recommendation algorithms, including User-based CF, Item-based CF, and SlopeOne recommendations algorithms, with five different similarity analysis metrics including Jaccard, Cosine, CosineIR, euclidean, and Pearson. To maintain flexibility for these different CF algorithms and metrics, we adopt custom instruction sets to manipulate the learning and prediction accelerators. We implement a hardware prototype on a real Xilinx Zynq FPGA development board. Experimental results show that the proposed learning and prediction accelerators can achieve 8.0X speedup and 1.7X speedup compared with an Intel i7 processor respectively. The accelerator has the energy benefits of up to 137.4X compared with an NVIDIA Tesla K40C GPU, with the affordable hardware cost.

In recent years, a variety of accelerators on FPGAs have been proposed to speed up the convolutional neural network (CNN) in many domain-specific application fields. Besides, some optimization algorithms, such as fast algorithms and network sparsity, have greatly reduced the theoretical computational workload of CNN inference. There are currently a few accelerators on FPGAs that support both the fast Winograd algorithm (WinoA) and network sparsity to minimize the amount of computation. However, on the one hand, these architectures feed data into processing elements (PEs) in units of blocks, some boundary losses caused by sparse irregularities cannot be avoided. On the other hand, these works have not discussed the design space exploration under the sparse condition. In this article, we propose a novel accelerator called WINONN. We fully discuss the challenges faced by supporting WinoA, weight sparsity, and activation sparsity simultaneously. To minimize the online encoding overhead caused by activation sparsity, an efficient encoding format called multibit mask (MBM) is proposed. To handle the irregularities of sparse data, we proposed a novel Scatter-Compute-Gather method in hardware design, combined with a freely sliding buffer to achieve fine-grained data loading to minimize the boundary waste. Finally, we combine a theoretical analysis and experimental method to explore the design space, allowing WINONN to get the best performance on a specific FPGA. Our high scalability design enables us to deploy sparse Winograd accelerators on very small embedded FPGAs, which is not supported in previous works. The experimental results on VGG16 show that we achieve the highest digital signal processing unit (DSP) efficiency and highest energy efficiency compared with the state-of-the-art sparse architectures.

Machine learning has been widely applied in various emerging data-intensive applications, and has to be optimized and accelerated by powerful engines to process very large scale data. Recently, the instruction set based accelerators on Field Progarmmable Gate Arrays (FPGAs) have been a promising topic for machine learning applications. The customized instructions can be further scheduled to achieve higher instruction-level parallelism. In this article, we design a ubiquitous accelerator with out-of-order automatic parallelization for large-scale data-intensive applications. The accelerator accommodates four representative applications, including clustering algorithms, deep neural networks, genome sequencing, and collaborative filtering. In order to improve the coarse-grained instruction-level parallelism, the accelerator employs an out-of-order scheduling method to enable parallel dataflow computation. We use Colored Petri Net (CPN) tools to analyze the dependences in the applications, and build a hardware prototype on the real FPGA platform. For cluster applications, the accelerator can support four different algorithms, including K-Means, SLINK, PAM, and DBSCAN. For collaborative filtering applications, it accommodates Tanimoto, euclidean, Cosine, and Pearson Correlation as Similarity metrics. For deep learning applications, we implement hardware accelerators for both training process and inference process. Finally, for genome sequencing, we design a hardware accelerator for the BWA-SW algorithm. Experimental results show that the accelerator architecture can reach up to 25X speedup against Intel processors with affordable hardware cost, insignificant power consumption, and high flexibility.

Due to the low power consumption and reconfigurability of FPGA, the use of FPGA for accelerating calculations is becoming more and more hot, including deep learning. However, due to limited hardware resource, single FPGA-based accelerator cannot configure optimal parameters for each layer, and its performance is also limited by the data memory bandwidth. To accelerate the calculation of neural network, this work designs a calculation module, and based on this module, further optimizes the data transmission path between multi-FPGA, thus achieving nearly linear performance growth between performance and the number of FPGAs.

Large-scale graphs processing, which draws attentions of researchers, applies in a large range of domains, such as social networks, web graphs, and transport networks. However, processing large-scale graphs on general processors suffers from difficulties including computation and memory inefficiency. Therefore, the research of hardware accelerator for graph processing has become a hot issue recently. Meanwhile, as a power-efficiency and reconfigurable resource, FPGA is a potential solution to design and employ graph processing algorithms. In this paper, we propose Domino, an asynchronous and energy-efficient hardware accelerator for graph processing. Domino adopts the asynchronous model to process graphs, which is efficient for most of the graph algorithms, such as Breadth-First Search, Depth-First Search, and Single Source Shortest Path. Domino also proposes a specific data structure based on row vector, named Batch Row Vector, to present graphs. Our work adopts the naive update mechanism and bisect update mechanism to perform asynchronous control. Ultimately, we implement Domino on an advanced Xilinx Virtex-7 board, and experimental results demonstrate that Domino has significant performance and energy improvement, especially for graphs with a large diameter(e.g., roadNet-CA and USA-Road). Case studies in Domino achieve 1.47x-7.84x and 0.47x-2.52x average speedup for small-diameter graphs(e.g., com-youtube, WikiTalk, and soc-LiveJournal), over GraphChi on the Intel Core2 and Core i7 processors, respectively. Besides, compared to Intel Core i7 processors, Domino also performs significant energy-efficiency that is 2.03x-10.08x for three small-diameter graphs and 27.98x-134.50x for roadNet-CA which is a graph with relatively large diameter.

Large-scale graphs processing, which draws attentions of researchers, applies in a large range of domains. However, large-scale graphs processing on traditional platforms suffers from difficulties including computation and memory inefficiency. To enhance the computation-efficiency and energy-efficiency, in this paper, we exploit graph processing services on the energy-efficient hardware accelerator, called Domino. Domino adopts the asynchronous model to process graphs, which is efficient for many graph algorithms, such as Breadth-First Search, Depth-First Search, and Single Source Shortest Path. Domino also proposes a specific data structure based on row vectors to present graphs, named Batch Row Vector. Besides, our work employs naive update mechanism and bisect update mechanism to perform the asynchronous control. Ultimately, we implement Domino on an advanced Xilinx Virtex-7 board, and experimental results demonstrated that Domino has a significant performance and energy improvement. Case studies in Domino achieve 1.47x-7.84x and 0.47x-2.52x average speedup for small-diameter graphs(e.g., com-youtube, WikiTalk, and soc-LiveJournal), over GraphChi on the Intel Core2 and Core i7 processors, respectively. Besides, compared to Intel Core i7 processors, Domino also performs a significant energy-efficiency that is 2.03x-10.08x for three small-diameter graphs and 27.98x-134.50x for roadNet-CA which is a graph with relatively large diameter.

Recently, field-programmable gate arrays (FPGAs) have been widely used in the implementations of hardware accelerator for convolutional neural networks (CNNs). However, most of these existing accelerators are designed in the same idea as their ASIC counterparts, in which all operations from different layers are mapped to the same hardware units and working in a multiplexed way. This manner does not take full advantage of reconfigurability and customizability of FPGAs, resulting in a certain degree of computational efficiency degradation. In this paper, we propose a new architecture for FPGA-based CNN accelerator that maps all the layers to their own on-chip units and working concurrently as a pipeline. A comprehensive mapping and optimizing methodology based on establishing roofline model oriented optimization model is proposed, which can achieve maximum resource utilization as well as optimal computational efficiency. Besides, to ease the programming burden, we propose a design framework which can provide a one-stop function for developers to generate the accelerator with our optimizing methodology. We evaluate our proposal by implementing different modern CNN models on Xilinx Zynq-7020 and Virtex-7 690t FPGA platforms. Experimental results show that our implementations can achieve a peak performance of 910.2 GOPS on Virtex-7 690t, and 36.36 GOP/s/W energy efficiency on Zynq-7020, which are superior to the previous approaches.

Neural networks have been widely used in a large range of domains, researchers tune numbers of layrs, neurons and synapses to adapt various applications. As a consequence, computations and memory of neural networks models are both intensive. As large requirements of memory and computing resources, it is difficult to deploy neural networks on resource-limited platforms. Sparse neural networks, which prune redundant neurons and synapses, alleviate computation and memory pressure. However, conventional accelerators cannot benefit from the sparse feature.

Convolutional neural networks(CNNs) have been widely applied in various applications. However, the computation-intensive convolutional layers and memory-intensive fully connected layers have brought many challenges to the implementation of CNN on embedded platforms. To overcome this problem, this work proposes a power-efficient accelerator for CNNs, and different methods are applied to optimize the convolutional layers and fully connected layers. For the convolutional layer, the accelerator first rearranges the input features into matrix on-the-fly when storing them to the on-chip buffers. Thus the computation of convolutional layer can be completed through matrix multiplication. For the fully connected layer, the batch-based method is used to reduce the required memory bandwidth, which also can be completed through matrix multiplication. Then a two-layer pipelined computation method for matrix multiplication is proposed to increase the throughput. As a case study, we implement a widely used CNN model, LeNet-5, on an embedded device. It can achieve a peak performance of 34.48 GOP/s and the power efficiency with the value of 19.45 GOP/s/W under 100MHz clock frequency which outperforms previous approaches.

Large-scale graphs processing attracts more and more attentions, and it has been widely applied in many application domains. FPGA is a promising platform to implement graph processing algorithms with high power-efficiency and parallelism. In this paper, we propose OmniGraph, a scalable hardware accelerator for graph processing. OmniGraph can process graphs with different sizes adaptively and is adaptable to various graph algorithms. OmniGraph improves the preprocessing methodology based on Interval-Shard and consists of three computation engines, vertices on-chip && edges on-chip engine, vertices on-chip && edges off-chip engine, and vertices off-chip && edges off-chip engine. Experimental results on the state-of-the-art Xilinx Virtex-7 board demonstrate that case studies in OmniGraph achieve 1.03x-8.13x average speedup comparing to GraphChi on Intel core2 processors.

Today, artificial neural networks (ANNs) are widely used in a variety of applications, including speech recognition, face detection, disease diagnosis, etc. And as the emerging field of ANNs, Long Short-Term Memory (LSTM) is a recurrent neural network (RNN) which contains complex computational logic. To achieve high accuracy, researchers always build large-scale LSTM networks which are time-consuming and power-consuming. In this paper, we present a hardware accelerator for the LSTM neural network layer based on FPGA Zedboard and use pipeline methods to parallelize the forward computing process. We also implement a sparse LSTM hidden layer, which consumes fewer storage resources than the dense network. Our accelerator is power-efficient and has a higher speed than ARM Cortex-A9 processor.

As a traditional algorithm, the string match meets a challenge with the development of the massive volume of data because of gene sequencing. Surveys show that there will be a huge amount of short read segments during the process of gene sequencing and the need for a highly efficient is urgent. The BWA is an effective algorithm to deal with the short read mapping. Compared with other short read mapping algorithms, the BWA algorithm has a smaller size, and this does not influence its effect. However, there is still not a system is used to accelerate the BWA algorithm especially. Thus we decide to build a system to expedite the algorithm and make it satisfied with the application of gene sequencing. In this paper, we present genome sequencing services on scalable energy-efficient accelerators. Especially, we first introduce the BWA algorithm and claim the reason for the choice of the algorithm. Then, we implement an accelerator based on FPGA to improve the performance of the algorithm. Compared to the other major platforms in accelerating the algorithm, we discuss the advantages of the FPGA platform and the limit of the other platform. Last, we build our hardware platform with a Xilinx ZYNQ FPGA development board, and the result shows that our accelerator can achieve a promising speedup and resource utilization and make it balanced between power and cost.

The Intrusion Detection Systems (IDS) is becoming important and quite timing/space consuming due to the increasing volume of explosive data flood. During the past decades, there have been plenty of studies proposing software mechanisms to exploit the temporal locality in the IDS systems. However, it requires considerable memory blocks to store the redundancy table, therefore, the performance as well as the memory utilization is still worth pursuing. To tackle the above weakness, in this paper, we present xFilter, which explores the temporal locality to capture the redundancy, and propose a novel architecture to store and operate the redundancy table on FPGA. To demonstrate the performance of the xFilter structure, we designed a high efficient accelerator for Aho-Corasick (AC) algorithm used in Snort to detect the attack strings. To show the performance of xFilter, we implement a hardware prototype using Xilinx Zynq FPGA platform. Experimental results show that the xFilter accelerator can achieve 5.1x speedup against software implementation with insignificant hardware cost. Furthermore, the proposed hardware redundancy table mechanism can achieve 1.6x speedup against the traditional hardware accelerator.

Large-scale data is often represented as graphs in the field of modern cloud computing. Graph processing attracts more and more attentions when utilizing the cloud computing service. With the increasing attentions to process massive graphs (e.g., social networks, web graphs, transport networks, and bioinformatics), many state-of-the-art open source graph computing systems on a single node have been proposed, including GraphChi, X-Stream, and GridGraph. GraphChi adopts a vertex-centric model while the latter two adopt an edge-centric model. However, there is a lack of evaluations and analyses to the performance of these systems, which makes it difficult for users to choose the best system for their applications. In this paper, to make the graph processing provide excellent cloud services to users, we propose an evaluation framework, conduct a series of extensive experiments to evaluate the performance and analyze the bottlenecks of these systems on graphs with different characteristics and different kinds of algorithms. The metrics we adopt in this paper are principles to design graph computing systems on a single node, such as RunTime, CPU Utilization, and Data Locality. The results demonstrate the trade-offs among different graph frameworks and X-Stream is more suitable to process transport networks on WCC and BFS, compared to GridGraph. Besides, we present several discussions on GridGraph. The results of our work are concluded as a reference for users, researchers, and developers.

As the emerging field of machine learning, deep learning shows excellent ability in solving complex learning problems. However, the size of the networks becomes increasingly large scale due to the demands of the practical applications, which poses significant challenge to construct a high performance implementations of deep learning neural networks. In order to improve the performance as well as to maintain the low power cost, in this paper we design deep learning accelerator unit (DLAU), which is a scalable accelerator architecture for large-scale deep learning networks using field-programmable gate array (FPGA) as the hardware prototype. The DLAU accelerator employs three pipelined processing units to improve the throughput and utilizes tile techniques to explore locality for deep learning applications. Experimental results on the state-of-the-art Xilinx FPGA board demonstrate that the DLAU accelerator is able to achieve up to 36.1× speedup comparing to the Intel Core2 processors, with the power consumption at 234 mW.