Point cloud diffusion models, as an emerging 3D generation method, hold broad prospects in 3D modeling, AR/VR, and so on. However, their reliance on costly full-precision neural network computations during extended denoising process limits their practical application. To address this challenge, we propose RAPID, an accelerator co-designed with a space-aware quantization method. First, RAPID uses K-means to partition points into groups and computes scaling factors in each, mitigating accuracy issues caused by uneven distribution. Second, it employs a mixed-precision quantization scheme that uses low precision for internal point groups and high precision for detail-rich edge groups, ensuring generation quality while minimizing bit-width. Third, it reuses computation results for groups with little change between timesteps, reducing redundant calculations. Moreover, RAPID’s hardware features a mixed-precision PE array for efficient computations at various bit-widths, and a filter for dynamic bit-width allocation and result reuse. Evaluations show that, compared to the NVIDIA RTX A5000 GPU and state-of-the-art accelerators, RAPID achieves average speedups of 9.22×, 4.66×, 3.69×, and 3.01×, and energy savings of 61.74×, 4.30×, 3.94×, and 2.76×, with negligible accuracy loss.

Point cloud processing has broad applications in autonomous driving and robotics. Serialization-based point cloud transformers map unordered point clouds onto directed curves, use sparse convolution for down-sampling and apply attention in local windows to capture spatial relationships. Despite achieving great accuracy, these models face inference latency challenges: neighbor search in sparse convolution exhibits low parallelism; attention computation remains complex, especially with larger window sizes; softmax introduces data dependencies. This paper proposes APTO, an accelerator for serialization-based models. It uses voxels' z-curve indices to perform neighbor searches in parallel, employs a position-aware pruning strategy using neighboring voxel counts to eliminate useless attention computations, and adopts a fine-grained attention dataflow for parallel processes with minimal data dependencies. Besides, its hardware has dedicated computation cores for efficient processing. Evaluations show that APTO achieves average 10.22×, 3.53× and 2.70× speedups over RTX 4090 GPU, PointAcc, and SpOctA, with 153.59×, 8.57× and 7.25× energy savings.

As network bandwidth outpaces host CPU compute capability, Smart Network Interface Cards (SmartNICs) are increasingly deployed to offload network functions from the host CPU. FPGA-based SmartNICs excel due to their programmability at hardware speed, enabling high-performance and customized offloading. Securing offloaded network functions on FPGA-based SmartNICs is a critical challenge in the cloud, as the sensitive user cannot fully trust the cloud service provider (CSP). CPU Trusted Execution Environments (TEEs) protect software code, not FPGA hardware circuits. Existing FPGA TEEs fail to provide packet I/O protection, System-on-Chip (SoC) CPU utilization, and user-friendly memory access interfaces. To address this gap, we introduce SNO, the first TEE for FPGA-based SmartNICs with the secure boot, the SNO Manager for attestation and network function lifecycle orchestration, and the SNO Guard for I/O encryption and authentication. SNO increases SoC CPU utilization (+6.6% for 8-CPU SoC) by co-locating the SNO Manager with the CSP software while isolating the security-critical components of SNO Manager inside SoC CPU TEE, reduces performance overhead by integrating a fully-pipelined AES-GCM engine and overlapped execution, and offers a user-friendly (86.9% user code reduction) streaming interface. The experimental results show that SNO introduces a relative latency overhead of 7.7–143.2% (corresponding to absolute overheads up to 96 nanoseconds) across five network functions, significantly offset by microsecond-level latency savings from offloading.

This paper presents Hermes, an approach to address two bottlenecks in Open vSwitch (OvS) implemented on Data Processing Units (DPUs). The first bottleneck stems from memory bandwidth contention in the hardware path, while the second bottleneck arises from increased upcalls to software during OpenFlow ruleset updates. Hermes leverages the Range-Query Recursive Model Index (RQRMI) to overcome these bottlenecks through two methods: 1) a three-level hardware path design that combines hash-based flow tables and RQRMI inference module, which reduces memory bandwidth consumption, and 2) a hardware-accelerated training module that enables rapid model retraining for synchronizing hardware path with OpenFlow rulesets. Our prototype shows Hermes increases throughput by up to$3.7 \times$over traditional OvS offloading schemes, while reducing upcalls by 71% and improving throughput by$1.6 \times$during ruleset updates.

Domain-specific architecture has greatly improved performance and energy efficiency in in-memory databases, especially for accelerating single-functional computing logic in analytic query processing, such as sort, join and aggregation. However, as data volumes surge exponentially, these dedicated accelerators are struggling to satisfy the burgeoning demand for handling intricate and multifaceted workloads. A major challenge lies in establishing a flexible framework that engages these ‘coarse-grained’ units without incurring extra overheads from hardware integration, programming, compilation, runtime and operating systems.In this paper, the kernel processing unit (KPU) is proposed to optimize CPU-accelerator heterogeneous systems for in-memory databases. KPU provides a unified interface to consolidate all database query operators. In terms of KPU hardware architecture, kernel customization and data transmission are two critical bottlenecks. To address the challenges, multiple independently designed homogeneous table cores are integrated to support flexible high-performance SQL queries, and a customized efficient data management system (DMS) works collaboratively to maximize the utilization of on-chip memory bandwidth. Additionally, a database application-specific KPU instruction set architecture (KISA) dedicated to parallel analytical query processing is proposed to enable parallel KPU programming. To trade off between accelerator computing capacity and data transfer latency, KPU designs an offloading mechanism to map SQL queries between the CPU and accelerator adaptively based on a performance model and a function simulator. The experiments demonstrate that KPU surpasses the general-purpose CPU and GPU by an average of 24.5× and 8.75×, respectively.

Graphs play an important role in various applications. With the rapid expansion of vertices in real life, existing large-scale graph processing frameworks on CPUs and GPUs encounter challenges in optimizing cache usage due to irregular memory access patterns. To address this, graph reordering has been proposed to improve the locality of the graph, but introduces significant overhead without delivering substantial end-to-end performance improvement. While there have been many FPGA-based accelerators for graph processing, achieving high throughput often requires complex graph prepossessing on CPUs. Therefore, implementing an efficient end-to-end graph processing system remains challenging. This article introduces GRACE, an end-to-end FPGA-based graph processing accelerator with a graph reordering engine and a pull-based vertex-centric programming model (PL-VCPM) Engine. First, GRACE employs a customized high-degree vertex cache (HDC) to improve memory access efficiency. Second, GRACE offloads the graph preprocessing to FPGA. We customize an efficient graph reordering engine to complete preprocessing. Third, GRACE adopts a graph pruning strategy to remove the activation and computation redundancy in graph processing. Finally, GRACE introduces a graph conflict board (GCB) to resolve data conflicts and a multiport cache to enhance parallel efficiency. Experimental results demonstrate that GRACE achieves$7.1 \times $end-to-end performance speedup over CPU and$1.8 \times $over GPU, as well as$27.3 \times $and$8.7 \times $energy efficiency over CPU and GPU. Moreover, GRACE delivers up to$34.9 \times $performance speedup compared to the state-of-the-art FPGA accelerator.

High-resolution (HR) videos have gained popularity with the widespread adoption of high-definition displays. Super-resolution (SR) techniques aim to recover HR frames from low-resolution (LR) frames. While deep neural network (DNN)-based SR methods have outperformed traditional techniques in quality, they face performance challenges. FPGA-based SR accelerators have been developed to optimize the performance and power efficiency. However, most of these accelerators process only uncompressed video frames and perform per-frame DNN inference, overlooking the temporal-spatial information inherent in compressed video bitstreams. We propose a novel compressed video SR workflow that includes a codec information reuse algorithm and a dedicated FPGA accelerator named Co-ViSu. Our approach leverages the observation that non-key frames can be reconstructed using codec information and HR key-frames, significantly reducing DNN computations. The Co-ViSu algorithm employs subpixel interpolation to enhance high-frequency details and an MV-aware method to improve SR reconstruction quality. The Co-ViSu hardware integrates decoder, SR, and encoder engines within a parallel pipeline architecture, utilizing codec information reuse to bypass non-key frame decoding, eliminate complex DNN computations, and accelerate encoding processes. Experimental results demonstrate that Co-ViSu achieves performance improvements ranging from$3.6\times $to$9.4\times $and a$4.2\times $gain in energy efficiency with minimal quality loss compared to traditional flow. Additionally, Co-ViSu offers a$2.1\times $increase in throughput compared to state-of-the-art solutions.

In modern disaggregated datacenters, the storage network which interconnects the compute and memory pools becomes the performance bottleneck. The high-end RDMA devices cannot meet the complex requirements of storage networks, due to the limited RDMA semantics and throughput. Existing solutions essentially follow the monolithic design, so they suffer from underutilized resources and high scaling costs. In this paper, we design TianMen, which offloads the storage network by extending RDMA semantics and customizing communication hardware structure. Specifically, we use DPU as the infrastructure, leveraging the rich storage and compute resources. TianMen enables fully disaggregated storage system that bypasses the server-side CPU, and supports elastic resource pools. Experimental results show that, compared with state-of-the-art solutions: 1) Tian-Men achieves 1 RTT for GET/PUT operations and up to 6× access acceleration; 2) TianMen provides CPU bypass storage network management, including 3.2× speedup of metadata consistency management, per-request load balancing, and 10s microsecond-level fault recovery latency; 3) TianMen saturates the communication bandwidth when processing small payload, increasing the bandwidth utilization by 34.2%. And TianMen achieves 2.27× throughput compared to the commercial RNICs.

Video Frame Interpolation (VFI) aims to generate intermediate frames between consecutive frames. Recent DNN-based VFI offers superior quality but suffers from performance issues. However, very few studies have focused on VFI hardware acceleration and existing work overlooks temporal information from compressed video bitstreams. In this paper, we propose a novel compressed VFI workflow and an accelerator, Co-Via. Co-Via exploits codec information reuse to reduce complex DNN computations and alleviate hardware pressure. FPGA-based Co-Via outperforms an RTX 4090 GPU 10.31X, offering a 43.08X energy efficiency boost. Its ASIC version achieves 2.4X higher throughput and 3.6X energy efficiency than the state-of-the-art solution.

Huffman decoding is crucial in data compression, and the self-synchronization-based parallel decoding algorithm enables subsequence-level parallelism. This paper introduces PHD, the first accelerator designed for self-synchronization-based parallel Huffman decoding on a Field-Programmable Gate Array (FPGA). Designing PHD poses challenges, including managing fine-grained parallelism, addressing limited on-chip memory, and handling inter-codeword dependency. PHD incorporates bit-level, subsequence-level, and tile-level parallelism, utilizes hybrid memory to store the codebook efficiently, and introduces the ONCE MORE optimization to reduce decoding loop iterations. Experimental results demonstrate that PHD outperforms the state-of-the-art GPU-based baseline regarding latency (9.4X to 12.8X reduction) and energy consumption (12.4X to 18.2X reduction).

The minimum spanning tree (MST) plays an important role in variant fields, such as chip design and network analysis. With the rapid expansion of vertices in real-life graphs, the bottleneck problem of MST algorithms in large-scale graphs grows more prominent. While there have been many FPGA-based accelerators for large-scale graph algorithms such as Graph Random Walk, and various algorithms to accelerate MST on CPUs and GPUs, effectively implementing MST algorithms for large-scale graphs on FPGAs remains quite challenging. This is due to several reasons: The neighbor vertices in the graph require extensive random memory access and the memory access characteristics vary across different stages and iterations. There are a large number of useless computations due to the existence of internal edges within a component (intra-edge). Parallel MST algorithm suffers from significant communication overhead due to the minimum edge data update conflicts and memory read-write conflicts.This paper proposes AMST to accelerate large-scale graph MST computation on FPGA. First, AMST employs a customized hash-based high-degree vertex cache (HDC) to improve memory access efficiency. Second, AMST adopts a graph pruning strategy that skips intra-edge and sorts edges by weight to eliminate useless computation and memory access. Finally, AMST utilizes a sorting networking module and a multi-port HDC to improve parallel efficiency. The experimental results demonstrate that AMST achieves an average performance speedup of 17.52× over CPU and 1.89× over GPU, as well as 74.96× over CPU and 10.45× over GPU on energy efficiency.

This paper presents DPU-Direct, an accelerator disaggregation system that connects accelerator nodes (ANs) and CPU nodes (CNs) over a standard Remote Direct Memory Access (RDMA) network. DPU-Direct eliminates the latency introduced by the CPU-based network stack, and PCIe interconnects between network I/O and the accelerator. The DPU-Direct system architecture includes a DPU Wrapper hardware architecture, an RDMA-based Accelerator Access Pattern (RAAP), and a CN-side programming model. The DPU Wrapper connects accelerators directly with the RDMA engine, turning ANs into disaggregation-native devices. The RAAP provides the CN with low-latency and high throughput accelerator semantics based on standard RDMA operations. Our FPGA prototype demonstrates DPU-Direct’s efficacy with two proof-of-concept applications: AES encryption and key-value cache, which are computationally intensive and latency-sensitive. DPU-Direct yields a 400x speedup in AES encryption over the CPU baseline and matches the performance of the locally integrated AES accelerator. For key-value cache, DPU-Direct reduces the average end-to-end latency by 1.66x for GETs and 1.30x for SETs over the CPU-RDMA-Polling baseline, reducing latency jitter by over 10x for both operations.

High-resolution (HR) videos have become popular due to the widespread adoption of high-definition displays. Super-resolution (SR) techniques aim to recover HR frames from low-resolution (LR) frames. Recently, deep neural network (DNN)-based SR methods have achieved superior quality compared to traditional methods. FPGA-based SR accelerators have been proposed to optimize performance and power efficiency. However, most accelerators tailored for video SR only accept uncompressed video frames and operate per-frame DNN inference, ignoring the temporal-spatial information in compressed video bitstreams. In contrast, we observe that non-key frames can be directly constructed using codec information and HR key-frames, saving a significant amount of DNN computing. In this paper, we propose a novel compressed video SR flow and a specific FPGA accelerator called Co-ViSu that integrates decoder, SR, and encoder engines. Co-ViSu exploits codec information reuse scheme to skip non-key frame decoding, avoid complex DNN computation and speed up encoding. Our experimental results show that Co-ViSu achieves 3.6x to 9.4x performance, 4.2x energy efficiency gain with only 0.17dB quality loss compared to the traditional flow, and 2.1x throughput than state-of-the-art.

Video transcoding is a general but compute-intensive technology in video streaming services. Traditional single-encoder accelerators transcode multiple streams independently in the multi-output scenario. However, this mode neglects redundant computation and introduces high hardware complexity. To solve these issues, we propose a multi-encoder accelerator supporting reuse scheme. We introduce four fast algorithms based on parameter sharing to simplify encoding complexity. To further optimize the architecture, we also propose the standalone stream insertion (SSI) to increase the pipeline efficiency, and co-optimize memory access. Implementation results show that multi-encoder can reduce 68.03% computation complexity. Moreover, the area and power efficiency improve 3.05x and 2.62x.

Application-specific accelerator is a prominent way for analytic query processing. To achieve a substantial improvement over the state-of-the-art in performance while maintaining programmability, we propose a kernel processing unit (KPU) framework and apply it to SQL acceleration. Kernel customization and data transmission are two critical bottlenecks, we separately optimize them in the key core and shadow core with a self-designed data management system. A software stack named RACE with a performance model and function simulator is also introduced. The experiments demonstrate that KPU-SQL outperforms the CPU and GPU by 24.5x and 8.75x on average, respectively.

Remote Direct Memory Access (RDMA) is a promising technology for achieving low latency and high bandwidth access to remote memory. However, performance interference exists when multiple tenants share an RDMA Network Interface Card (RNIC) in the cloud environment. Although some initial studies have investigated the root cause and possible solutions to RDMA performance interference, there is no research to analyze and solve the performance interference from the RNIC architecture. Compared with the existing software approach, optimizing RNIC architecture can introduce less performance and CPU overhead. This paper addresses performance isolation by modeling, analyzing, and optimizing the transmit-side (TX) RNIC architecture. First, we introduce a baseline TX RNIC architecture to explain the existing performance interference. Then, we propose separate caching and slicing execution to avoid the bandwidth-sensitive tenants affecting latency-sensitive tenants. Later, we add isolated backpressure and adaptive Weighted Round-robin scheduling to ensure the bandwidth-sensitive tenants share the bandwidth equally. Our experiments show that these optimizations achieve near-optimal performance isolation.

The graph coloring algorithm plays a crucial role in many applications such as social network analysis. However, since the minimal graph coloring problem is NP-complete, which is increasingly computationally and memory-intensive as the number of vertices in the graph grows rapidly. Despite numerous FPGA-based works proposed to accelerate large-scale graph processing algorithms, such as Single Source Shortest Path, and various coloring algorithms, such as linear programming algorithms, efficiently implementing the greedy coloring algorithm for large-scale graphs on FPGA still remains highly challenging due to several reasons: ① The coloring algorithm requires color state traversal to determine the final color after traversing neighbor vertices. The time complexity of color traversal is equal to the neighbor vertices traversal, which is inefficient. ② Neighbor vertices traversal requires extensive random memory accesses on vertex color data. ③ Coloring different vertices in parallel is difficult due to potential color update conflicts between adjacent vertices.

Training large scale convolutional neural networks (CNNs) is an extremely computation and memory intensive task that requires massive computational resources and training time. Recently, many accelerator solutions have been proposed to improve the performance and efficiency of CNNs. Existing approaches mainly focus on the inference phase of CNN, and can hardly address the new challenges posed in CNN training: the resource requirement diversity and bidirectional data dependency between convolutional layers (CVLs) and fully-connected layers (FCLs). To overcome this problem, this paper presents a new accelerator architecture for CNN training, called TNPU, which leverages the complementary effect of the resource requirements between CVLs and FCLs. Unlike prior approaches optimizing CVLs and FCLs in separate way, we take an alternative by smartly orchestrating the computation of CVLs and FCLs in single computing unit to work concurrently so that both computing and memory resources will maintain high utilization, thereby boosting the performance. We also proposed a simplified out-of-order scheduling mechanism to address the bidirectional data dependency issues in CNN training. The experiments show that TNPU achieves a speedup of 1.5x and 1.3x, with an average energy reduction of 35.7% and 24.1% over comparably provisioned state-of-the-art accelerators (DNPU and DaDianNao), respectively.

Convolutional Neural Networks (CNNs) have been widely used in machine learning tasks. While delivering state-of-the-art accuracy, CNNs are known as both compute- and memory-intensive. This paper presents the SqueezeFlow accelerator architecture that exploits sparsity of CNN models for increased efficiency. Unlike prior accelerators that trade complexity for flexibility, SqueezeFlow exploits concise convolution rules to benefit from the reduction of computation and memory accesses as well as the acceleration of existing dense architectures without intrusive PE modifications. Specifically, SqueezeFlow employs a PT-OS-sparse dataflow that removes the ineffective computations while maintaining the regularity of CNN computations. We present a full design down to the layout at 65 nm, with an area of 4.80 mm2and power of 536.09 mW. The experiments show that SqueezeFlow achieves a speedup of 2:9× on VGG16 compared to the dense architectures, with an area and power overhead of only 8.8 and 15.3 percent, respectively. On three representative sparse CNNs, SqueezeFlow improves the performance and energy efficiency by 1:8× and 1:5× over the state-of-the-art sparse accelerators.

There are two approaches to improve the performance of Convolutional Neural Networks (CNNs): 1) accelerating computation and 2) reducing the amount of computation. The acceleration approaches take the advantage of CNN computing regularity which enables abundant fine-grained parallelisms in feature maps, neurons, and synapses. Alternatively, reducing computations leverages the intrinsic sparsity of CNN neurons and synapses. The sparsity represents as the computing “bubbles”, i.e., zero or tiny-valued neurons and synapses. These bubbles can be removed to reduce the volume of computations. Although distinctly different from each other in principle, we find that the two types of approaches are not orthogonal to each other. Even worse, they may conflict to each other when working together. The conditional branches introduced by some bubble-removing mechanisms in the original computations destroy the regularity of deeply nested loops, thereby impairing the intrinsic parallelisms. Therefore, enabling the synergy between the two types of approaches is critical to arrive at superior performance. This paper proposed a relaxed synchronous computing architecture, FlexFlow-Pro, to fulfill this purpose. Compared with the state-of-the-art accelerators, the FlexFlow-Pro gains more than 2.5× performance on average and 2× energy efficiency.

Neural networks (NNs) have achieved great success in a broad range of applications. As NN-based methods are often both computation and memory intensive, accelerator solutions have been proved to be highly promising in terms of both performance and energy efficiency. Although prior solutions can deliver high computational throughput for convolutional layers, they could incur severe performance degradation when accommodating the entire network model, because there exist very diverse computing and memory bandwidth requirements between convolutional layers and fully connected layers and, furthermore, among different NN models. To overcome this problem, we proposed an elastic accelerator architecture, called SynergyFlow, which intrinsically supports layer-level and model-level parallelism for large-scale deep neural networks. SynergyFlow boosts the resource utilization by exploiting the complementary effect of resource demanding in different layers and different NN models. SynergyFlow can dynamically reconfigure itself according to the workload characteristics, maintaining a high performance and high resource utilization among various models. As a case study, we implement SynergyFlow on a P395-AB FPGA board. Under 100MHz working frequency, our implementation improves the performance by 33.8% on average (up to 67.2% on AlexNet) compared to comparable provisioned previous architectures.

Convolutional Neural networks (CNNs) have achieved great success in a broad range of applications. As CNN-based methods are often both computation and memory intensive, sparse CNNs have emerged as an effective solution to reduce the amount of computation and memory accesses while maintaining the high accuracy. However, dense CNN accelerators can hardly benefit from the reduction of computations and memory accesses due to the lack of support for irregular and sparse models. This paper proposed a concise convolution rule (CCR) to diminish the gap between sparse CNNs and dense CNN accelerators. CCR transforms a sparse convolution into multiple effective and ineffective ones. The ineffective convolutions in which either the neurons or synapses are all zeros do not contribute to the final results and the computations and memory accesses can be eliminated. The effective convolutions in which both the neurons and synapses are dense can be easily mapped to the existing dense CNN accelerators. Unlike prior approaches which trade complexity for flexibility, CCR advocates a novel approach to reaping the benefits from the reduction of computation and memory accesses as well as the acceleration of the existing dense architectures without intrusive PE modifications. As a case study, we implemented a sparse CNN accelerator, SparseK, following the rationale of CCR. The experiments show that SparseK achieved a speedup of 2.9× on VGG16 compared to a comparably provisioned dense architecture. Compared with state-of-the-art sparse accelerators, SparseK can improve the performance and energy efficiency by 1.8× and 1.5×, respectively.

Convolutional Neural Network (CNN) accelerators are rapidly growing in popularity as a promising solution for deep learning based applications. Though optimizations on computation have been intensively studied, the energy efficiency of such accelerators remains limited by off-chip memory accesses since their energy cost is magnitudes higher than other operations. Minimizing off-chip memory access volume, therefore, is the key to further improving energy efficiency. However, we observed that sticking to minimizing the accesses of one data type as many prior work did cannot fit the varying shapes of convolutional layers in CNNs. Hence, there exists a dilemma of minimizing the accesses of which data type. To overcome the problem, this paper proposed an adaptive layer partitioning and scheduling scheme, called SmartShuttle, to minimize off-chip memory accesses for CNN accelerators. Smartshuttle can adaptively switch among different data reuse schemes and the corresponding tiling factor settings to dynamically match different convolutional layers. Moreover, SmartShuttle thoroughly investigates the impact of data reusability and sparsity on the memory access volume. The experimental results show that SmartShuttle processes the convolutional layers at 434.8 multiply and accumulations (MACs)/DRAM access for VGG16 (batch size = 3), and 526.3 MACs/DRAM access for AlexNet (batch size = 4), which outperforms the state-of-the-art approach (Eyeriss) by 52.2% and 52.6%, respectively.

Understanding of computing skills and concepts is no longer viewed as optional, but is knowledge that is required to progress in several fields outside of computer science, even outside STEM fields. Despite the need for this knowledge, non-STEM majors' participation in computing courses is still lacking. This is especially true for women and other minority groups. In our work, our main motivation was creating a course at our institution, Bucknell University to make computer science more accessible to non-STEM majors on our campus. We adopted an approach of situating course design in the experiences and perspectives of undergraduate students themselves. To achieve this, the course curriculum and materials were mainly designed and created by three undergraduate students, all below the junior level, who connected course topics to a wide variety of topics outside computing. This project is based on the precept that students from varied disciplinary backgrounds and intermediate computing knowledge can provide a unique lens to articulate the needs of their fellow students. In order to serve the purpose of emphasizing creativity and society, we chose a sketch based programming language, namely p5.js (a JavaScript library which brings Processing to the web), which gives students the possibility and flexibility to create projects that are directly relevant to their own interests, while gaining a basic conceptual understanding of Computer Science