3D Gaussian Splatting (3DGS) has emerged as a powerful technique for novel view synthesis, combining high-quality reconstruction with efficient rendering. It has been widely adopted in domains such as AR/VR, robotics, and autonomous driving. However, achieving real-time performance on resource-constrained platforms remains challenging due to strict power and area budgets. Prior accelerators improve hardware performance but still overlook key inefficiencies, including insufficient rasterization efficiency, poor sorting scalability, and pipeline imbalance. This paper presents an architecture-algorithm co-design to address these challenges. First, we propose axis-shared rasterization, which precomputes and reuses common terms along the X- and Y-axes, reducing multiply-and-accumulate (MAC) operations by up to 38% while preserving high parallelism. Second, we develop a novel order-independent transmittance method that removes the need for explicit sorting by leveraging a lightweight multilayer perceptron (MLP) to directly approximate the transmittance of each Gaussian, enabling efficient alpha blending with negligible quality loss. Third, we design a unified reconfigurable PE array that supports both rasterization and MLP inference, sustaining high utilization without costly sorting hardware. Our experiments demonstrate that our design preserves rendering quality while achieving a 1.33 to 1.88x speedup over state-of-the-art 3DGS accelerators. Our code is open source at https://github.com/WangZhican/ISCA26_3DGS_Acc.

Mixture-of-Experts (MoE) has been adopted by many leading large models to reduce computational requirements. However, frequent inter-GPU communication in MoE expert parallelism (EP) becomes a performance challenge. We observe substantial redundant inter-GPU data transfers in MoE that can be potentially addressed by in-switch computing. Unfortunately, the existing solution, NVLink SHARP (NVLS), can only support static collectives with regular patterns, incapable of dynamic communication with irregular patterns in MoE. To bridge the functionality gap, we propose DySHARP, an integral dynamic in-switch computing solution to accelerate MoE, encompassing both communication primitives and communication-aware scheduling: 1) Dynamic multimem addressing co-designs ISA, architecture, and runtime, as a dynamic extension to NVLS, reducing redundant traffic. However, the resulting traffic reduction is inherently asymmetric between two directions, preventing it from directly translating into speedup. 2) Token-centric kernel fusion deeply fuses the dispatch-computation-combine pipeline, resolving this asymmetry to translate traffic reduction into actual speedup. Compared with the state-of-the-art solution, DySHARP achieves up to 1.79× speedup.

The lithography process inherently introduces device-to-device variation in the fabrication of resistive random-access memory (RRAM) array, introducing electrical mismatch and limiting the practical applications of RRAM-based analog computing circuits. In this work, we propose a lithography model-aware layout synthesis framework to minimize the proximity effect in the lithography process, thus reducing the electrical mismatch amongst these devices for analog computing applications. A dummy RRAM cell insertion technique is proposed to reduce the geometrical mismatch among RRAM cells, and a bi-objective alternate optimization method is proposed to efficiently optimize the geometric parameters and the structure of RRAM layouts. In addition, an approximation method for evaluating the quality of the RRAM array layout is proposed to reduce the runtime of synthesis. The experimental results show that our proposed framework significantly reduces the deviation between printed and expected patterns.

Multichiplet deep neural network (DNN) accelerators have evolved as promising modular solutions, offering enhanced performance, scalability, and cost-effectiveness. These architectures, however, suffer from the escalating communication bottleneck with increasing scale, primarily stemming from rising hop count and worsening link underutilization. The bottleneck is exacerbated by conventional fixed interconnection networks’ nonadaptability to diverse DNN dataflows. Moreover, an efficient routing tailored for large-scale networks with deadlock-freedom is needed for performance. To address these scalability challenges, leveraging the low-latency links and abundant interconnection resources with reconfigurability in the active interposer, we propose HiRe, a hierarchical reconfigurable network-routing co-design architecture for large-scale multichiplet DNN accelerators. The architecture introduces reconfigurable nodes (RNs) across on-chip and interchiplet hierarchical networks, enabling dynamic bypassing and network reconfiguration. Based on the network, it incorporates an efficient deadlock-free routing that combines simulated annealing (SA)-based communication scheduling with greedy path selection. Through the network-routing co-design, HiRe reduces the hop count and enhances the link utilization. The HiRe architecture is implemented and synthesized in a 55-nm CMOS process. Experimental results demonstrate that HiRe achieves a 14.3%–45.2% EDP reduction and a 14.1%–35.0% latency reduction compared to state-of-the-art (SOTA) innovations, effectively mitigating the large-scale communication bottleneck.

In this work, an energy-efficient bit-parallel static random-access memory (SRAM)-based computing-in-memory (SRAM-CIM) is proposed for general-purpose in-memory arithmetic operations to adapt diverse computing tasks. A compact diagonal-block parallel (DBP) mapping scheme and a novel arithmetic flow are proposed to address the hardware underutilization issue in conventional two-sided stationary bit-parallel CIM architectures. Specifically, the DBP mapping method is implemented to enhance the throughput by reorganizing the intermediate and final results into diagonal memory blocks, effectively reducing the vacant CIM cells caused by the dynamic bit width during computing. In addition, the proposed hardware-efficient arithmetic flows employ: 1) a pipelined ADD scheme to reduce the critical path latency in near-memory computing units; and 2) shift-based arithmetic operations that halve the hardware resources required for multiplication and division while reducing energy consumption. The post-layout simulations on 28-nm CMOS technology show that the proposed DBP-CIM achieves higher energy efficiency and throughput for general-purpose arithmetic operations, compared with state-of-the-art works. Furthermore, evaluations on the general-purpose benchmarks demonstrate that the DBP-CIM reduces energy consumption and computing cycles by up to 55.9% and 60.9%, compared to the conventional bit-parallel CIM.

Heterogeneous computing-in-memory (CIM) offers a promising solution for efficient neural network (NN) accelerations by leveraging the characteristics of different types of memories. However, the previous heterogeneous CIM either requires additional data transfer between different types of isolated memories or solely relies on in situ embedded single-level (SL) or three-level (TL) nonvolatile memories (NVMs), making it difficult to trade off between storage density and robustness benefits. In this article, a new heterogeneous CIM architecture (NVS-SPT) with enhanced storage density and inference robustness is proposed to enable full on-chip acceleration of practical-scale NNs while scalable to larger models. A splittable nonvolatile computing-in-static random access memory (SRAM) cell pairs (nvS2RAM-CIM) is proposed with hybrid in situ embedded SL and TL resistive random access memory (ReRAM) groups, allowing flexible configuration as split or linked state to enhance storage density and restore yield. A layerwise hybrid-coding search (LHCS) algorithm with bitwise and tritwise data-aware mapping (BTM) method is proposed to determine the optimal weight coding patterns with high array utilizations. In addition, a merged hybrid-coding block (MHCB) generation scheme is employed to enable high computing parallelism by merging the dense computing patterns. The proposed NVS-SPT demonstrates up to$4.2\times $higher storage density compared with previous heterogeneous CIM with pure SL-ReRAMs and achieves up to 44.7% enhanced NN accuracy, compared with previous unified ternary coding. Furthermore, the proposed NVS-SPT exhibits up to$1.72\times $and$1.44\times $enhanced energy efficiency with$3.10\times $and$1.42\times $higher computing density, compared with previous heterogeneous CIM based on pure SL- or TL-ReRAMs, respectively.

With the widespread deployment of long-context large language models (LLMs), efficient and high-quality generation is becoming increasingly important. Modern LLMs employ batching and key-value (KV) cache to improve generation throughput and quality. However, as the context length and batch size rise drastically, the KV cache incurs extreme external memory access (EMA) issues. Recent LLM accelerators face substantial processing element (PE) under-utilization due to the low arithmetic intensity of attention with KV cache, while existing KV cache compression algorithms struggle with hardware inefficiency or significant accuracy degradation. To address these issues, an algorithm-architecture co-optimization, KVO-LLM, is proposed for long-context batched LLM generation. At the algorithm level, we propose a KV cache quantization-aware pruning method that first adopts salient-token-aware quantization and then prunes KV channels and tokens by attention guided pruning based on salient tokens identified during quantization. Achieving substantial savings on hardware overhead, our algorithm reduces the EMA of KV cache over 91% with significant accuracy advantages compared to previous KV cache compression algorithms. At the architecture level, we propose a multi-core jointly optimized accelerator that adopts operator fusion and cross-batch interleaving strategy, maximizing PE and DRAM bandwidth utilization. Compared to the state-of-the-art LLM accelerators, KVO-LLM improves generation throughput by up to $7.32 \times$, and attains $5.52 \sim 8.38 \times$ better energy efficiency.

Bit-serial computation shows promise for accelerating deep neural networks (DNNs) by exploiting inherent bit sparsity. However, the original unstructured bit sparsity poses two major challenges for existing bit-serial accelerators (BSA): (1) workload imbalance from irregular bit distribution, and (2) inefficient memory access due to unpredictable non-zero bit locations. To address these issues, this paper proposes BitPattern, an algorithm/hardware co-design to efficiently accelerate bitserial computation through bit-pattern pruning. At the algorithm level, we employ bit-pattern pruning to identify optimal combinations of predefined patterns and apply compression encoding to minimize weight storage. We further devise a pattern-similaritybased merging method to balance the bit-serial workload. At the hardware level, we co-design a bit-serial accelerator with a dedicated bit-pattern decoder and PE to leverage the potential of structured bit-pattern sparsity. The evaluation on several deep learning benchmarks shows that BitPattern can achieve $1.72 \times$ memory reduction with negligible accuracy loss, and up to $2.11 \times$ speedup and $1.86 \times$ energy saving compared to state-of-the-art bit-serial accelerators.

Large Language Models (LLMs) excel in natural language processing tasks but pose significant computational and memory challenges for edge deployment due to their intensive resource demands. This work addresses the efficiency of LLM inference by algorithm-hardwaredataflow tri-optimizations. We propose a novel voting-based KV cache eviction algorithm, balancing hardware efficiency and algorithm accuracy by adaptively identifying unimportant kv vectors. From a dataflow perspective, we introduce a flexible-product dataflow and a runtime reconfigurable PE array for matrix-vector multiplication. The proposed approach effectively handles the diverse dimensional requirements and solves the challenges of incrementally varying sequence lengths. Additionally, an element-serial scheduling scheme is proposed for nonlinear operations, such as softmax and layer normalization (layernorm). Results demonstrate a substantial reduction in latency, accompanied by a significant decrease in hardware complexity, from $O(N)$ to $O(1)$. The proposed solution is realized in a custom-designed accelerator, VEDA, which outperforms existing hardware platforms. This research represents a significant advancement in LLM inference on resource-constrained edge devices, facilitating real-time processing, enhancing data privacy, and enabling model customization.

Transformers have become prevalent in various Artificial Intelligence (AI) applications, spanning natural language processing to computer vision. Owing to their suboptimal performance on general-purpose platforms, various domain-specific accelerators that explore and utilize the model sparsity have been developed. Instead, we conduct a quantitative analysis of Transformers. (Transformers can be categorized into three types: Encoder-Only, Decoder-Only, and Encoder-Decoder. This paper focuses on Encoder-Only Transformers.) to identify key inefficiencies and adopt dataflow optimization to address them. These inefficiencies arise from1)diverse matrix multiplication,2)multi-phase non-linear operations and their dependencies, and3)heavy memory requirements. We introduce a novel dataflow design to support decoupling with latency hiding, effectively reducing the dependencies and addressing the performance bottlenecks of nonlinear operations. To enable fully fused attention computation, we propose practical tiling and mapping strategies to sustain high throughput and notably decrease memory requirements from$O(N^{2}H)$to$O(N)$. A hybrid buffer-level reuse strategy is also introduced to enhance utilization and diminish off-chip access. Based on these optimizations, we propose a novel systolic array design, named DESA, with three innovations:1)A reconfigurable vector processing unit (VPU) and immediate processing units (IPUs) that can be seamlessly fused within the systolic array to support various normalization, post-processing, and transposition operations with efficient latency hiding.2)A hybrid stationary systolic array that improves the compute and memory efficiency for matrix multiplications with diverse operational intensity and characteristics.3)A novel tile fusion processing that efficiently addresses the low utilization issue in the conventional systolic array during the data setup and offloading. Across various benchmarks, extensive experiments demonstrate that DESA archives$5.0\boldsymbol{\times\thicksim}8.3\boldsymbol{\times}$energy saving over 3090 GPU and$25.6\boldsymbol{\times\thicksim}88.4\boldsymbol{\times}$than Intel 6226R CPU. Compared to the SOTA designs, DESA achieves$11.6\boldsymbol{\times\thicksim}15.0\boldsymbol{\times}$speedup and up to$2.3\times$energy saving over the SOTA accelerators.

Neural radiance field has become a fundamental rendering technique across diverse applications such as augmented/virtual reality and autonomous driving. It achieves exceptional rendering quality and reduces model construction cost mainly by introducing a novel neural representation, Instant Neural Graphics Primitives (Instant-NGP). Despite its superiority, Instant-NGP poses severe problems of intensive computation, memory inefficiency and pipeline inefficiency, owing to numerous neural network queries, irregular memory access and intricate sampling procedure. To address these issues, this article proposes NeRA, an algorithm-architecture co-optimization framework that facilitates the efficient neural rendering of Instant-NGP. For intensive computation, we reconstruct the rendering flow and propose a deferred neural decoding algorithm to aggregate the network queries, which reduces the computational workload by 85.6% and only incurs <0.5 % loss in rendering quality. For memory inefficiency, a base-interpolation algorithm is proposed to condense the scattered memory access and improves the equivalent bandwidth of on-chip memory by 2.38×. Furthermore, a voxel-centric data flow is proposed to fully reuse the cached data and save 88.7% of the external memory access. For pipeline inefficiency, a highly-pipelined hardware architecture with decoupled spatial skipping and interleaved sampling is constructed to eliminate the bubbles and invalid samples in the pipeline, which boosts the overall throughput by 2.41×. Extensively evaluated on representative benchmarks, NeRA attains 1.2 2.9× in rendering throughput, 1.7 36.5× in energy-efficiency and 3.6 8.3× in area-efficiency, compared to the state-of-the-art related architectures.

The attention mechanism is becoming a vital building block across various modern neural networks, e.g., Transformers. However, it encounters low efficiency when deployed on the general-purpose GPU/CPU platform, which motivates the dedicated accelerator design. Existing accelerators are commonly devised by exploring the potential sparsity in attention mechanism using a hardware-software codesign scheme, which suffers from complicated training, fine-tuning processes, and possible accuracy degradation. More importantly, the sparse pattern only focuses on certain datasets with less generality, and the fine-grained sparse pattern could also bring hardware inefficiency. Instead, we try to solve these issues from another perspective: by systematically analysing the inherent dataflow characteristics of the attention mechanism, we propose the co-operative systolic arrays (COSAs) with an optimized dataflow to support the general purpose attention mechanism and pursue higher computational efficiency. COSA system exploits the high parallelism from the inherent model and leverages run-time configurable hybrid dataflows, i.e., weight and output stationary (OS) for a systolic array (SA) to support the varying matrix multiplication in the attention mechanism. Regarding the cascaded matrix multiplications, COSA proposes levels of fusion methodologies to reduce the off-chip access and enhance processing element (PE) utilization, such as directly using the result of OS as the weight of weight stationary SA by deep fusion. Additionally, the COSA system also provides the solution to hide the latency and radically save the buffer size related to the softmax. Experiment results show that, across various benchmarks, COSA can achieve$2.29-2.60\times $throughput improvement over the traditional SA of the same MAC number, with up to 94.7% PE utilization rate and$8.2\times $less off-chip memory access. Compared with the general-purpose platforms,$7.6-12.4\times $energy efficiency over NVIDIA GeForce 3090 GPU and$35.2-80.9\times $energy efficiency over Intel 6226R server CPU.

Hybrid convolutional neural network (CNN) and Transformer networks are emerging in computer vision, combining convolutional, linear, and attention layers to achieve high accuracies with moderate model sizes. Developing the accelerators for hybrid networks is pivotal to simultaneously optimize the static matrix multiplication (MM) in convolutional and linear layers, as well as dynamic MM in attention layers. However, the existing accelerators are primarily designed for either CNNs or Transformers, resulting in increased data movement to support dynamic MM and potential under-utilization of hardware for static MM. To enhance computational performance and energy efficiency for hybrid networks, we propose HyCTor, an accelerator featuring flexible output-stationary (OS) and weight-stationary (WS) dataflows, along with a multicore extension for higher throughput. The parallel array of HyCTor supports interlayer slicing and intralayer splicing to improve the utilization for static MM, and enables seamless switching between OS and WS dataflow to minimize the data movement in dynamic MM. By leveraging structured sparsity in OS dataflow and unstructured sparsity in WS dataflow, the computational efficiency is further boosted for each layer through flexible dataflow selection based on the sparsity ratio. Besides, a novel QuadLoop-mesh topology is proposed to address the complex data dependencies in hybrid networks and minimize data transmission distances in the multicore HyCTor. Experimental results on ResNet-18, ViT-B, and TransIAR-AF show that the proposed single-core HyCTor achieves$1.83\times $,$1.65\times $, and$2.41\times $speedup than state-of-the-art (SOTA) accelerators with 100% utilization rate in most layers, and$3.82\times $–$38.5\times $speedup than RTX4090 GPU. The energy efficiency of HyCTor is improved by$1.81\times $–$8.77\times $compared with SOTA accelerators. Moreover, the 4-core HyCTor achieves speedups of$3.32\times $,$2.58\times $, and$2.91\times $, while the 16-core HyCTor achieves speedups of$7.05\times $,$4.05\times $, and$9.64\times $compared to 1-core HyCTor on three networks.

Vision-centric 3D perception has become a key mechanism in autonomous driving. It achieves exceptional perceptual performance mainly by introducing a novel attention,multi-view cross-attention(MVCA), for learnable feature extraction and fusion from surround-view cameras. Despite its superiority, MVCA encounters severe inefficiencies in sample, processing elements (PE), and pipelined processing, owing to the redundant and non-uniform sampling-aggregation and rigorous inter-operator dependencies. To address these issues, this article proposes a dedicated MVCA accelerator, MVAtor, with algorithm-architecture co-optimization for vision-centric 3D perception based on multi-view inputs flexibly. For sample inefficiency, a 3-tier hybrid static-dynamic sample and a sensitivity-aware feature pruning approach are proposed to eliminate the 86.03% sample overhead and 24.48% memory requirement, only incuring <1% accuracy loss with no need of fine-tuning. For PE inefficiency, a spatial pruner and sequential sampler collaboration strategy is proposed to improve the sampler utilization without compromising pruner’s throughput, which outperforms the previous design by 53.7~96.1% energy-delay product reduction. For pipeline inefficiency, a fine-grained-tiling assisted highly-pipelined architecture is constructed in MVAtor by exploiting the decoupling opportunities on inter-view sparsity, thereby saving 61.03% external memory access while boosting the overall throughputs by 1.83×. Extensively evaluated on representative benchmarks, MVAtor attains 1.38~7.67× and 1.67~11.15× improvement on energy and area efficiency respectively, compared to the state-of-the-art related accelerators.

Large Language Models (LLMs) have achieved significant success in various Natural Language Processing (NLP) tasks, becoming essential to modern intelligent computing. Their large memory footprint and high computational cost hinder efficient deployment. Post-Training Quantization (PTQ) is a promising technique to alleviate this issue and accelerate LLM inference. However, the presence of outliers impedes the advancement of LLM quantization to lower bit levels. In this paper, we introduce OFQ-LLM, an algorithm-hardware co-design solution that adopts outlier-flexing quantization to efficiently accelerate LLM at low-bit levels. The key insight of OFQ-LLM is that normal data can be efficiently quantized in a slightly reduced data encoding space, while the rest encoding space can be used for flexible outlier values. During quantization, we use rescale-based clipping (RBC) to optimize accuracy for normal data and group outlier clustering (GOC) to flexibly represent outlier values. At the hardware level, we introduce a memory-aligned outlier-flexing encoding scheme to encode activations and weights in LLMs at a low bit level. The outlier-normal mixed hardware architecture is devised to leverage the encoding scheme and accelerate LLMs with high speed and high energy efficiency. Our experiments show that OFQ-LLM achieves better accuracy compared to state-of-the-art (SOTA) low-bit LLM PTQ works. OFQ-LLM-based accelerator surpasses the SOTA outlier-aware accelerators by up to$2.69\times $core energy efficiency, up to$3.83\times $speed up and$2.44\times $energy reduction in LLM prefilling phase, and up to$2.01\times $speed up and$2.88\times $energy reduction in LLM decoding phase, with superior accuracy.

Multi-scale deformable attention (MSDeformAttn) has emerged as a key mechanism in various vision tasks, demonstrating explicit superiority attributed to multi-scale grid-sampling. However, this newly introduced operator incurs irregular data access and enormous memory requirement, leading to severe PE under-utilization. Meanwhile, existing approaches for attention acceleration cannot be directly applied to MSDeformAttn due to lack of support for this distinct procedure. Therefore, we propose a dedicated algorithm-architecture co-design dubbed DEFA, the first-of-its-kind method for MSDeformAttn acceleration. At the algorithm level, DEFA adopts frequency-weighted pruning and probability-aware pruning for feature maps and sampling points respectively, alleviating the memory footprint by over 80%. At the architecture level, it explores the multi-scale parallelism to boost the throughput significantly and further reduces the memory access via fine-grained layer fusion and feature map reusing. Extensively evaluated on representative benchmarks, DEFA achieves 10.1-31.9X speedup and 20.3-37.7X energy efficiency boost compared to powerful GPU platforms. It also rivals the related accelerators by 2.2-3.7X energy efficiency improvement while providing pioneering support of MSDeformAttn.

This paper addresses two key limitations in existing Image Signal Processing (ISP) approaches: the suboptimal performance in low-light conditions and the lack of trainability in traditional ISP methods. To tackle these issues, we propose a novel, trainable ISP framework that incorporates both the strengths of traditional ISP techniques and advanced Multi-Scale Retinex (MSR) algorithms for night-time enhancement. Our method consists of three primary components: an ISP-based Luminance Harmonization layer to initially optimize luminance levels in RAW data, a deep learning-based MSR layer for nuanced decomposition of image components, and a specialized enhancement layer for both precise, region-specific luminance enhancement and color denoising. The proposed approach is validated through rigorous experiments on machine vision benchmarks and objective visual quality indicators. Our results demonstrate not only a significant improvement over existing methods but also robust adaptability under diverse lighting conditions. This work offers a versatile ISP framework with promising applications beyond its immediate scope.

As neural networks advance, the diversity and latency proportion of non-matrix-multiplication operators (NMO) are on the rise. Providing a versatile and efficient acceleration for this intricate set of NMOs poses great challenges in hardware design. In this work, we analyze the algorithmic structure of NMOs and propose graph-based ALU (GALU) to improve efficiency. The key idea is to organize functional units into a dataflow graph with a configurable interconnection, which reduces computation time and memory access. Further, we provide architectural support to integrate GALU into a multi-thread processor called VEGA, which supports various NMO structures. At the hardware level, we devise swift interconnection reconfiguration (SIR) for GALU to reduce the latency caused by reconfiguration. We also design a fine-grained instruction scheduler to fully utilize SIR. At the software level, a three-stage compilation framework is developed to enhance the usability. Experiments demonstrate that GALU achieves a 2.27x speedup with only an 18.11 % increase in area overhead. Compared with NVIDIA Jetson Orin, the VEGA prototype achieves a 3.84x speedup on typical NMOs and achieves a 2.03 x end-to-end speedup at the network level.

Image compression is an essential technique for graphics processing units to support high-resolution video and high-quality 3D rendering. Many works sacrifice compression ratio (CR) or image quality in order to achieve a higher throughput or lower latency. In this paper, a lossless high-throughput compression-decompression engine optimized for CR is proposed. At the algorithm level, the engine utilizes pixel inter-channel correlation and positional correlation to jointly improve CR. At the hardware level, the diagonal-level parallel decoding (DLPD) is used to obtain a high throughput (44.1 Gbytes/sec). The proposed engine is designed and validated on the JPEG AIC-3 dataset. The average CR on the dataset is 2.23, an improvement of 0.35 over the state-of-the-art high-throughput work with a 24.2 % increase in throughput.

Multi-chiplet DNN accelerator is a promising solution to balancing performance and cost. However, the limited communication bandwidth between chiplets exacerbates the performance bottleneck of the interconnect network. Besides, in DNN dataflows, the same weight or activation is often shared by multiple processing elements (PEs). One-to-many dataflows, also known as multicast, are widespread. Existing works lack specific optimizations for DNN dataflows, thereby yielding suboptimal multicast efficiency. To overcome these challenges, we propose MEIN, a Multicast-Efficient Interconnect Network for multi-chiplet DNN accelerators. Firstly, we introduce a highly efficient routing algorithm tailored for DNN dataflows. It optimizes the multicast tree structure to reduce path latency, while also improving the path selection mechanism to minimize link contention. Secondly, we propose a lightweight router microarchitecture that enhances the hardware resource utilization by simplifying multicast ports. Based on the gem5 simulator, our evaluation demonstrates that MEIN achieves the latency reduction by 19.6%-81.9% as compared to the state-of-the-art related works.

As a primary component of Transformers, attention mechanism suffers from quadratic computational complexity. To achieve efficient implementations, its hardware accelerator designs have aroused great research interest. However, most existing accelerators only support a single type of application and a single type of attention, making it difficult to meet the demands of diverse application scenarios. Additionally, they mainly focus on the dynamic pruning of attention matrices, which requires the deployment of pre-processing units, thereby reducing overall hardware efficiency. This paper presents CoDA which is an algorithm, dataflow and architecture co-design framework for versatile and efficient attention accelerators. The designed accelerator supports both NLP and CV applications, and can be configured into the mode supporting low-rank attention or low-rank plus sparse attention. We apply algorithmic transformations to low-rank attention to significantly reduce computational complexity. To prevent an increase in storage overhead resulting from the proposed algorithmic transformations, we carefully design the dataflows and adopt a block-wise fashion. Down-scaling softmax is further supported by architecture and dataflow co-design. Moreover, we propose a softmax sharing strategy to reduce the area cost. Our experiment results demonstrate that the proposed accelerator outperforms the state-of-the-art designs in terms of throughput, area efficiency and energy efficiency.

To meet the demand in a wide range of practical applications, precision-scalable deep neural network (DNN) accelerators are becoming an unavoidable trend. On the other hand, it has been demonstrated that a DNN accelerator may achieve better computation efficiency through exploiting the sparsity. Therefore, DNN accelerators with both precision scalability and sparsity exploitation are expected to have better performance. In this article, we propose an efficient precision-scalable DNN accelerator that can exploit the sparsity of activations. The precision scalability is obtained from the decomposable multiplier which is inspired by the well-known design, Bit Fusion. Besides, a zero-skipping scheme is adopted to leverage the inherent sparsity of activations. We first modify the architecture of the conventional fusion unit (FU) to make it amenable to the zero-skipping scheme. Then, a segmentation approach is devised to tackle the memory access conflict. Furthermore, a sparsity-aware mapping method is proposed to balance the workload of processing elements (PEs). Moreover, we present a bit-splitting strategy which can take advantage of the sparsity in the bit level. Compared with the state-of-the-art precision-scalable designs, our proposed accelerator can provide speedups of$4.12\times $,$4.07\times $, and$6.62\times $in the precision modes$8b\times 8b$,$4b\times 4b$, and$2b\times 2b$, respectively. Meanwhile, it also achieves$3.92\times $peak area efficiency and competitive peak energy efficiency.

Chiplet-based deep neural network (DNN) accelerator is a promising solution to balance the performance and manufacturing cost. However, different from monolithic chips, interconnect network design and architectural partitioning for multiple chiplets would result in a huge design space and make it difficult to keep scalability and high hardware utilization. Moreover, how to efficiently map DNN workloads onto multiple DRAM dies and compute dies is another major challenge. To alleviate the above issues, in this work, we propose INDM, a chiplet-based interconnect network and dataflow mapping co-optimization for DNN accelerators. First, we propose an efficient hierarchical interconnect network composed of a multiring on-die network and a cluster-based interdie network, to facilitate the data reuse and traffic pattern in DNN workloads. Second, architectural partitioning and topology exploration for chiplet-based DNN accelerators are proposed to find the optimal architecture configurations. Third, an interdie communication-aware dataflow mapping is proposed to minimize traffic congestion during DNN layer switching. We implement the proposed chiplet-based interconnect network design and dataflow mapping algorithm for a set of popular DNN models, including VGG-16, ResNet-18, DarkNet-19, ResNet-50, and ResNet-101. Experimental results show that as compared with the state-of-the-art related work, such as NN-Baton and SIMBA, our work achieves 26.00%–73.81% energy-delay-product (EDP) reduction and 26.93%–79.78% latency reduction.

Large language models (LLMs) have sparked a new revolution in the field of natural language processing (NLP), and have garnered tremendous attention in both academic research and everyday life, thanks to their unprecedented performance in a wide range of applications. However, their deployment remains a significant challenge, primarily due to their intensive computational and memory requirements. Hardware acceleration and efficient quantization are promising solutions to address the two issues. In this paper, a quantization and hardware architecture co-design is presented for matrix-vector multiplications (MVMs) of LLMs. During quantization, we uniformly group weights and activations to ensure workload balance for hardware. To enhance the performance of quantization, we further propose two approaches called channel sorting and channel selection, which can be applied simultaneously. To support the proposed quantization scheme, we develop two precision-scalable MVM hardware architectures. They are specifically designed for high speed and high energy efficiency, respectively. Experimental results show that our proposed quantization scheme achieves state-of-the-art performance among all the reported post-training schemes that quantize both weights and activations into integers. Compared to MVM architecture of the state-of-the-art LLM accelerator OliVe, our design exhibits significant advantages in terms of area efficiency and energy efficiency.

Vision Transformers (ViTs) have achieved remarkable success in computer vision (CV) and are increasingly recognized as the new backbone for vision-language multi-modal tasks. Despite their success, the high computational cost associated with ViTs hinders their inference efficiency. In this paper, we introduce BSViT, a bit-serial Vision Transformer accelerator enhanced by algorithm-hardware co-design. BSViT can efficiently accelerate both plain and hierarchical Vision Transformer inference. At the algorithm level, we propose a post-training quantization scheme named dynamic patch and weight bit-group quantization. We first introduce a dynamic patch quantization (DPQ) scheme to dynamically allocate bit-width to different image patches based on their importance, thus reducing bit width and saving computation without significantly impacting accuracy. Second, we propose a weight bit-group quantization (BGQ) scheme to evenly distribute bits within groups and achieve workload balance across processing elements (PEs). At the hardware level, we propose a term-separate bit-serial accelerator to efficiently support DPQ and BGQ. We introduce dense and sparse bit-serial PEs to manipulate the dense least significant term (LST) and sparse most significant term (MST) workloads. A dense-sparse hybrid dataflow is devised to efficiently balance the two kinds of workloads. Our experiments show that BSViT can achieve up to$1.95\times $speedup and$2.72\times $energy efficiency compared to state-of-the-art (SOTA) bit-serial accelerators and achieve up to$3.69\times $energy efficiency compared to SOTA Transformer accelerators.

With the advancement of artificial intelligence, the collaboration of multiple deep neural networks (DNNs) has been crucial to existing embedded systems and cloud systems, especially for automatic driving applications as well as augmented and virtual reality (AR/VR) applications. To trade off between cost and performance, chiplet-based DNN accelerators have emerged as a promising solution for accelerating DNN workloads. However, most existing mapping methods for multiple DNNs target for the monolithic chip, which fail to solve the problems faced by the emerging multi-chiplet architecture, such as the problems of distributed memory access, complex heterogeneous interconnect network, and the scaling-up of computing resources. In this work, we propose M2M, a fine-grained mapping framework for accelerating multiple DNNs on a multi-chiplet architecture. It includes a temporal and spatial task scheduling for reconfigurable dataflow accelerators and a communication-aware task mapping in a heterogeneous interconnect network. To enhance communication efficiency and reduce the overall latency, we further propose a fine-tuned quality-of-service (QoS) policy for network-on-package (NoP) links. To the best of our knowledge, this is the first fine-grained mapping framework for multiple DNNs on a multi-chiplet architecture. We implemented the proposed fine-grained mapping framework using genetic algorithm and simulated annealing algorithm. Experimental results show that our work achieves 7.18%–61.09% latency reduction under vision, language, and mixed workloads when compared with the state-of-the-art related work.

Grid-based feature learning network plays a key role in recent point-cloud based 3D perception. However, high point sparsity and special operators lead to large memory footprint and long processing latency, posing great challenges to hardware acceleration. We propose FLNA, a novel feature learning accelerator with algorithm-architecture co-design. At algorithm level, the dataflow-decoupled graph is adopted to reduce 86% computation by exploiting inherent sparsity and concat redundancy. At hardware design level, we customize a pipelined architecture with block-wise processing, and introduce transposed SRAM strategy to save 82.1% access power. Implemented on a 40nm technology, FLNA achieves 13.4 − 43.3× speedup over RTX 2080Ti GPU. It rivals the state-of-the-art accelerator by 1.21× energy-efficiency improvement with 50.8% latency reduction.

Attention mechanism acceleration is becoming increasingly vital to achieve superior performance in deep learning tasks. Existing accelerators are commonly devised dedicatedly by exploring the potential sparsity in neural network (NN) models, which suffer from complicated training, tuning processes, and accuracy degradation. By systematically analyzing the inherent dataflow characteristics of attention mechanism, we propose the Co-Operative Systolic Array (COSA) to pursue higher computational efficiency for its acceleration. In COSA, two systolic arrays that can be dynamically configured into weight or output stationary modes are cascaded to enable efficient attention operation. Thus, hybrid dataflows are simultaneously supported in COSA. Furthermore, various fusion methodologies and an advanced softmax unit are designed. Experimental results show that the COSA-based accelerator can achieve 2.95-28.82× speedup compared with the existing designs, with up to 97.4% PE utilization rate and less memory access.

Point-cloud-based 3D perception has attracted great attention in various applications including robotics, autonomous driving and AR/VR. In particular, the 3D sparse convolution (SpConv) network has emerged as one of the most popular backbones due to its excellent performance. However, it poses severe challenges to real-time perception on general-purpose platforms, such as lengthy map search latency, high computation cost, and enormous memory footprint. In this paper, we propose SpOctA, a SpConv accelerator that enables high-speed and energy-efficient point cloud processing. SpOctA parallelizes the map search by utilizing algorithm-architecture co-optimization based on octree encoding, thereby achieving 8.8-21.2× search speedup. It also attenuates the heavy computational workload by exploiting inherent sparsity of each voxel, which eliminates computation redundancy and saves 44.4-79.1% processing latency. To optimize on-chip memory management, a SpConv-oriented non-uniform caching strategy is introduced to reduce external memory access energy by 57.6% on average. Implemented on a 40nm technology and extensively evaluated on representative benchmarks, SpOctA rivals the state-of-the-art SpConv accelerators by 1.1-6.9× speedup with 1.5-3.1× energy efficiency improvement,

Non-maximum suppression (NMS) is an important post-processing method to eliminate overlapping bounding boxes in object detection neural networks. Suffering from quadratic computational complexity and frequent memory access, NMS has become a bottleneck of detection latency. To deal with this problem, we propose out-of-order NMS (O3NMS), a hardware- software co-optimization approach to reduce latency as well as area overhead of NMS accelerator. In order to reduce startup latency, we devise the O3NMS algorithm that removes pre-sort operation. To efficiently support O3NMS algorithm, we design a specialized hardware accelerator. Our design has been implemented in both Xilinx FPGA and SIMC 40nm technology. Experiments demonstrate O3NMS accelerator achieves 2.51 x speedup as well as 37 % reduction in FPGA source utilization compared with the state-of-the-art (SOTA) NMS accelerator.

Conventional physics-based memristor device modeling methods highly rely on human expertise, which results in a long development period. To address the aforementioned challenges, we propose a new generalized memristor (GEM) device modeling framework based on the artificial neural network (ANN) technique, which has a minimum dependency on the underlying physics, resulting in a fast turn-around development time for customized memristor devices. GEM framework models the switching and conducting behaviors of the memristor devices separately, avoiding the signal-dependence issue in the prior time-series data modeling method. The result of the GEM framework is a compact model that supports general-purpose circuit simulators. Experimental results show that our compact model achieves a ratio of root-mean-square error to peak-to-peak (RMSE/PP) of 3.6% compared to the physics-based device model. Performance analysis of memristor-based logic and memristor crossbar circuits are conducted to demonstrate the effectiveness of our proposed GEM framework for the design and analysis of memristor-based circuits.

The design and verification of memristor crossbar circuits and systems demand computationally efficient models. The conventional device-level memristor model with a circuit simulator such as simulation program with integrated circuit emphasis (SPICE) to solve a memristor crossbar is time exhaustive. Hence, we propose a neural network-based memristor crossbar modeling method, XBarNet. By transforming memristor crossbar modeling to pixel-to-pixel regression, XBarNet avoids the iterative procedure in the conventional SPICE method, accelerating the runtime significantly. Meanwhile, XBarNet models the interconnect resistance and nonlinear$I-V$effect of memristor crossbars, which minimizes the simulation errors. We first propose a feature extraction method to bridge a memristor crossbar circuit and a neural network. Then, the network based on the convolutional autoencoder architecture is developed and the filter pruning technique is applied onto XBarNet to reduce the runtime computational cost. The experimental result shows our proposed XBarNet achieves over$78\times $runtime speed up and$1.7\times $memory reduction with only 0.28% relative error comparing to the SPICE simulator.

Coarse-Grained Reconfigurable Arrays (CGRAs) are promising to have low power consumption and high energy-efficiency characteristics as accelerators. Recent years, many research works focus on improving the programmability of the CGRAs by enabling the fast reconfiguration during execution. The performance of these CGRAs critically hinges upon the scheduling power of the compiler. One of the critical challenges is to reduce memory access conflicts using static compilation techniques. Memory accessing conflict brings the synchronization overhead which causes the pipelining stall and reduces CGRA performance. Existing compilers usually tackle this challenge by orchestrating the data placement of the on-chip global memory (OGM) in CGRA to let the parallel memory accesses avoid the bank conflict. However, we find bank conflict is not the only reason that causes the memory access conflicts. In some CGRAs, the bandwidth of the data network between OGM and processing element array (PEA) is also limited due to the low power design principle. The unbalanced network bandwidth loads is another reason that causes memory access conflicts. Furthermore, the redundant data access across iterations is one of the primary causes of memory access conflicts. Based on these observations, we provide a comprehensive and generalized compilation flow to reduce the memory conflicts. Firstly, we develop a loop transformation model to maximize the inter-iteration data reuse of the loops to reduce the memory accessing operations under the software pipelining scheme. Secondly, we enhance the bandwidth utilization of the network between OGM and PEA and avoid the bank conflict by providing a conflict-aware spatial mapping algorithm which can be easily integrated into existing CGRA modulo scheduling compilation flow. Experimental results show our method is capable of improving performance by an average of 44% comparing with state-of-the-art CGRA compiling flow.

When coarse-grained reconfigurable array (CGRA) architecture is shifting towards general-purpose, some complex control flows, such as nested loop, conditional branch and data dependence, may embarrass it and reduce the processing element (PE) array utilization by breaking the intact dataflow graph (DFG) into multiple regions with inconsistent control regions. This paper proposes subgraph decoupling and rescheduling, which decouples the inconsistent regions into control-independent subgraphs. Each subgraph can be rescheduled with zero-cost domino context switching and parallelized to fully utilize the PE resources. Then, we propose lightweight hardware changes based on general CGRA architecture to enable our design. The experiment results show that our proposal can improve the performance and energy efficiency by 1.35× and 1.18× over a static-mapped CGRA (Plasticine), and by 1.27× and 1.45× over an instruction-driven CGRA (TIA).

Resistive-RAM (ReRAM) based deep neural network (DNN) accelerator has shown great potential to address the memory wall problem for its processing-in-memory (PIM) capacity. However, ReRAM DNN accelerator still faces various challenges in its early architecture design phase due to the unpredictable variability and limitation of the ReRAM device. Software simulation helps but the simulation time is long with detailed ReRAM device model for large-scale DNNs. In this paper, we propose fast FPGA-based emulation for the DNN accelerator of ReRAM device. The emulation sets a primitive DNN accelerator architecture in FPGA and leverages FPGA hardware resources to provide massive parallelism for reducing emulation time. Meanwhile, it is co-designed with runtime software stacks to make the hardware emulation more flexible via instruction compilation and scheduling for different DNN needs. Our experiments show that the emulation can get over 194.7X speedup for large-scale DNNs against NeuroSim as a software simulator for ReRAM DNN accelerator. Therefore, the proposed emulation helps to build better ReRAM accelerators for large DNNs with much higher speed and flexibility.

The advancement of object detection algorithms makes them widely used in autonomous systems. However, due to high computational complexity of Convolutional Neural Networks(CNN), stringent latency requirement is hard to meet for real-time object detection. To address this problem, a low-latency accelerator architecture is proposed in this paper. A fine-grained column-based pipeline architecture with padding skip technique is implemented to reduce the start-up time of pipeline. In order to cut down the computational time of CNN, double signed-multiplication correcting circuit is introduced. In addition, pooling unit with share buffer is proposed to reduce storage cost for pooling layer. To demonstrate our new architecture, we implement the YOLOv2-tiny deep neural network (you-only-look-once) with input size 1280×384 on ZC706 development board, improving the latency by 2.125× to 2.34× compared to previous FPGA accelerator for YOLOv2-tiny.

In this letter, we propose an effective approach to learn a convolutional neural network (CNN) model with target enhancement and attenuated nonmaximum suppression (NMS) technique (TEANS) for object detection in optical remote sensing images. TEANS mainly consists of two steps. First, the target enhancement architecture, including target upsampling and reconvolution, is designed into a given deep ResNet-101 model for accurate object detection, especially for small ones. Second, the attenuated NMS technique is used for overcoming wrong eliminations of serried object proposals. For verifying the effectiveness of the TEANS method, evaluations are implemented on a publicly available 15-class optical remote sensing object detection data set. Experimental results show that TEANS can achieve 5.55%, 18.77%, 26.81%, 55.07%, 28.48%, 6.01%, and 5.51% improvements in mean Average Precision (mAP), respectively, compared with standard Faster R-CNN, R-FCN, YOLOv2, SSD, USB-BBR, YOLOv3, and MS-VANs frameworks.

The conventional mapping method between RRAM array and convolutional weights faces two key challenges: 1) nonoptimal energy efficiency and 2) RRAM's temporal variation. To address these challenges, we propose shift and duplicate kernel (SDK) convolutional weight mapping architecture. Each kernel is duplicated multiple times and rearranged on different bitlines in a shifted manner, enabling higher intralayer computational parallelism, and reducing the number of input data loading. Hence, this architecture reduces the computational latency and energy consumption in both forward and backward propagation phases. Furthermore, we have introduced a parallel-window size allocation algorithm and a kernel synchronization method. Our proposed parallel-window size allocation algorithm aims to balance the interlayer pipeline architecture, thus improving the overall energy efficiency and area efficiency. Our proposed kernel synchronization method uses an averaging method to suppress the effect of temporal variation during weight update, enhancing the system's robustness for training. From our experiment results, our proposed architecture achieves ~6.8× area efficiency and ~2.1× energy efficiency over the conventional interlayer pipeline architecture. Significant improvement in classification accuracy by 21.7% under a temporal variation of 1%-5% is achieved during on-chip training task on the Cifar-10 dataset.

This article presents an 8 × 8 lattice-reduction-aided (LRA) soft-output multiple-input multiple-output (MIMO) detector for Chinese enhanced ultrahigh throughput (EUHT) wireless local area network (LAN) standard. The preprocessing algorithm combining simplified-sorting Cholesky decomposition and low-complexity decoupled lattice reduction (LDLR) is proposed to reduce computational complexity and latency with parallelism improvement. In addition, K-best detection adopts a sorting-reduced strategy utilizing approximate ordered sequence. Compared with other published LRA K-best detection algorithms, simulation results show that our proposed algorithm has performance improvement. In addition, in order to save hardware resources, a folded K-best architecture and an optimized intermediate storage strategy are introduced. Furthermore, a fully pipelined VLSI architecture is designed in Semiconductor Manufacturing International Corporation (SMIC) 40-nm 1P9M technology to support the 8 × 8.64 -QAM MIMO-OFDM system. The detector can achieve 3.85-Gb/s data throughput at 641-MHz clock frequency with 0.71-μs latency. The proposed detector is competitive in terms of latency, throughput, and area efficiency to state-of-the-art works and can meet the data-rate requirement of the EUHT standard.

The Resistive-RAM (RRAM) based deep neural network (DNN) accelerators have shown great potential as they are good at solving matrix-vector multiplication (MVM). However, this computing paradigm does not benefit other NN operations like activation, which may be built upon various transcendental functions and require customized circuit as in current RRAM-based NN accelerators. In this paper, we propose the RRAM-CORDIC algorithm and crossbar design which enable various transcendental activation calculations on a RRAM crossbar just like MVM. By applying encoding and multi-iteration transformation, the RRAM-CORDIC can exploit higher MAC (multiply-and-accumulation) parallelism that is traditionally uneconomic in CMOS but now efficient in RRAM crossbar. In addition, it can work in a pipelined manner with high computing throughput. Experiment results show that the RRAM-CORDIC algorithm can sustain high accuracy on different transcendental functions, and deliver less than 0.5% NN accuracy loss on typical DNN inference. The elimination of CMOS circuit in turn can trade more computing resources for MVM in the same area budget that improves the performance up to 47% for different networks.

Coarse-grained reconfigurable array (CGRA) driven by dataflow execution is gaining reviving interest as an accelerator architecture of higher energy efficiency. However, with wider adoption in a variety of applications, it is facing complex data and control flows that cause multi-rate execution across different dataflow graphs in CGRAs which degrades the performance. In this paper, we propose a unified storage structure to decouple the multi-rate dataflow for decoupled execution. The structure leverages small distributed buffers with lightweight control. By chaining or aligning these buffers to form larger storage with different control schemes, it caters for different needs of dataflow decoupling when kernels are mapped onto CGRAs. Our experiment results show that by applying the proposed structure in conventional CGRAs, it can save dozens of PEs for dataflow computing, and improve the CGRA performance by an average of 2.53× for applications from different domains. Therefore, we provide a more efficient CGRA design when facing multi-rate dataflow execution.

The SRAM-based field-programmable gate array (FPGA) is extremely susceptible to single event upsets (SEUs) on configuration memory which can lead to soft error and malfunction of the circuit. Facing the ever-growing number of configuration bits in modern FPGAs, traditional scrubbing is getting harder to find errors in time, resulting in mismatching between the SEU sensitivity and scrubbing performance. This article proposes a hierarchical scrubbing technique that makes full use of the SEU sensitivity based on the adaptive mean time to detect (MTTD) for each frame. It distinguishes the configuration frames with multipriority and uses different scrubbing methods for different priorities. Also, a model has been built for solving the MTTD allocating problem and enabling an effective scrubbing when SEU occurrence. Moreover, the corresponding hardware architecture is supported and the fault injection-based evaluation on a Xilinx Kintex-7 FPGA is done. The result shows that it can improve mean upsets to failure from 1.56 × to 146.93 × , which is proportional to the mean time to failure (MTTF) improvement.

The metric-first-based multiple-input-multiple-output (MIMO) detection algorithm can achieve optimal performance with large stack size, which leads to huge memory consumption and extremely high-sorting complexity. This article presents the algorithm and architecture of a soft-input-soft-output metric-first MIMO detection. The proposed algorithm divides the global stack into multiple local stacks for each nonleaf of the tree. Furthermore, each level of the search tree is performed in parallel to improve the throughput and hardware efficiency. In the proposed algorithm, the hybrid enumeration strategy significantly reduces the computational complexity by avoiding the full enumeration and sorting. The simulation results show the novel algorithm can achieve good performance with lower complexity than other metric-first methods. The proposed detector has been designed for a 4 × 4 64-QAM MIMO system and implemented in SMIC 65-nm CMOS technology. The detector can operate at 333-MHz clock frequency and achieve a maximum throughput of 799.2 Mb/s at a 17.3-dB signal-to-noise ratio with area equivalent 242 kg and power consumption of 102.3 mW, and the hardware efficiency is 3.3 Mb/s/kg. Compared with other detectors based on the metric-first algorithm, this article has an obvious advantage in terms of throughput and hardware efficiency.

In recent years, action recognition has been widely explored and attains significant performance improvement. In this paper, we propose a real-time action recognition specified convolutional 3D (AR-C3D) neural network for human-computer interaction. The CNN structure is optimized to decrease the complexity. Furthermore, Winograd algorithm is adopted to accelerate computation. It achieves 89.9% accuracy in the application which refers to the robot classifies the video captured by itself and would either imitate human's action or give verbal feedback. The Artix-7 FPGA implementation result outperforms previous work in terms of resource utilization and no external storage is consumed. One video can be processed in 6.6ms, and the power consumption is only 2.7W.

The SRAM-based FPGA is extremely susceptible to Single Event Upsets (SEUs) on configuration memory which can lead to soft error and malfunction of the circuit. Facing the ever-growing number of configuration bits in modern FPGAs, conventional traversal scrubbing is getting harder to find errors in time, resulting in longer Mean Time to Detect (MTTD) before the error can be corrected by an effective scrubbing. This paper proposes a rapid scrubbing technique that enables an effective scrubbing as early as possible on a SEU occurrence. It applies position-aware Duplication with Compare (DWC) on the critical circuit that reduces the redundancy cost, and links the application circuit with configuration frames that enables the error locating in a greatly reduced number of configuration frames. Our fault injection-based evaluation on a Xilinx Kintex-7 FPGA shows that it can deliver an average of 45% MTTD and 16% Mean Time to Failure (MTTF) improvement with little cost when compared with the conventional traversal scrubbing provided by the Xilinx Soft Error Mitigation (SEM) module.

Object detection in aerial images is widely applied in many applications. In recent years, faster region convolutional neural network shows a great improvement on object detecting in natural images. Considering the size and distribution characteristic of object in remote sensing images, the region proposal network (RPN) should be changed before being adopted. In this letter, a scale adaptive proposal network (SAPNet) is proposed to improve the accuracy of multiobject detection in remote sensing images. The SAPNet consists of multilayer RPNs which are designed to generate multiscale object proposals, and a final detection subnetwork in which fusion feature layer has been applied for better multiobject detection. Comparative experimental results show that the proposed SAPNet significantly improves the accuracy of multiobject detection.

The coming era of big data revives the Processing-in-memory (PIM) architecture to relieve the memory wall problem that embarrasses the modern computing system. However, most existing PIM designs just put computing units closer to memory, rather than a complete integration of them due to their incompatibility in CMOS manufacturing. Fortunately, the emerging Resistive-RAM (ReRAM) offers new hope to this dilemma owing to its inherent memory and computing capability using the same device. In this article, we propose a ReRAM memory structure with efficient PIM capability of both logic and add operations. It first leverages non-linearity to suppress sneak current and thus sustains high memory density. Using a differential bit cell, it also enables efficient processing of arbitrary logic functions using the same memory cells with non-destructive operations. Then, a novel PIM adder is proposed, which customizes a sneak current path as the carry-chain for fast carry propagation and improves adder performance significantly. In the experiment, the proposed PIM demonstrates higher efficiency in both computing area and performance for logic and addition, which greatly increases the ReRAM PIM applicability for future computable architectures.

High Efficiency Video Coding (HEVC) standard has a superior video compression rate compared with previous H.264/AVC. At the same time, Ultra-high-definition (UHD) video applications are becoming a reality under the development of the display technology. In this paper, a hardware-friendly multi-layer HEVC motion estimation (ME) algorithm for UHD applications are proposed. To keep the computational regularity of the traditional full-search (FS) ME algorithm as well as reduce the computational complexity of ME in a large search range (SR), the basic layer of the proposed algorithm is to combine FS scheme in a core area with downsampling search scheme in a large peripheral area. Moreover, the finer layer of the algorithm employs a hexagon search scheme to perform further ME around the optimal match point generated by the basic layer. Integrating the proposed algorithm into the HM 15.0, experimental results show that our hardware-friendly algorithm can achieve 97.8% of computations reduction while only 0.77% of BD-rate loss on average. Consequently, the proposed algorithm is feasible for HEVC ME hardware design for UHD applications.

High Efficiency Video Coding (HEVC) is new video coding standard beyond H.264/AVC. In this paper, an area and throughput efficient 2-D IDCT/IDST VLSI architecture for HEVC standard is presented. Adopting proposed data flow scheduling and shared constant multiplication structure, the architecture supports variable block size IDCT from 4×4 to 32×32 pixels as well as 4×4 pels IDST. Using 65nm technology, the synthesis results show that the maximum work frequency is 500MHz and the architecture hardware cost is about 145.4K gate count. Compared with previous work, our design achieves more than 50% reduction in hardware cost and 66% improvement in throughput efficiency. Experimental results show that the proposed architecture is able to deal with real-time HEVC IDCT/IDST of 4K×2K (4096×2048)@30 fps video sequence at 412MHz in average. In consequence, it offers a cost-effective solution for the future UHDTV applications.

Multiple-input multiple-output (MIMO) technique can significantly increase data throughput without sacrificing additional bandwidth. However, data detection at the receiver and its VLSI implementation is challenge due to high computation complexity. This paper presents the VLSI architecture and implementation for a 4×4 64-QAM soft-output K-Best MIMO detector. A novel deeply pipelined architecture which makes use of all the full-length ZF-augmented discarded paths (DPs) is designed to reduce complexity and improve BER performance. Furthermore, to save area and latency, two improvement methods-abandoning DPs of bottom levels and performing ZF-augmentation at the last stage are proposed. The presented detector improves the BER performance by 2.3dB at BER=10-3compared to the conventional soft K-Best scheme when using the minimum mean squared error-sorted QR decomposition (MMSE-SQRD). It can achieve a peak throughput of 855 Mbps while consuming 223K gates, 301pJ/bit and 102 cycles for latency in SMIC 0.13μm CMOS process.

The sorted QR decomposition (SQRD) has become a critical prerequisite for non-linear detection algorithms such as K-best and sphere decoding for multiple-input multiple-output (MIMO) systems. However, due to the sorting and norm updating procedures, these systems are difficult to achieve high-throughput applications for lack of efficiency and parallelism. In this paper, an efficient VLSI implementation combining modified array architecture with sorting operations is proposed to increase parallel processing abilities. In addition, a novel sorting look ahead updating scheme is employed to advance sorting operations, which reduces the processing latency. Moreover, ℓ1-norm is adopted instead of the original ℓ2-norm to further simplify the hardware complexity. The proposed SQRD preprocessor implemented in SMIC 0.13μm CMOS technology achieves the throughput up to 25×106 SQRD per second which outperforms other works with equal functionality.

Layered structure is widely used in the design of Low-Density Parity-Check (LDPC) code decoders due to its fast convergence speed. However, correct checking process is difficult to implement in layered decoder, which results in unnecessary iterations. In this paper, an early termination strategy is presented for layered LDPC decoder to avoid redundant number of iterations. This approach makes use of the comparison between current log-likelyhood ratios (LLRs) and updated LLRs of all variable nodes to determine termination criteria of iterations. Furthermore, a non-uniform quantization scheme and an extrinsic messages memory optimization scheme are developed for memory savings. Based on these proposed methods, an LDPC decoder for the Chinese digital mobile TV applications is implemented using a SMIC 130nm CMOS process. The decoder consumes only 171 Kbits memory while achieving 267Mbps for code rate 1/2, and 401Mbps for code rate 3/4.

Reconfigurable computing arrays facilitate the flexibility with high performance for regular and computation-intensive algorithms in multimedia processing. However, the efficiency of the irregular and control-intensive algorithms becomes the performance bottleneck of reconfigurable multimedia systems. In this paper, we propose the design and VLSI implementation of a novel memory efficient macroblock prediction and boundary strength (Bs) calculation engine. The control-intensive algorithms, including intra mode prediction, motion vector prediction, and Bs calculation, are implemented with 4x4 block level pipeline to achieve real-time decoding for H.264/AVC high profile and Chinese AVS Jizhun profile. Compared with existing designs, our design achieves 60% registers reduction for neighboring block load and update. Implementation results indicate that the proposed architecture can support 1920×1088@30fps of H.264 and AVS decoding at 86 MHz.

A DVB receiving chip with PCI interface for PC is presented. The chip supports DVB protocols and integrates useful interfaces, including I2C, SmartCard and PCI. A card with this chip could change PC into digital TV terminal. The architecture of FPGA prototype system together with some main design issues is introduced. The experimental result shows that the chip could accomplish required functionalities.