Meeting growing demands for low latency and cost efficiency in production-grade large language model (LLM) serving systems requires integrating advanced optimization techniques. However, dynamic and unpredictable input-output lengths of LLM, compounded by these optimizations, exacerbate the issues of workload variability, making it difficult to maintain high efficiency on AI accelerators, especially DSAs with tile-based programming models. To address this challenge, we introduce XY-Serve, a versatile, Ascend NPU native, end-to-end production LLM-serving system. The core idea is an abstraction mechanism that smooths out the workload variability by decomposing computations into unified, hardware-friendly, fine-grained meta primitives. Then, kernels can efficiently execute without concerning the irregularity of workload. After this abstraction mechanism, for Attention, we propose a meta-kernel that computes the basic pattern of GEMM-Softmax-GEMM with architectural-aware tile sizes. For Linear, we introduce a virtual padding scheme that adapts to dynamic shape changes while using highly efficient GEMM primitives with assorted fixed tile sizes. XY-Serve sits harmoniously with vLLM. Experimental results show up to 95% end-to-end throughput improvement compared with current publicly available baselines on Ascend NPUs. We also set a new performance record for Linear (average 14.6% faster) and Attention (average 21.5% faster) kernels relative to existing libraries. Lastly, we demonstrate the generality of our technologies on GPU platform.

With the end of Moore’s Law, multi-chiplet systems have emerged as a promising solution featuring heterogeneity across chiplets, interconnects and packaging. Existing simulators lack support for such multi-level heterogeneity, making accurate architectural exploration difficult. We propose Hetero-ChipletSim (HCS), a simulation methodology that directly integrates heterogeneous chiplet models while incorporating die-to-die(D2D) interconnect and packaging effects, enabling fast and accurate evaluation of multi-chiplet systems. Sensitivity analysis provides insights into design trade-offs under heterogeneous integration.

Digital computing-in-memory (CIM) is a promising computing paradigm for the neural network (NN) acceleration. However, during the actual deployment process of digital CIM chips, we find that existing digital CIM designs face severe computing reliability issues, which are crucial for real product development but remain underexplored. Therefore, this work pioneers a systematic computing reliability analysis for digital CIM across off-memory and in-memory levels. We find that both the off-memory floating-point (FP) exponent alignment and the in-memory random cell bit-flip errors impair digital CIM's computing reliability, causing significant truncation and bit-flip accuracy loss. Critically, existing reliability solutions are incompatible with the unique multi-row accumulation structure of digital CIM, which either severely damage digital CIM's performance or result in prohibitive overhead. To address above challenges, we propose HR-DCIM: a highreliability FP digital CIM architecture featuring unified lowcost iterative error correction. Specifically, for the off-memory reliability, we propose an exponent-mantissa joint-alignment mechanism to repurpose inherent invalid bits of aligned mantissas as compensation bits to reduce alignment truncation loss, without damaging digital CIM's performance. Then, for the in-memory reliability, we propose a remainder aliasing-based unified multiply-accumulation (MAC) error correction mechanism to correct possible MAC errors caused by various cell error cases with low-cost iteration. Experimental results show that the proposed techniques enable digital CIM to maintain high performance and efficiency across various operating voltage conditions without significant accuracy loss.

With the increasing demand for high computational power in Large Language Models, wafer-scale chips have emerged as a solution, providing the necessary integration and computing capability to meet these needs. However, their ultralarge area and extreme heat dissipation introduce critical thermal management challenges under liquid-cooling environments. In addressing this issue, we identify two key opportunities and three major challenges: the behavior-thermal black box, the waferscale simulation bottleneck, and the runtime heat-schedule drift. To tackle these challenges, we propose ReThermal, a holistic scheduling framework that integrates three innovations. First, we introduce behavior-driven thermal modeling to capture workload-induced compute, communication, and heat coupling patterns at the system level. Second, we develop a DNNaccelerated wafer-scale thermal simulator that enables fast and accurate temperature prediction, significantly reducing simulation time. Third, we implement an adaptive thermal-aware scheduling strategy that coordinates compile-time and runtime decisions to dynamically optimize task placement. Evaluations show that ReThermal reduces peak temperature by up to 8.0° C and improves throughput by up to 39.23 %, providing a scalable and effective thermal control solution for future liquid-cooled wafer-scale systems.

As large language models (LLMs) continue to scale up, mixture-of-experts (MoE) has become a common technology in SOTA models. MoE models rely on expert parallelism (EP) to alleviate memory bottleneck, which introduces all-to-all communication to dispatch and combine tokens across devices. However, in widely-adopted GPU clusters, high-overhead crossnode communication makes all-to-all expensive, hindering the adoption of EP. Recently, wafer-scale chips (WSCs) have emerged as a platform integrating numerous devices on a wafer-sized interposer. WSCs provide a unified high-performance network connecting all devices, presenting a promising potential for hosting MoE models. Yet, their network is restricted to a mesh topology, causing imbalanced communication pressure and performance loss. Moreover, the lack of on-wafer disk leads to high-overhead expert migration on the critical path. To fully unleash this potential, we first propose Entwined Ring Mapping (ER-Mapping), which co-designs the mapping of attention and MoE layers to balance communication pressure and achieve better performance. We find that under ER-Mapping, the distribution of cold and hot links in the attention and MoE layers is complementary. Therefore, to hide the migration overhead, we propose the Non-invasive Balancer (NI-Balancer), which splits a complete expert migration into multiple steps and alternately utilizes the cold links of both layers. Evaluation shows ER-Mapping achieves communication reduction up to 62 %. NIBalancer further delivers 54 % and 22 % improvements in MoE computation and communication, respectively. Compared with the SOTA NVL72 supernode, the WSC platform delivers an average 39 % higher per-device MoE performance owing to its scalability to larger EP.

Training large language models (LLMs) imposes extreme demands on computation, memory capacity, and interconnect bandwidth, driven by their ever-increasing parameter scales and intensive data movement. Wafer-scale integration offers a promising solution by densely integrating multiple single-die chips with high-speed die-to-die (D2D) interconnects. However, the limited wafer area necessitates trade-offs among compute, memory, and communication resources. Fully harnessing the potential of wafer-scale integration while mitigating its architectural constraints is essential for maximizing LLM training performance. This imposes significant challenges for the co-optimization of architecture and training strategies. Unfortunately, existing approaches all fall short in addressing these challenges. To bridge the gap, we propose WATOS, a co-exploration framework for LLM training strategy and wafer-scale architecture. We first define a highly configurable hardware template designed to explore optimal architectural parameters for waferscale chips. Based on it, we capitalize on the high D2D bandwidth and fine-grained operation advantages inherent to wafer-scale chips to explore optimal parallelism and resource allocation strategies, effectively addressing the memory underutilization issues during LLM training. Compared to the state-of-the-art (SOTA) LLM training framework Megatron and Cerebras' weight streaming wafer training strategy, WATOS can achieve an average overall throughput improvement of$2.74 \times$and$1.53 \times$across various LLM models, respectively. In addition, we leverage WATOS to reveal intriguing insights about wafer-scale architecture design with the training of LLM workloads.

Large language models (LLMs) demand significant memory and computation resources. Wafer-scale chips (WSCs) provide high computation power and die-to-die (D2D) bandwidth but face a unique trade-off between on-chip memory and compute resources due to limited wafer area. Therefore, tensor parallelism strategies for wafer should leverage communication advantages while maintaining memory efficiency to maximize WSC performance. However, existing approaches fail to address these challenges. To address these challenges, we propose the tensor stream partition paradigm (TSPP), which reveals an opportunity to leverage WSCs' abundant communication bandwidth to alleviate stringent on-chip memory constraints. However, the 2D mesh topology of WSCs lacks long-distance and flexible interconnects, leading to three challenges: 1) severe tail latency, 2) prohibitive D2D traffic contention, and 3) intractable search time for optimal design. We present TEMP, a framework for LLM training on WSCs that leverages topology-aware tensor-stream partition, trafficconscious mapping, and dual-level wafer solving to overcome hardware constraints and parallelism challenges. These integrated approaches optimize memory efficiency and throughput, unlocking TSPP's full potential on WSCs. Evaluations show TEMP achieves$1.7 \times$average throughput improvement over state-of-the-art LLM training systems across various models.

Attention-based models have revolutionized AI, but the quadratic cost of self-attention incurs severe computational and memory overhead. Sparse attention methods alleviate this by skipping low-relevance token pairs. However, current approaches lack practicality due to the heavy expense of added sparsity predictor, which severely drops their hardware efficiency. This paper advances the state-of-the-art (SOTA) by proposing a bit-serial enable stage-fusion (BSF) mechanism, which eliminates the need for a separate predictor. However, it faces key challenges: 1) Inaccurate bit-sliced sparsity speculation leads to incorrect pruning; 2) Hardware under-utilization due to finegrained and imbalanced bit-level workloads. 3) Tiling difficulty caused by the row-wise dependency in sparsity pruning criteria. We propose PADE, a predictor-free algorithm-hardware codesign for dynamic sparse attention acceleration. PADE features three key innovations: 1) Bit-wise uncertainty interval-enabled guard filtering (BUI-GF) strategy to accurately identify trivial tokens during each bit round; 2) Bidirectional sparsity-based out-of-order execution (BS-OOE) to improve hardware utilization; 3) Interleaving-based sparsity-tiled attention (ISTA) to reduce both I/O and computational complexity. These techniques, combined with custom accelerator designs, enable practical sparsity acceleration without relying on an added sparsity predictor. Extensive experiments on 22 benchmarks show that PADE achieves$7.43 \times$speed up and$31.1 \times$higher energy efficiency than Nvidia H100 GPU. Compared to SOTA accelerators, PADE achieves$5.1 \times, 4.3 \times$and$3.4 \times$energy saving than Sanger, DOTA and SOFA.

The rapid expansion of large language models (LLMs) parameter scales imposes unprecedented demands on compute, memory, and communication resources for inference deployment. Wafer-scale chips, leveraging advanced packaging technologies, deliver high-density integration of compute and memory with high die-to-die (D2D) communication bandwidth, providing a compelling architectural approach to satisfy these resource requirements. However, its unprecedented chip area introduces significant architectural design complexities. Waferscale chips feature a multi-level architecture spanning the wafer, die, and core levels, involving numerous critical design parameters and trade-offs, which still lack systematic understanding and exploration. Moreover, this poses major challenges for LLM serving scheduling. Existing methods, largely adapted from GPUbased systems, fail to fully leverage the advantages of waferscale chips and mitigate their limitations, making it difficult to efficiently translate massive hardware resources into actual performance gains. To address these challenges, we introduce FACE, a coexploration framework for jointly optimizing multi-level architecture and serving scheduling. We first establish a flexible and extensible wafer-scale hardware template to systematically explore the optimal architecture and micro-architecture parameters. Leveraging the fine-grained control and high interconnect bandwidth of wafer-scale chips, FACE implements an LLM scheduling strategy that achieves fully overlapped prefill-decode execution and efficient KV cache management, maximizing hardware resource utilization to improve LLM service quality. Our evaluation demonstrates that FACE can achieve an average overall performance improvement of 3.68 × across various LLM models and datasets compared to the state-of-the-art (SOTA) LLM serving system on wafer-scale chips.

Large language models (LLMs) rely on self–attention for contextual understanding, demanding high-throughput inference and large–scale token parallelism (LTPP). Existing dynamic sparsity accelerators falter under LTPP scenarios due to stage-isolated optimizations. Revisiting the end-to-end sparsity acceleration flow, we identify an overlooked opportunity: crossstage coordination can substantially reduce redundant computation and memory access. We propose STAR, a cross-stage computetation and memory–efficient algorithm–hardware co-design tailored for Transformer inference under LTPP. STAR introduces a leading-zero-based sparsity prediction using log-domain add only operations to minimize prediction overhead. It further employs distributed sorting and a sorted updating FlashAttention mechanism, guided by a coordinated tiling strategy that enables fine-grained stage interaction for improved memory efficiency and latency. These optimizations are supported by a dedicated STAR accelerator architecture, achieving up to 9.2× speedup and 71.2× energy efficiency over A100, and surpassing SOTA accelerators by up to 16.1× energy and 27.1× area efficiency gains. Further, we deploy STAR onto a multi-core spatial architecture, optimizing dataflow and execution orchestration for ultra-long sequence processing. Architectural evaluation shows that, compared to the baseline design, Spatial-STAR achieves a 20.1× throughput improvement.

Many-core architecture is a promising architecture to accelerate increasingly larger neural networks (NNs). Most many-core architectures couple a standalone CPU core and a tensor core together as a compute node. However, the existing architectures suffer from inefficiency at the architecture, data flow, and control flow levels: The standalone scalar CPU core with deep out-of-order pipeline and low data parallelism per instruction incurs high hardware overhead and low throughput; Fixed proportions of CPU and tensor cores execute computations alternately in each cluster, leading to core under-utilization under diverse workloads; The MIMD parallelism strategy causes redundant instruction cache (I-Cache) accesses, which increases power consumption. To tackle the above limitations, we propose MRCIM, a many-core reconfigurable computing-in-memory (CIM) processor with reconfigurable cores featuring both CPU and tensor modes. 1) We design a reconfigurable CPU core by reusing the CIM-based tensor core’s inherent memory and computing logic to simplify the pipeline logic and improve the data parallelism of conventional CPU. 2) We propose interleaved workload execution (IWE) and adaptive workload mapping (AWM) scheduling strategies, which dynamically adjust the proportion of CPU core and tensor core in a cluster, making them work in parallel with high utilization. 3) We propose a hybrid MIMD/SIMD control flow to bypass unnecessary I-Cache accesses by instruction forwarding and sharing, thereby reducing power consumption. Experimental results show MRCIM achieves 166.48x~446.67x speedup and 96.76x~309.01x energy saving over Intel i9-13900k CPU, 12.62x~27.62x speedup and 5.49x~17.82x energy saving over NVIDIA RTX 4090 GPU. Compared with state-of-the-art NN processor architectures, our MRCIM achieves average 6.84x, 7.51x, and 3.66x speedup and average 4.57x, 3.03x, and 3.11x energy saving over Simba, LUT-ICC, and MAICC.

Transformer models have achieved remarkable success in various domains. Given concerns about user privacy, there is an urgent need for on-device fine-tuning of Transformer models at the edge. Transformer fine-tuning faces three key challenges: 1)$O(n^{3})$re-computations during BP/WG save only$O(n^{2})$storage, limiting batch size for fine-tuning speedup. 2)Weakly related tokens account for 87.9% of computations but contribute only 8.7% to accuracy. 3)89.1% of multiplications in matrix multiplications (MM) involve dual near-zero operands, leading to a$1.9\times $increase in logic toggling energy due to frequent exponent/mantissa variations near zero. This paper proposes a Transformer-based processor supporting energy-efficient fine-tuning with three key features to tackle the above challenges. 1)An exponent-stationary re-computing scheduler (ESRS) reduces 44.2% of the storage requirement for each batch. 2)An aggressive linear fitting unit (ALFU) saves 47.4% of the computations in each iteration. 3)A logarithmic domain processing element (LDPE) decreases 36.3% of energy for MM in fine-tuning. Fabricated with 22nm technology, the proposed processor has an area of 6.4 mm2. The proposed Transformer processor achieves a peak energy efficiency of 54.94 TFLOPS/W. It reduces fine-tuning energy by$4.27\times $and offers$3.57\times $speedup for GPT-2.

Graph Neural Networks (GNNs) have demonstrated exceptional performance in real-world applications, which often involve large-scale graphs with billions of vertices and numerous features per vertex. Large-scale workload requires multi-node systems to enhance computing power and memory capacity. However, accelerating large-scale GNNs on multi-node systems faces two key challenges. (1) Graph irregularity and high-dimensional features lead to excessive redundant inter-node communication. (2) Computational dependency in GNN results in waiting issues and underutilization of computing resources in accelerator nodes. To address the challenges, this work proposes PAMA, a pre-aggregation-based multi-node architecture for GNN acceleration. For challenge (1), we propose a pre-aggregation approach to avoid redundant feature transmissions, which is facilitated by a complementary communication scheme. For challenge (2), a batched staggered aggregation-transformation pipeline dataflow is proposed to alleviate the waiting issues. Additionally, a reconfigurable computing core that dynamically adapts to different workloads is designed to further improve computing resource utilization. The evaluation results show that PAMA achieves a$9.5-16 \times$speedup over the baseline multi-node system.

Rapid evolution in application algorithms, exemplified by advancements in artificial intelligence, wireless communication, and sciencific computing, necessitates a focus on developing energy-efficient, highly-flexible parallel computing architectures. This urgency is further amplified by the need for agile hardware development techniques to mitigate design complexity and reduce costs. Among emerging agile hardware development techniques, generative HDL stands out due to its straightforward grammatical structure and compatibility with hardware design thinking, yet it remains underutilized. In response, this paper introduces a novel four-layer agile developing flow, termed DIAG, innovatively leveraging unique Plugin-Service technology. The DIAG framework is applied to an extensible reconfigurable architecture generator, enabling the generation of diverse CGRA designs suitable for accelerating task computations across multiple application domains. Our comprehensive experiments on the CGRA generator design validate the efficiency of the DIAG flow and underscore generative-HDL's significant potential for complex, large-scale hardware development.

The growing demands of modern applications, such as AI, graph computing, and big data processing, are driving the increase in algorithmic scale and computational workload.As a result, Multi-PEA CGRA has been a popular choice because of its high computing power.However, such kinds of Architecture are confronted with control problem due to the large amount of PEA required management on Architecture.To address this challenge, this paper propose a built-in control design(Control Element, CE) for multi-PEA CGRA to improve computing efficiency.In this paper, this paper has compared the execution time, and power consumption with and without CE.Experiments demonstrate that our CE can reduce 97.3% execution time, in which the proportion of PEA work time is 76.2% and at least 79.7% power consumption.

Coarse-grained reconfigurable architecture (CGRA) has emerged as a promising solution for accelerating computationally intensive applications, particularly in the field of artificial intelligence. One of the primary challenges for CGRA compilers is generating effective mapping results for complex applications within a limited time-frame. This paper presents an enhanced pre-scheduling method that integrates Integer Linear Programming (ILP) and Graph Neural Networks (GNN), along with a corresponding two-stage mapping approach. This combination significantly reduces the search space and accelerates the solution process for mapping problems. Experimental results demonstrate performance improvements ranging from $29.4 \%$ to $406.7 \%$, along with compilation time reductions of up to $1106.8 \times$ compared to existing compilation techniques, as well as excellent scalability.

Digital computing-in-memory (CIM) is an emerging solution to break through the limitations of memory wall by integrating digital logic into SRAM, which is able to achieve high area and energy efficiency with no accuracy loss. Digital CIM’s SRAM cells are still prone to errors like conventional SRAM due to noise and variation, especially under low-voltage operation for high energy efficiency. The SRAM cell errors cause multiplyaccumulation (MAC) result errors, which may seriously damage the neural network inference accuracy. However, traditional SRAM’s error correcting code (ECC) that corrects errors in one read-out row is incompatible with digital CIM, which reads out multiple rows simultaneously for computation. Detecting and correcting computational MAC errors and SRAM cell errors (i.e., MAC-cell errors) in digital CIM remain largely unexplored.To address digital CIM’s unique MAC-cell error resilience needs, we propose ER-DCIM, an error-resilient digital CIM with run-time MAC-cell error correction to guarantee computation correctness. The proposed residue code-based MAC error correction mechanism is the first to correct additive errors in the MAC result in real time during DCIM computation. Then, we propose a progressive cell error correction mechanism to correct underlying cell error in a timely manner, avoiding performance loss due to stalling computation. Further, we design a mode switcher to repurpose redundant error-resilient logic reserved for low-voltage mode to improve performance in high-voltage mode. Experimental results show that the proposed techniques enable digital CIM to maintain high throughput and energy efficiency without accuracy loss in both low-voltage and high-voltage modes.

The rising cost and complexity of cutting-edge process nodes have impeded large monolithic System-on-Chip to follow Moore’s Law, forcing chip designers to embrace Multi-Chiplet architectures. Multi-chiplet designs achieve cost reduction while maintaining near-monolithic performance by disaggregating a large die into smaller chiplets and integrating them through advanced packaging. The payback of this Disaggregation-Integration paradigm critically depends on the efficacy of chiplet Partition and Placement framework. However, existing frameworks fail to harness the potential merits offered by Partition-Placement Co-Optimization. Serving as an input provider for placement, partition phase typically adjusts block-to-die assignments to guide subsequent placement. This sequential dependency implies an inherent Partition-Placement (P2P) Inconsistency problem: solutions optimal solely in partition or placement may finally cause an inferior solution. Hence, this paper proposes P2P-Chiplet, a Partition-Placement Co-Optimization framework for multi-chiplet designs. Firstly, an optimized ACG structure, named as HeteroACG, is introduced to aggregate topological partition and physical placement optimization spaces. Then, the sequential partition-placement flow is decomposed into interleaved fine-grained epochs and an alternating progressive optimization strategy is employed to preserve P2P Consistency and bring better co-optimized solutions. Finally, experimental results show that, compared with existing chiplet partition-placement frameworks, our proposed P2P Chiplet notably mitigates potential performance bottlenecks while effectively reducing costs within acceptable overhead.

Wafer-scale computing, with its high integration density and die-to-die bandwidth, offers a promising solution to the exponentially growing computational demands of large AI models. However, mapping large-scale AI training workloads onto wafer-scale architectures poses unique challenges compared to traditional GPU or AI accelerator clusters-namely, limited on-chip memory, non-uniform collective communication, and an exponentially large, sparsely populated search space. To address these challenges, we introduce RAM-Wafer, an innovative reinforcement learning (RL)-based automatic mapping framework designed specifically for wafer-scale computing. Built by extending the production-level AI compiler framework OpenXLA, our compiler-based end-to-end mapping solution incorporates an accurate and fast performance model that accounts for the distinctive constraints of wafer-scale systems. Moreover, our RL-based mapping method efficiently explores the vast search space to identify near-optimal mapping solutions in a fraction of the time required by conventional methods. Extensive experimental results demonstrate that RAM-Wafer outperforms manual expert baselines by 22 % and 9.5 % on Dojo and Waferscale GPU platforms, respectively, and achieves improvements of 15 % and 6.5 % compared to a genetic algorithm (GA). Additionally, RAM-Wafer reduces search time by$28 \times$, cutting mapping time from 2.3 hours to just 5 minutes.

The deployment of large language models (LLMs) imposes significant demands on computing, memory, and communication resources.Wafer-scale technology enables the high-density integration of multiple single-die chips with high-speed Die-to-Die (D2D) interconnections, presenting a promising solution to meet these demands arising from LLMs.However, given the limited wafer area, a trade-off needs to be made among computing, storage, and communication resources.Maximizing the benefits and minimizing the drawbacks of wafer-scale technology is crucial for enhancing the performance of LLM service systems, which poses challenges to both architecture and scheduling.Unfortunately, existing methods cannot effectively address these challenges.To bridge the gap, we propose WSC-LLM, an architecture and scheduling co-exploration framework.We first define a highly configurable general hardware template designed to explore optimal architectural parameters for wafer-scale chips.Based on it, we

As cluster scales for LLM training expand, waferscale chips, characterized by the high integration density and bandwidth, emerge as a promising approach to enhancing training performance.The role of Network on Wafer (NoW) is becoming increasingly significant, which puts an emphasis on two facts: physical and logical topology.However, existing networks fail to co-design both aspects.Additionally, physical topology typically focuses on optimizing communication or computation separately, neglecting opportunities to improve overall training performance.In this paper, we propose a physical design (PD) constraint-aware joint optimization strategy, developing mesh-switch physical topology and a dual-granularity logical topology.Mesh-switch leverages the high integration density of mesh and the efficient communication performance of fat tree, optimizing the allocation of on-chip

The rapid advancements in large language models (LLMs) have significantly increased hardware demands.Wafer-scale chips, which integrate numerous compute units on an entire wafer, offer a highdensity computing solution for data centers and can extend Moore's Law at system level.However, current wafer-scale data center architectures face inefficiencies, such as uncoordinated resource allocation and lack of co-optimization for system area, preventing optimal integration density and performance within given cost and physical constraints.We propose a co-exploration approach of computing and hardware architectures to bridge this gap.We first develop an optimized wafer-scale single-cabinet data center model, integrating configurable on-chip memory dies and employing a vertically stacked hardware architecture.Based on this model, we introduce Titan, an automated exploration framework for intra-chip and inter-chip architecture design and optimization.Based on the architecture features of wafer-scale systems with optimal integration density, Titan establishes parameter dependencies to co-design the computing and hardware architectures.To reduce the design cycle for wafer-scale systems, Titan introduces vertical area constraints and pre-checks physical limits by integrating a series of reliability prediction models.It also integrates hardware

Transformer-Based models have turned out to be very successful in many artificial intelligence (AI) tasks, outperforming traditional convolutional neural networks (CNNs), especially in the field of Natural Language Processing (NLP). Their success relies upon a self-attention mechanism which, when compared to CNNs, has a global rather than a local receptive domain. This article proposes an energy-efficient edge-device Transformer inference processor termed Smart Transformer Processing Element (STPE). Firstly, STPE sets up a Multi-Mode Indexing and Sparsity Scheme (MISS) for token association, and further reduces the computational load through in-situ computation; secondly, STPE exploits the Local Properties of Attention Mechanism (LPAM) to further reduce redundant and repetitive calculations in Transformer operations by means of a search band calculation and error correction mechanism; thirdly, STPE has designed a Quantization and Compression Parallel Method (QCPM) to improve the computing speed and hardware utilization under weak related (WR) token. Employing 28nm CMOS synthesis tools, the area of the proposed STPE processor is 7.33 mm2. Its peak energy efficiency is 84.15TOPS/W, which is 14.7 times higher than that of the H100 graphics processing unit (GPU) and 3.06 times higher than that of the most advanced Transformer processor.

Recently, Graph Neural Networks (GNNs) have shown great potential in terms of accuracy for problems that are well-described by graph representations, such as problems of path planning. However, implementing GNNs on mobile platforms is challenging as it requires a significant amount of computation and large memory. This article proposes a High-Performance Edge GNN Inference Processor termed GPE (GNN Processing Element). Firstly, GPE sets up Multi-Dimensional Indexing and Dynamic Pruning Schemes (MIDPS) for GNN networks, and achieves cross layer interconnection of multiple neighboring nodes via NOC (Network on Chip); secondly, GPE utilizes Graph Structure Adjacency Table Information (GSATI) of a GNN to further reduce redundant and repetitive calculations by means of repeated matching and difference transfer mechanisms; thirdly, GPE has a graph-based Multi Parallelism Simplification and Operation Method (MPSOM) to improve computing speed and hardware utilization under small data volumes. Using 28nm CMOS synthesis tools, the area of the proposed GPE processor is 5.37 square millimeters. Its peak energy efficiency is 21.5TOPS/W, which is 3.76 times higher than that of the H100 GPU (Graphics Processing Unit), while the energy consumption of GNN is 80.9% lower than the previous SOTA (State of Art) work.

LUT-based processing-in-memory (PIM) architectures enable generalpurpose in-situ computing by retrieving precomputed results.However, they suffer from limited computing precision, redundancy, and high latency of off-table access.To address these challenges, we present 3D-PATH, a novel PIM architecture that employs 3D hybrid bonding to integrate a DRAM-LUT, enhancing system capacity and reducing access latency.To further optimize efficiency, 3D-PATH introduces a hierarchical fast-LUT design that reduces storage redundancy and accelerates computation.Additionally, 3D-PATH extends computing precision by efficiently supporting floating-point operations via representation transformation and parallel interleaving banks.While hybrid bonding offers significant benefits, it induces heat dissipation challenges.To address this, we implement thermal-aware hardware that ensures the DRAM Die temperature maintains below the threshold of 85°C.Evaluations on arithmetic and AI workloads demonstrate that 3D-PATH achieves up to 12.68× higher throughput than GPUs and 2.27-7.54×over prior LUT-PIMs, while delivering a 12.24× improvement in floating-point energy efficiency over GPU and 2.13× over a 3D baseline.

Spatial architecture is a high-performance paradigm that employs control flow graphs and data flow graphs as computation model, and producer/consumer models as execution model. However, existing spatial architectures struggle with control flow handling challenges. Upon thoroughly characterizing their PE execution models, we observe that they lack autonomous, peer-to-peer, and temporally loosely-coupled control flow handling capability. This degrades its performance in intensive control programs. To tackle the existing control flow handling challenges, Marionette, a spatial architecture with an explicit-designed control flow plane, is proposed. We elaborately develop a full stack of Marionette architecture, from ISA, compiler, simulator to RTL. Marionette's flexible Control Flow Plane enables autonomous, peer-to-peer, and temporally loosely-coupled control flow management. Its Proactive PE Configuration ensures computation-overlapped and timely configuration to promote Branch Divergence handling capability. Besides, Marionette's Agile PE Assignment improves pipeline performance of imperfect loops. Compared to state-of-the-art spatial architectures, the experimental results demonstrate that Marionette outperforms Softbrain, TIA, REVEL, and RipTide by geomean 2.88$\mathbf{\times}$, 3.38$\mathbf{\times}$, 1.55$\mathbf{\times}$, and 2.66$\mathbf{\times}$in a variety of challenging intensive control programs.

Deep neural networks (DNNs) have brought about a transformative impact across various sectors. However, the proliferation of DNNs has led to a surge in computational intensity and data traffic, thereby imposing substantial demands on the power capacity and battery life of computing systems. Computing-in-memory (CIM) is considered a promising architecture to resolve or mitigate the memory wall challenge by integrating computational elements within memory arrays. Yet prior studies on CIM have seldom capitalized on sparsity in both activations and weights simultaneously. Furthermore, the exploitation of two-sided sparsity—sparsity in both activations and weights—presents new challenges, such as imbalanced workload and low hardware substrate utilization. To harness the full potential of two-sided sparsity for acceleration, we present Dyn-Bitpool, an accelerator that introduces innovations on two fronts: 1) a balanced workload scheme, “pool first and cross lane sharing”, which maximizes performance gains enabled by the bit-level sparsity in activations; and 2) a dynamic topology for CIM arrays to effectively address the low CIM macro utilization issue caused by the value-level sparsity in weights. These collective advancements yield an average speedup of 1.91x and 2.67x for Dyn-Bitpool on eight prevalent neural networks, outperforming two cutting-edge CIM-based accelerators.

Through chiplet reuse, chiplet-based system designs have emerged as a cost-effective solution for system-on-chips (SoCs), yet considerable silicon interposer costs often negate the benefits. Though general reusable interposers (GRIs) can lower the cost, they often compromise on performance and energy efficiency. In this article, a domain-specific reusable active interposer (active DSRI) approach is proposed for a better cost-efficiency tradeoff. Moreover, a chiplet platform based on an active DSRI designed for the intelligent radar/sonar (IRS) domain is introduced to facilitate rapid and customized SoC development. This platform offers flexible and energy-efficient interconnections tailored for IRS, platform infrastructure functions, and peripherals to simplify the chiplets. Furthermore, it integrates lightweight, composable standard 3-D interfaces across the chiplets and interposer, delivering up to 96-Gb/s bandwidth, 11.1-ns latency, and 0.62-pJ/bit energy efficiency, well controlling the cost and power penalties of SoC partition. Demonstrated with a customized hand gesture recognition sonar system (HGRSS) baseband SoC implemented on the proposed platform, it achieves similar performance to a monolithic SoC, with a recognition frame rate of 6286 frames/s, where overhead of the 3-D interface is only 6.86% in area and 4.84% in power. Our approach proves cost-effective, energy efficient, and customizable, moving system volume breakeven point forward by$3.22\sim 3.36$times, and reducing the cost by 58.5%~59.8%. This represents a pioneering demonstration of reusable chiplets in HGRSS, showcasing the potential of our approach for broader domains.

Memory partitioning is a widely used technique to reduce access conflicts on multi-bank memory in high-level synthesis. Previous memory partitioning methods mainly focus on a given access pattern extracted from stencil applications. Restricted by the pattern shape, these methods are prone to sub-optimal bank numbers or large overhead on address generation. In this work, we propose a pattern-morphing-based memory partitioning method, PMP, that only requires reduced hyperplane families to achieve the minimal bank number. To reduce the side effect of extra data padding, an integer linear programming problem is formulated for pattern morphing. Compared to the previous hyperplane-based memory partitioning, the experimental results show that our approach could achieve the optimal partition factor while saving 22% in LUTs, 21% in FlipFlops, 10% in DSPs, and 40% in memory overhead, on average.

Computing-in-memory (CIM), a promising computing paradigm, has demonstrated great energy-efficiency by integrating computing units into memory. However, previous research on CIM has rarely utilized sparsity in activation and weight concurrently. Moreover, new challenges arise when harnessing sparsity in both activation and weight (two-sided sparsity), such as unbalanced workload and low hardware substrate utilization.

This work proposes GSPO, an automatic unified framework that jointly applies graph substitution and parallelization for DNN inference. GSPO uses a joint optimization computation graph (JOCG) to represent graph substitution and parallelization at the operator level. Then, a novel cost model customized for joint optimization is used to evaluate the computation graph execution time quickly. With the graph partition and backtracking search algorithm, GSPO can find the optimal joint optimization solution within an acceptable search time. Compared to existing frameworks applying graph substitution or parallelization, GSPO can achieve up to 27.1% end-to-end performance improvement and reduce search time by up to 94.3%.

With the continuous advancement of artificial intelligence, neural networks exhibit an escalating parameter size, demanding increased computational power and excessive memory access. Low bit-width quantization emerges as a viable solution to address this challenge. However, conventional low bit-width uniform quantization suffers from a mismatch with the weight and activation data distribution in neural networks, resulting in accuracy degradation.

Demands for efficient computing under memory wall have led to computation-in-memory (CIM) accelerators that leverage memory structure to perform in-situ computing. The content addressable memory (CAM) processing is a CIM paradigm that accomplishes general purpose functions, via sequences of search and update operations on CAM. However, the conventional CAM-based CIM is customized for vector-vector operation only and requires long search-update iterations for computing.

Coarse-Grained Reconfigurable Arrays (CGRAs) are a promising architecture to cope with the challenges of increasing demand for high performance and high energy efficiency. However, the actual achieved performance of CGRA is highly dependent on the mappers. Traditional mappers using heuristics or combinatorial optimization can hardly learn from past experience, suffering from poor quality and portability. Recently, machine learning has been introduced to partial components in CGRA compilers, leaving other components to traditional heuristics, which is also prone to a sub-optimum, To this end, this paper proposes an end-to-end learning framework, E2EMap, for CGRA mapping that can cover the full mapping process. To reduce the complexity of the learning model, a reverse mapping problem is formulated, where various routing strategies can be thoroughly explored. To solve the problem, policy gradient reinforcement learning is introduced to learn from scratch. Experimental results demonstrate that E2EMap can achieve up to 2.23 x mapping quality across different CGRA settings while consuming even less compilation time as compared to state-of-the-art works.

Large language models (LLMs) have been showing surprising performance in processing language tasks, bringing a new prevalence to deploy LLM from cloud to edge. However, being a scaling auto-regressive Transformer with a huge parameter amount and generating output one by one, LLM introduces overwhelming memory footprints and computation during its inference, especially from its linear layers. For example, generating 32 output tokens with LLaMA-7B LLM requires 14GB of weight data and performs over 400 billion operations (98% from linear layers), which is far beyond the capability of consumer-level GPU and traditional accelerators. To solve these issues, we propose a memory-compute-efficient LLM accelerator, MECLA, with a parameter-efficient scaling sub-matrix partition method (SSMP). It decomposes large weight matrices into several tiny-scale source sub-matrices (SS) and derived sub-matrices (DS). Each DS can be obtained by scaling the corresponding SS with a scalar. For memory issues, SSMP avoids accessing the full weight matrix but only requires small SS and DS scaling scalars. For computation issues, the proposed MECLA processor fully exploits the intermediate data reuse of matrix multiplication via on-chip matrix regrouping, inner-product multiplication re-association, and outer-product partial sum reuse. Experiments on 20 benchmarks show that MECLA reduces memory access and computation by 83.6% and 72.2%. It achieves an energy efficiency of 7088GOPS/W. Compared to V100 GPU and state-of-the-art Transformer accelerator SpAtten and FACT, MECLA saves 113.14×, 12.99×, and 1.62× higher energy efficiency.

While extensive research has focused on optimizing performance and efficiency in vision-based AI accelerators, an unexplored phenomenon, Clustering Similarity Effect, presents a significant opportunity for further improvement. This effect reveals that clusters of neighboring data points exhibit similar values, enabling the potential to skip redundant computations.To fully capitalize on the potential of the Clustering Similarity Effect (CSE), this work integrates hybrid bonding DRAM technology. We conduct a comprehensive analysis of the associated design considerations and integration overhead. Leveraging these insights, we propose a novel CSE-aware architecture specifically tailored for hybrid bonding memory. This architecture facilitates similarity detection and adapts to the inherent data characteristics associated with CSE.Compared with state-of-the-art 2D/2.5D AI accelerators, the hybrid bonding baseline demonstrates an average energy efficiency improvement of $2.89 \times \sim 14.28 \times$ and an area efficiency improvement of $2.67 \times \sim 7.68 \times$. Incorporating the similarity optimizations further enhances energy efficiency and area efficiency improvement to $5.69 \times \sim 28.13 \times$ and $3.82 \times \sim 10.98 \times$, respectively.

Benefiting from the self-attention mechanism, Transformer models have attained impressive contextual comprehension capabilities for lengthy texts. The requirements of high-throughput inference arise as the large language models (LLMs) become increasingly prevalent, which calls for large-scale token parallel processing (LTPP). However, existing dynamic sparse accelerators struggle to effectively handle LTPP, as they solely focus on separate stage optimization, and with most efforts confined to computational enhancements. By re-examining the end-to-end flow of dynamic sparse acceleration, we pinpoint an ever-overlooked opportunity that the LTPP can exploit the intrinsic coordination among stages to avoid excessive memory access and redundant computation. Motivated by our observation, we present SOFA, a cross-stage compute-memory efficient algorithm-hardware co-design, which is tailored to tackle the challenges posed by LTPP of Transformer inference effectively. We first propose a novel leading zero computing paradigm, which predicts attention sparsity by using log-based add-only operations to avoid the significant overhead of prediction. Then, a distributed sorting and a sorted updating FlashAttention mechanism are proposed with cross-stage coordinated tiling principle, which enables fine-grained and lightweight coordination among stages, helping optimize memory access and latency. Further, we propose a SOFA accelerator to support these optimizations efficiently. Extensive experiments on 20 benchmarks show that SOFA achieves$9.5\times$speed up and$71.5\times$higher energy efficiency than Nvidia A100 GPU. Compared to eight SOTA accelerators, SOFA achieves an average$15.8\times$energy efficiency,$10.3\times$area efficiency and$9.3\times$speed up, respectively.

Coarse-Grained Reconfigurable Arrays (CGRAs) are a promising architecture for data-intensive applications. For parallel data accesses, uniform memory partitioning is usually introduced to CGRA for better pipelining performance. However, uniform memory partitioning not only suffers from a local minimum, but also introduces non-negligible overhead for banking function, which may greatly degrade the performance of CGRA. To this end, this paper introduces non-uniform memory partitioning and proposes a data-reuse-friendly CGRA (DARIC). With well elaborated configurable bank groups cooperated with register chains, elastic FIFOs can be achieved for non-uniform memory partitioning. Based on the resource graph of DARIC, a mapping algorithm supporting path sharing is proposed. Finally, the experimental results show that DARIC can achieve 2.35 × throughput and 2.59 × energy efficiency while having even less area and power overhead, as compared to the state-of-the-art.

Recently, the edge-device DNN training has become of high importance, while the computation and access energy consumption of are too large. This paper proposes a CPE (Compress Process Element) with three characteristics. Firstly, CPE has a method of Reordering and Reusing Data (RRD) by controlling the output to reorder data. Secondly, CPE owns a Multi-directional Redundant Skip (MRS) mechanism, which anticipates all zeros and duplicate fields in advance. Thirdly, CPE contains a scheme to transform The Calculation Format (TCF), which transforms the input into another form. Evaluated with 28nm CMOS process, using CPE achieves 2.02 × energy reduction and offer 1.73 × speed up outperforming state-of-the-art trainable processor GANPU.

Coarse-Grained Reconfigurable Architecture (CGRA), especially the one with multiple parallelized Processing Element Arrays (PEA), possesses flexible programmability and high parallel computational efficiency, which relies upon an efficient memory architecture to deliver the corresponding computing power. Multi-PEA oriented CGRA allows for mapping various applications and thus demands a flexible memory to adapt to the ever-changing workloads, whose parallel access also requires an efficient multi-port memory. However, the existing memory designs for CGRA are hard to satisfy those requirements since conventional rigid memories fail to provide the desired flexibility due to fixed structure, and traditional multi-port designs are impractical due to large overhead. Therefore, this paper proposes a hardware/software (HW/SW) hybrid reconfigurable multi-port memory architecture (RMP-MEM) with an instructive analysis for the multi-PEA oriented CGRA. RMP-MEM supports adaptive memory partition and programmer-defined access modes to adapt the different features of memory accesses. Also, RMP-MEM achieves an efficient multi-port implementation by a partially shared mechanism. Furthermore, the microarchitecture of RMP-MEM is optimized multi-directionally, resulting in a significant performance gain. The experimental results indicate that RMP-MEM reduces the parallel access latency by 81.1% and exhibits 28.3% energy efficiency improvement compared to prior designs.

Transformer model is becoming prevalent in various AI applications with its outstanding performance. However, the high cost of computation and memory footprint make its inference inefficient. We discover that among the three main computation modules in a Transformer model (QKV generation, attention computation, FFN), it is the QKV generation and FFN that contribute to the most power cost. While the attention computation, focused by most previous works, only has decent power share when dealing with extremely long inputs. Therefore, in this paper, we propose FACT, an efficient algorithm-hardware co-design optimizing all three modules of Transformer. We first propose an eager prediction algorithm which predicts the attention matrix before QKV generation. It further detects the unnecessary computation in QKV generation and assigns mixed-precision FFN with the predicted attention, which helps improve the throughput. Further, we propose FACT accelerator to efficiently support eager prediction with three designs. It avoids the large overhead of prediction by using log-based add-only operations for prediction. It eliminates the latency of prediction through an out-of-order scheduler that makes the eager prediction and computation work in full pipeline. It additionally avoids memory access conflict in the mixed-precision FFN with a novel diagonal storage pattern. Experiments on 22 benchmarks show that our FACT improves the throughput of the whole Transformer by 3.59× on the geomean average. It achieves an enviable 47.64× and 278.1× energy saving when computing attention, compared to previous attention-optimization-only SOTA works ELSA and Sanger. Further, FACT achieves an energy efficiency of 4388 GOPS/W performing the whole Transformer layer on average, which is 94.98× higher than Nvidia V100 GPU.

Spatial architecture is a high-performance architecture that uses control flow graphs and data flow graphs as the computational model and producer/consumer models as the execution models. However, existing spatial architectures suffer from control flow handling challenges. Upon categorizing their PE execution models, we find that they lack autonomous, peer-to-peer, and temporally loosely-coupled control flow handling capability. This leads to limited performance in intensive control programs.

Coarse-grained reconfigurable architectures (CGRAs) can provide both energy efficiency and performance for embedded systems, and thus they are increasingly deployed in the areas of aerospace, automotive engineering, and security where reliability is also a main criterion. However, the state-of-the-art fault-tolerant strategies for CGRAs apply either temporal or spatial scheme, including redundancy, periodic detection, workload balancing, and reconfiguration, failing to exploit the feature of dynamic and partial reconfiguration of CGRAs. Also, vulnerable judging circuits and inflexible mode shifting bottleneck the reliability design of fault-tolerant CGRAs. This article proposes a novel multimode fault-tolerant framework for CGRAs, which combines spatial-redundant data paths with temporal-redundant voters and thus reduces the vulnerable judging circuits while balancing the performance and reliability. This framework can also enable a changing reliability level at runtime via an online configuration transformation method based on precompiled patterns. Within the proposed framework, we systematically searched the design space spanning various combinations of the mainstream schemes with a Markov process model to compare the effectiveness and accordingly selected five points as available modes in our design after comprehensive consideration of fault tolerance and time overhead on CGRA. The framework is comprehensively evaluated on a cycle-accurate CGRA simulator, considering both permanent and transient faults. The experimental results show that the fault coverage rate of single transient faults or permanent faults has increased from 71.74% to 93.84%, which means the fault tolerance of the system has been increased by 31.03% compared with the state-of-the-art methods. There is also a great improvement in mean-time-to-failure (MTTF) and reconfiguration latency over baseline designs.

Coarse-grained reconfigurable architectures (CGRAs) are energy-efficient and processing-flexible platforms to perform parallel computation. CGRAs combine the advantages of flexibility of general-purpose processors (GPPs) and energy efficiency of application-specific integrated circuits (ASICs). During the compilation process, the CGRA compiler needs to convert the high-level language codes into a data flow graph, and then map it onto CGRA to generate instruction flow and configuration context. The instruction mapping schemes of the CGRA compiler have a great impact on the efficiency and energy consumption of CGRAs. Furthermore, the quality of the instruction mapping schemes of the CGRA compiler highly depends on how the compiler maps data dependencies using different CGRA resources. This article proposes an enhanced transfer-aware loop mapping method, TAEM 2.0, based on state-of-the-art TAEM algorithm. Based on a parallel iterative IBBMCX algorithm and comprehensive CGRA resources analysis strategy, this method efficiently processes the complex situations of utilizing all those heterogeneous resources on CGRA and significantly accelerates the compilation process. Experimental results show TAEM 2.0 can accelerate the compilation process by$4.40\times $while generating the same or better mapping results on CGRA, when compared to the state-of-art mapping technique.

Processing-in-memory (PIM) is a promising architecture for neural network (NN) acceleration. Most previous PIMs are based on analog computing, so their accuracy and memory cell array utilization are limited by analog deviation and ADC overhead. Digital PIM is an emerging type of PIM architecture that integrates digital logic in memory cells, which can make full utilization of the cell array without accuracy loss. However, digital PIM’s rigid crossbar architecture and full array activation raise new challenges in sparse NN acceleration. Conventional unstructured or structured sparsity cannot perform well on both the weight and input side of digital PIM. We take the opportunities from digital PIM’s bit-serial processing and in-memory customization, to tackle the above challenges by the co-designing sparse algorithm, multiplication dataflow, and PIM architecture. At the algorithm level, we propose double-broadcast hybrid-grained pruning to exploit weight sparsity with better accuracy and efficiency balance. At the dataflow level, we propose a bit-serial Booth in-SRAM multiplication dataflow for stable acceleration from the input side. At the architecture level, we design a sparse digital PIM (SDP) accelerator with customized SRAM-PIM macros to support the proposed techniques. SDP achieves$3.59\times $,$8.15\times $,$3.11\times $area efficiency, and$6.95\times $,$29.44\times $,$39.40\times $energy savings, over state-of-the-art sparse NN architectures SIGMA, SRE, and Bit Prudent.

Compute-in-memory (CIM) is a promising technique that reduces data movement in neural network (NN) acceleration. To achieve higher efficiency, some recent CIM accelerators exploit NN sparsity based on CIM’s small-grained operation unit (OU) feature. However, new problems arise in a practical multi-macro accelerator: The mismatch between workload parallelism and CIM macro organization causes spatial under-utilization; The multiple macros’ different computation time leads to temporal under-utilization. To solve the under-utilization problems, we propose a Sparsity-balanced Practical CIM accelerator (SPCIM), including optimized dataflow and hardware architecture design. For the CIM dataflow design, we first propose a reconfigurable cluster topology for CIM macro organization. Then we regularize weight sparsity in the OU-height pattern and reorder the weight matrix based on the sparsity ratio. The cluster topology can be reshaped to match workload parallelism for higher spatial utilization. Each CIM cluster’s workload is dynamically rebalanced for higher temporal utilization. Our hardware architecture supports the proposed dataflow with a spatial input dispatcher and a temporal workload allocator. Experimental results show that, compared with the baseline sparse CIM accelerator that suffers from spatial and temporal under-utilization, SPCIM achieves$2.94\times $speedup and$2.86\times $energy saving. The proposed sparsity-balanced dataflow and architecture are generic and scalable, which can be applied to other CIM accelerators. We strengthen two state-of-the-art CIM accelerators with the SPCIM techniques, improving their energy efficiency by$1.92\times $and$5.59\times $, respectively.

Nowadays, artificial intelligence (AI) technologies, especially deep neural networks (DNNs), play an vital role in solving many problems in both academia and industry. In order to simultaneously meet the demand of performance, energy efficiency and flexibility in DNN processing, various reconfigurable AI chips have been proposed in the past several years. They are based on FPGA or CGRA platforms and have domain-specific reconfigurability to customize the computing units and data paths for different DNN tasks without re-produce the chips. This paper surveys typical reconfigurable AI chips from three reconfiguration hierarchies: processing element level, processing element array level, and chip level. Each reconfiguration hierarchy covers a set of important optimization techniques for DNN computation which are frequently adopted in real life. This paper lists the reconfigurable AI chip works in chronological order, discusses the hardware development process for each optimization techniques, and analyzes the necessity of reconfigurability in AI tasks processing. The trends of each reconfiguration hierarchy and insights about the cooperation of techniques from different hierarchies are also proposed.

Skeleton-based human action cognition (HAR) has drawn increasing attention recently. As an emerging approach for skeleton-based HAR tasks, Spatial-Temporal Graph Convolution Network (STGCN) achieves remarkable performance by fully exploiting the skeleton topology information via graph convolution. Unfortunately, existing GCN accelerators lose efficiency when processing STGCN models due to two limitations. (1) At the dataflow level, the hardware parallelism of GCN accelerators cannot match the computation parallelism of STGCN models, leading to computing resource under-utilization. (2) At the computation level, GCN accelerators fail to exploit the inherent temporal redundancy in STGCN models. To overcome the limitations, this paper proposes STAR, an STGCN architecture for skeleton-based human action recognition. STAR is designed based on the characteristics of different computation phases in STGCN. For limitation (1), a spatial-temporal dimension consistent (STDC) dataflow is proposed to fully exploit the data reuse opportunities in all the different dimensions of STGCN. For limitation (2), we propose a node-wise exponent sharing scheme and a temporal-structured redundancy elimination mechanism, to exploit the inherent temporal redundancy specially introduced by STGCN. To further address the under-utilization induced by redundancy elimination, we design a dynamic data scheduler to manage the feature data storage and schedule the features and weights for valid computation in real time. STAR achieves$4.48\times $,$5.98\times $,$2.54\times $, and$103.88\times $energy savings on average over the HyGCN, AWB-GCN, TPU, and Jetson TX2 GPU.

Number Theoretical Transform (NTT) hardware accelerator becomes crucial building block in many cryptosystems like post-quantum cryptography. In this paper, we provide new insights into the construction of conflict-free memory mapping scheme (CFMMS) for multi-bank NTT architecture. Firstly, we offer parallel loop structure of arbitrary-radix NTT and propose two point-fetching modes. Afterwards, we transform the conflict-free mapping problem into conflict graph and develop novel heuristic to explore the design space of CFMMS, which turns out more efficient access scheme than classic works. To further verify the methodology, we design high-performance NTT/INTT kernels for Dilithium, whose area-time efficiency significantly outperforms state-of-the-art works on the similar FPGA platform.

Coarse-Grained Reconfigurable Architecture (CGRA) is a high-performance computing architecture. However, existing CGRA silicon utilization is low due to the lack of fine-grained parallelism inside Processing Element (PE) and general coarse-grained parallel approach on PE array. No fine-grained parallelism in PE not only leads to low silicon utilization of PE, but also makes the mapping loose and irregular. No generalized parallel method for the mapping cause low PE utilization on CGRA. Our goal is to design an execution model and a Mixed-granularity Parallel CGRA (MP-CGRA), which is capable to fine-grained parallelize operators excution in PEs and parallelize data transmission in channels, leading to a compact mapping. A coarse-grained general parallel method is proposed to vectorize the compact mapping. Evaluated with Machsuite, MP-CGRA achieves an improvement of 104.65% silicon utilization on PE array and a 91.40% performance per area improvement compared with baseline-CGRA.

This paper proposes the design of a computation-in-memory for stereo matching cost computation. The matching cost computation incurs large energy and latency overhead because of frequent memory access. To overcome previous design limitations, this work, named MC-CIM, performs matching cost computation without incurring memory access and introduces several key features. (1) Lightweight balanced computing unit is integrated within cell array to reduce memory access and improve system throughput. (2) Self-optimized circuit design enables to alter arithmetic operation for matching algorithm in various scenario. (3) Flexible data mapping method and reconfigurable digital peripheral explore maximum parallelism on different algorithm and bit-precision. The proposed design is implemented in 28nm technology and achieves average performance of 277 TOPs/W.

Monolithic SoCs can be decomposed into disparate chiplets that support integration with advanced pack-aging technologies. This concept is promising in reducing the manufacturing cost of large scale SoCs due to the higher yield rate and reusability of chiplets. The chiplets should be designed in a modular manner without holistic system knowledge so that they can be reused in different SoCs. However, the design modularity is a major challenge to the networks-on-chip (NoCs) of chiplets.New deadlocks may occur across both the chiplets and the interposer due to the integration, even if the NoC of each individually designed chiplet is deadlock free. However, conventional deadlock freedom approaches are unsuitable to handle such deadlocks because they require holistic knowledge and violate the modularity. Although there are several modular approaches that specifically target at integration-induced deadlocks, their routing is overly restricted and the injection control incurs additional latency. They also lack flexibility in dynamically changing topologies due to their complex software algorithm and the hard-wired components.In this paper, a key insight on the chiplet integration-induced deadlocks is gained, inspired by which a deadlock recovery framework (named UPP) is proposed. Specifically, it is verified that an integration-induced deadlock always involves a stalled upward packet moving from the interposer to the connected chiplet via the vertical link. Thus, UPP detects a deadlock by discovering the upward packet and recovers the system from deadlock by transmitting the upward packet to its destination. Hybrid flow control mechanisms are proposed to enable the upward packet to bypass the buffers and be transmitted via the normal router datapath. To guarantee the ejection of the upward packet after transmission, a lightweight protocol is proposed to reserve ejection queue entries of the network interface. Experimental results show that while adhering to design modularity, UPP provides an average runtime speedup of 3.1%∼10.3% with an area overhead of less than 4%.

To efficiently deploy state-of-the-art deep neural network (DNN) workloads with growing computational intensity and structural complexity, scalable DNN accelerators have been proposed in recent years, which are featured by multi-tensor engines and distributed on-chip buffers. Such spatial architectures have significantly expanded scheduling space in terms of parallelism and data reuse potentials, which demands for delicate workload orchestration. Previous works on DNN’s hardware mapping problem mainly focus on operator-level loop transformation for single array, which are insufficient for this new challenge. Resource partitioning methods for multi-engines such as CNN-partition and inter-layer pipelining have been studied. However, their intrinsic disadvantages of workload unbalance and pipeline delay still prevent scalable accelerators from releasing full potentials.In this paper, we propose atomic dataflow, a novel graph-level scheduling and mapping approach developed for DNN inference. Instead of partitioning hardware resources into fixed regions and binding each DNN layer to a certain region sequentially, atomic dataflow schedules the DNN computation graph in workload-specific granularity (atoms) to ensure PE-array utilization, supports flexible atom ordering to exploit parallelism, and orchestrates atom-engine mapping to optimize data reuse between spatially connected tensor engines. Firstly, we propose a simulated annealing based atomic tensor generation algorithm to minimize load unbalance. Secondly, we develop a dynamic programming based atomic DAG scheduling algorithm to systematically explore massive ordering potentials. Finally, to facilitate data locality and reduce expensive off-chip memory access, we present mapping and buffering strategies to efficiently utilize distributed on-chip storage. With an automated optimization framework being established, experimental results show significant improvements over baseline approaches in terms of performance, hardware utilization, and energy consumption.

Today, reconfigurable spatial architectures (RSAs) have sprung up as accelerators for compute- and data-intensive domains because they deliver energy and area efficiency close to ASICs and still retain sufficient programmability to keep the development cost low. The mapper, which is responsible for mapping algorithms onto RSAs, favors a systematic backtracking methodology because of high portability for evolving RSA designs. However, exponentially scaling compilation time has become the major obstacle. The key observation of this paper is that the key limiting factor to the systematic backtracking mappers is the waterfall mapping model which resolves all mapping variables and constraints at the same time using single-level intermediate representations (IRs).

Fully homomorphic encryption (FHE) is a novel encryption method that can perform operations on encrypted data. The performance of applications based on FHE is still low due to the high computational complexity of operations on the ciphertext. This article combines the characteristics of FHE and the redundant encoding method to achieve faster radix-2 arithmetic operations in BGV-like schemes. First, a ciphertext integer addition with multiplicative depth two, namely, redundant carry-free addition (RCFA), is proposed by applying the carry-free feature of the redundant encoding method. This addition is$7.207\times $faster and uses 15.0% of the occupied memory at 512 bits compared with the nonredundant method. After utilizing the single-instruction–multiple-data (SIMD) technique, RCFA with SIMD further improves the efficiency by$3903\times $at 512 bits compared with the nonredundant SIMD method. Its ciphertext size and occupied memory are only 34.59% and 1.6% those of the nonredundant SIMD method. Second, to achieve efficient ciphertext multiplications, redundant multiplication (RM) of dual SIMD data (RMDS) and RM of SIMD and non-SIMD data (RMSNS) methods are proposed; they obtain speedups of$65.6\times $and$631.0\times $, respectively, at 32 bits compared with the nonredundant implementations. Finally, SHA-256 is implemented to validate the efficiency of the proposed arithmetic operations. Compared with the nonredundant SIMD method, this article obtains a speedup of$4.34\times $.

Convolutional neural network (CNN) has demonstrated great success in pattern recognition scenarios at the cost of nearly billions of parameters and consequent convolution operations. Various dedicated hardware designs are proposed to accelerate the CNN computation in more energy-efficient manners. Especially, the bit-serial accelerator (BSA) is one of the most effective approaches on resource-limited platforms by eliminating zero-bit computations. However, the irregular distribution and varying number of effectual (nonzero) bits in weights significantly cause hardware underutilization, impeding further performance improvement of state-of-the-art BSAs. To address this issue, BitCluster, a hardware-friendly quantization method, is proposed to make each weight with the identical number of effectual bits for load-balanced computation. Considering distinct sensitivities to weight precision in different neural layers, layer-level BitCluster is proposed to design further for fine-grained weight quantization. It systematically determines the layerwise quantization configurations, which significantly improve the overall performance with$1.6\times $higher hardware utilization and$3.4\times $speedup on average than state-of-the-art BSAs, with$5\times $better energy efficiency on average.

Coarse-grained reconfigurable architecture (CGRA) is a promising programmable hardware with high power-efficiency and high performance. However, compiling and optimizing loops with irregular branches on CGRAs is a challenge to fulfill the performance potential. Existing predication techniques, such as partial predication (PP) and full predication (FP), conservatively implement software pipeline with a static initiation interval (II) obtained from the maximum graph, and thus only parts of the graph in each loop iteration will be actually executed, resulting in underexploited performance. To exploit more loop-level parallelism for irregular branches, this article proposes a novel dynamic-II pipeline (DIP) scheme, which realizes a pipeline with variable II by accommodating multiple iterations of short path in one static configuration. Since the DIP scheme is effective to only certain types of branches, this article designs a hybrid compilation framework integrating other complementary methods, which selects the appropriate method for source programs according to a proposed performance evaluation model. Experimental results show that: 1) the hybrid compilation framework can effectively extract branch features, correctly choose and implement corresponding branch processing methods within acceptable compile time and 2) as compared to PP and FP, DIP brings a significant total execution time (TET) reduction by 27.21% and 22.04% on average when the execution probability of a short branch is 50%.

Edge device deep neural network (DNN) training is practical to improve model adaptivity for unfamiliar datasets while avoiding privacy disclosure and huge communication cost. Nevertheless, apart from feed-forward (FF) as inference, DNN training still requires back-propagation (BP) and weight gradient (WG), introducing power-consuming floating-point computing requirements, hardware underutilization, and energy bottleneck from excessive memory access. This paper proposes a DNN training processor named PL-NPU to solve the above challenges with three innovations. First, a posit-based logarithm-domain processing element (PE) adapts to various training data requirements with a low bit-width format and reduces energy by transferring complicated arithmetics into simple logarithm domain operation. Second, a reconfigurable inter-intra-channel-reuse dataflow dynamically adjusts the PE mapping with a regrouping omega network to improve the operands reuse for higher hardware utilization. Third, a pointed-stake-shaped codec unit adaptively compresses small values to variable-length data format while compressing large values to fixed-length 8b posit format, reducing the memory access for breaking the training energy bottleneck. Simulated with 28nm CMOS technology, the proposed PL-NPU achieves a maximum frequency of 1040MHz with 343mW and 5.28mm$\mathbf {^{2}}$. The peak energy efficiency is 3.87TFLOPS/W for 0.6V at 60MHz. Compared with the state-of-the-art training processor, PL-NPU reaches$3.75\times $higher energy efficiency and offers$1.68\times $speedup when training ResNet18.

When deploying deep neural networks (DNNs), edge devices training is practical to improve model adaptivity for various user-specific scenarios while avoiding privacy disclosure. However, the training computation is intolerable for edge devices. It inspires sparse DNN training (SDT) into the limelight, which reduces training computation by dynamic weight pruning. Generally, SDT has two strategies based on the pruning granularity: the structured or the unstructured. Unfortunately, both of them suffer from limited training efficiency due to the gap between pruning granularity and hardware implementation. The former is hardware-friendly but has a low pruning ratio, indicating limited computation reduction. The latter has a high pruning ratio, but the unbalanced workload decreases utilization and irregular sparsity distribution causes considerable sparsity processing overhead. This paper proposes a software-hardware co- design to bridge the gap for improving the efficiency of SDT. On the algorithm side, a sub-structured pruning method, achieved with hybrid shape-wise and line-wise pruning, generates a high sparsity ratio and keeps the hardware-friendly property. On the hardware side, a sub-structured weight processing unit (SWPU) effectively handles the hybrid sparsity with three techniques. First, SWPU dynamically reorders the computation sequence with hamming-distance-based clustering, balancing the irregular workload. Second, SWPU performs runtime scheduling by exploiting the feature of sub-structured sparse convolution through a detect-before-load controller, which skips redundant memory access and sparsity processing. Third, SWPU performs sparse convolution by compressing operands with spatial disconnect log-based routing and recovers their location with bi-directional switching, avoiding the power-consumed routing logic. Synthesized with 28nm CMOS technology, SWPU can enable 0.56V-to-1.0V supply voltage with a maximum frequency of 675 MHz. It achieves a 50.1% higher pruning ratio than structured pruning and$1.53\times $higher energy efficiency than unstructured pruning. The peak energy efficiency of SWPU is 126.04TFLOPS/W, outperforming the state-of-the-art training processor by$1.67\times $. When training a ResNet-18 model, SWPU reduces$3.72\times $energy and offers$4.69\times $speedup than previous sparse training processors.

Approximate computing (AC) has been considered as a promising paradigm to improve the energy-efficiency of computing hardware for error-tolerant applications, with negligible quality degradation to the output. Dividers frequently limit the performance of a computing system; however, they have not received as much attention as multipliers and adders in AC. In this paper, an energy-efficient and high-performance approximate divider is proposed based on logarithmic conversion and piecewise constant approximation. In this design, the range for the conversion between binary and logarithmic numbers is first expanded from$\mathbf {[{0,1}]}$to$\mathbf {[-0.5,1]}$. A heuristic search algorithm is then devised to find the most accurate constant set to approximate the reciprocal of the divisor, by minimizing a statistical error. The hardware implementation is presented for both floating-point (FP) and integer dividers. With a high configurability, the proposed divider results in a mean relative error distance (MRED) from 2.78% to 0.046%, indicating a high accuracy among state-of-the-art approximate dividers. Compared to the half-precision FP divider, the proposed divider with a MRED of 0.74% can achieve nearly$\mathbf {90\times }$improvement in PDP. Moreover, compared to state-of-the-art approximate dividers, the proposed design is in the Pareto Frontier in terms of power delay product (PDP) and MRED. The three image processing application results demonstrate that the proposed divider can result in the highest peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) even with truncation.

Quantization is a prominent approach to compress model sizes of deep neural networks (DNNs), which clusters high-precision weights into a smaller set of quantization levels and represents high-precision weights by low-precision indexes. To achieve the same accuracy, nonuniform quantized DNNs (NUQ-DNNs) with unequal quantization intervals need lower index precision than uniform quantized DNNs (UQ-DNNs) with equal intervals, achieving smaller model sizes. Hence, deploying NUQ-DNNs on accelerators costs less on- and off-chip memory accesses than UQ-DNNs, which are more valuable for edge devices. However, accelerating NUQ-DNNs is nontrivial, since weight indexes cannot be directly used for computations. Previous NUQ-DNN accelerators adopt standard convolutions by decoding weight indexes into actual-weights multiplied with activations, causing abundant look-up overhead and redundant computations. In this work, we propose a weight-repetition-aware activation aggregating (WPAA) convolution approach to accelerate inference of variable-precision NUQ- and UQ-DNNs. By merging convolutions of multiple kernels, WPAA requires no look-up operation and removes redundant computations. Based on WPAA, we design a generic quantized DNN accelerator (GQNA). Furthermore, we propose a layer-adaptive kernel-reordering merging scheme to off-line adjust merging order of kernels for minimizing energy consumption of GQNA. Implemented under TSMC 28-nm technology, GQNA achieves 31.9 and 32.6 TOPS/W energy efficiency for 1-b UQ- and NUQ-VGG-16, respectively.

Deep neural network (DNN) has recently attracted tremendous attention in various fields. But the computing operation requirement and the memory bottleneck limit the energy efficiency of hardware implementations. Binary quantization is proposed to relieve the pressure of hardware design. And the Computing-In-Memory (CIM) is regarded as a promising method to resolve the memory wall challenge. However, the binary computing paradigm is mismatched with the CIM scheme, which incurs complex circuits and peripheral to realize binary operation in previous works. To overcome previous issues, this work presents Binary Representation Computation-In-Memory (BR-CIM) with several key features. (1) A lightweight computation unit is realized within the 6T SRAM array to accelerate binary computing and enlarge signal margin; (2) The reconfigurable computing scheme and mapping method support extendable bit precision to satisfy the accuracy requirement of various applications; (3) Simultaneous computing and weight loading is supported by column circuitry, which shortens the data loading latency; Several experiments are conducted to estimate algorithm accuracy, the computing latency, and power consumption. The energy efficiency reaches up to 1280 TOPs/W for binary representation. And the algorithm accuracy achieves 97.82%/76.4% on MNIST/CIFAR-100 dataset.

Coarse-Grained Reconfigurable Architectures (CGRAs) are attractive reconfigurable platforms with the advantages of high performance and power efficiency. In a CGRA based computing system, the computations are often mapped onto the CGRA with parallel memory accesses. To fully exploit the on-chip memory bandwidth, memory partitioning algorithms are widely used to reduce access conflicts. CGRAs have a fixed storage fabric and limited size memory due to the severe area constraints. Previous memory partitioning algorithms assumed that data could be completely transferred into the target memory. However, in practice, we often encounter situations where on-chip storage is insufficient to store the complete data. In order to perform the computation of these applications in the memory-limited CGRA, we first develop a memory partitioning strategy with continual placement, which can also avoid data preprocessing, and then divide the kernel into multiple subtasks that suit the size of the target memory. Experimental results show that, compared to the state-of-the-art method, our approach achieves a 43.2% reduction in data preparation time and an 18.5% improvement in overall performance. If the subtask generation scheme is adopted, our approach can achieve a 14.4% overall performance improvement while reducing memory requirements by 99.7%.

Multiply and accumulations(MAC) are fundamental operations for domain-specific accelerator with AI applications ranging from filtering to convolutional neural networks(CNN). This paper proposes an energy-efficient MAC design, supporting a wide range of bit-width, for both signed and unsigned operands. Firstly, based on the classic Booth algorithm, we propose the Booth algorithm to propose a multiply-add merged strategy. The design can not only support both signed and unsigned operations but also eliminate the delay, area and power overheads from the adder of traditional MAC units. Then a multiply-add merged design method for flexible bit-width adjustment is proposed using the fusion strategy. In addition, treating the addend as a partial product makes the operation easy to pipeline and balanced. The comprehensive improvement in delay, area and power can meet various requirements from different applications and hardware design. By using the proposed method, we have synthesized MAC units for several operation modes using a SMIC 40-nm library. Comparison with other MAC designs shows that the proposed design method can achieve up to 24.1% and 28.2% PDP and ADP improvement for bit-width fixed MAC designs, and 28.43% ~ 38.16% for bit-width adjustable ones. When pipelined, the design has decreased the latency by more than 13%. The improvement in power and area is up to 8.0% and 8.1% respectively.

In real noisy and reverberant environments, the performance of current single channel speech separation algorithms decreases significantly. Given this situation, this paper proposes a novel speech separation framework, called Graph convolution and Leading global Multi-scale separation network (GLMSnet). The graph convolution network (GCN) is introduced on high-level features for modeling global context and incorporating long-range information, and it can be arbitrarily inserted into the desired position. Furthermore, Global multi-scale convolution is proposed to aggregate different levels features and improve the audio quality of separation. The leading factor is applied to increase valid information of target speech. We evaluate our method on WHAMR! Database. The results show that our proposed method can obtain state-of-the-art speech separation effect in the presence of noise and reverberation. Compared with the most advanced model before, the performance is improved by 22.7%.

Nowadays, emerging integrated heterogeneous platforms play major roles to host autonomous systems. However, the security issue that comes with such heterogeneous architectures has not been thoroughly explored and imposes great threats and vulnerabilities to these systems. We set out to explore the security issues for the heterogeneous architectures and the corresponding mitigation mechanisms. We investigate the side-channel timing attack in a modern integrated CPU/GPU platform and propose a CPU/GPU co-encryption mechanism CoENC to mitigate the timing attack to provide a secure platform for autonomous systems. Evaluations demonstrate CoENC can effectively enhance the security 29~44 times compared to the baseline with an extra 14%~31% latency overhead.

To achieve high accuracy, DNN training usually consumes and generates myriads of data, which requires a large DRAM for efficient processing. The refresh power consumption in large DRAM has become a severe problem. Previous refresh energy saving methods have drawbacks on usability, flexibility or training supporting. We propose ADROIT, an adaptive dynamic refresh optimization framework for various DNNs and processing platforms. ADROIT dynamically adjusts the refresh rates for different types of data according to runtime loss feedback in DNN training. Data idle time, lifetime and size are taken into consideration to reduce the search space of refresh rate and remove most refresh operations. Experimental results show that ADROIT can reduce the refresh energy and total DRAM energy in DNN training by up to 98.9% and 24.7% respectively, while maintaining the accuracy. Moreover, ADROIT can automatically apply to different DNNs and hardware platforms without tedious manual configuration.

In-memory key-value store (KVS) has become crucial for many large-scale Internet services providers to build highperformance data centers. While most of the state-of-the-art KVS systems are optimized for read-intensive applications, a wide range of applications have been proven to be insert-intensive or scan-intensive, which scale poorly with the current implementations. With the availability of FPGA-based smart NICs in data centers, hardware-aided and hardware-based KVS systems are gaining their popularity. In this paper, we present HeteroKV, a scalable line-rate KVS on heterogeneous CPU-FPGA platforms, aiming to provide high throughput in read-, insert- and scan-intensive scenarios. To achieve this, HeteroKV leverages a heterogeneous data structure consisting of a b+ tree, whose leaf nodes are cache-aware partitioned hash tables. Experiments demonstrate HeteroKV's high performance in all scenarios. Specifically, a single node HeteroKV is able to achieve 430M, 315M and 15M key-value operations per second in read-, insert- and scan-intensive scenarios respectively, which are more than 1.5x, 1.4x and 5x higher than state-of-the-art implementations.

Bit-serial computation has been a prevailing convolution method to accelerate varying-precision DNNs by slicing a multi-bit data into multiple 1-bit data and transforming a multiplication into multiple additions, where additions of zero bits are ineffectual, while additions of non-zero bits are repetitive since multiple kernels are quite possible to possess non-zero bits at the same kernel positions. Previous bit-serial accelerators only remove ineffectual additions by skipping computation of zero bits, however, repetitive additions are unable to be eliminated since they compute convolution of each kernel independently. In this work, we propose fused kernel convolution algorithm to eliminate both ineffectual and repetitive additions in bit-serial computation by exploiting bit repetition and bit sparsity in weights, for both convolutional and fully-connected layers. It unifies convolutions of multiple kernels into convolution of one fused kernel by firstly grouping additions into different patterns and secondly reconstructing convolution results, minimizing addition count. Meantime, the memory accesses of activations and partial sums are decreased due to less convolution count. Then a fused kernel convolution based accelerator, FuseKNA, is designed with compact compute logic, which fully exploits value sparsity of activations and bit sparsity of weights. Benchmarked with a set of mainstream DNNs, FuseKNA improves performance by $4.47 \times$, $2.31 \times$ and $1.81 \times$, energy efficiency by $4.13 \times$, $3.06 \times$ and $2.53 \times$ over state-of-the-art Stripes, Pragmatic and Bit-Tactical.

Coarse-Grained Reconfigurable Architectures (CGRAs) are promising architectures with high energy efficiency and flexibility. The computation-intensive portions of an application (e.g. loops) are often executed on CGRAs for acceleration and modulo scheduling is commonly used for loop mapping. However, for imperfectly-nested loops, existing methods don't fully explore the structure of the loops before performing modulo scheduling, resulting in poor execution performance. To tackle this problem, we propose a polyhedral-based pipelining approach for mapping imperfectly-nested loops on CGRA. By efficiently exploring the transformation space for imperfectly-nested loops using the polyhedral model and taking total execution time as an optimization metric, our approach could improve the execution performance greatly. On a$4\times 4$mesh-connected CGRA, the experimental results show that our approach can reduce the total execution time of nested loop by 50.1 % on average, as compared to the state-of-the-art techniques. Moreover, the compilation time is moderate in practice.

Time Delay Neural Networks (TDNN)-based methods are widely used in dialect identification.However, in previous work with TDNN application, subtle variant is being neglected in different feature scales.To address this issue, we propose a new architecture, named dynamic multi-scale convolution, which consists of dynamic kernel convolution, local multi-scale learning, and global multi-scale pooling.Dynamic kernel convolution captures features between short-term and long-term context adaptively.Local multi-scale learning, which represents multiscale features at a granular level, is able to increase the range of receptive fields for convolution operation.Besides, global multi-scale pooling is applied to aggregate features from different bottleneck layers in order to collect information from multiple aspects.The proposed architecture significantly outperforms state-of-the-art system on the AP20-OLR-dialect-task of oriental language recognition (OLR) challenge 2020, with the best average cost performance (Cavg) of 0.067 and the best equal error rate (EER) of 6.52%.Compared with the known best results, our method achieves 9% of Cavg and 45% of EER relative improvement, respectively.Furthermore, the parameters of proposed model are 91% fewer than the best known model.

Near-Memory Processing (NMP) systems that integrate accelerators within DIMM (Dual-Inline Memory Module) buffer chips potentially provide high performance with relatively low design and manufacturing costs. However, an inevitable communication bottleneck arises when considering the main memory bus among peer DIMMs and the host CPU. This communication bottleneck roots in the bus-based nature and the limited point-to-point communication pattern of the main memory system. The aggregated memory bandwidth of DIMM- based NMP scales with the number of DIMMs. When the number of DIMMs in a channel scales up, the per-DIMM point-to-point communication bandwidth scales down, whereas the computation resources and local memory bandwidth per DIMM stay the same. For many important sparse data-intensive workloads like graph applications and sparse tensor algebra, we identify that communication among DIMMs and the host CPU easily dominates their processing procedure in previous DIMM-based NMP systems, which severely bottlenecks their performance.To tackle this challenge, we propose that inter-DIMM broadcast should be implemented and utilized in the main memory system of DIMM-based NMP. On the hardware side, the main memory bus naturally scales out with broadcast, where per- DIMM effective bandwidth of broadcast remains the same as the number of DIMMs grows. On the software side, many sparse applications can be implemented in a form such that broadcasts dominate their communication. Based on these ideas, we design ABC-DIMM, which Alleviates the Bottleneck of Communication in DIMM-based NMP, consisting of integral broadcast mechanisms and Broadcast-Process programming framework, with minimized modifications to commodity software-hardware stack. Our evaluation shows that ABC-DIMM offers an 8.33 × geo-mean speedup over a 16-core CPU baseline, and outperforms two NMP baselines by 2.59 × and 2.93 × on average.

Deadlock is a critical issue in faulty Networks-on-Chips (NoCs). Existing deadlock-free approaches on faulty NoCs suffer from low throughput and poor fairness when the network becomes oversaturated. This problem hinders their practical use as oversaturation scenarios are more frequent on faulty NoCs. To address this issue, a deflection-based deadlock recovery framework is proposed for higher oversaturation performance on faulty NoCs. First, we observe the low oversaturation performance of existing deadlock recovery approaches, and analyze the positive feedback loop that can amplify the negative impact of deadlocks and congestions, which necessitate handling both deadlocks and congestions in a deadlock recovery framework. Second, we propose a novel deadlock recovery framework, which includes an accurate, timely deadlock detection and a highly efficient deadlock recovery. Both the deadlock detection and recovery reduce the average packet traversal latency, thereby improving the average oversaturation throughput. Third, we propose a distributed implementation to make the entire network enter and exit the deflection mode, which is conducted by broadcasting special messages via a bufferless subnetwork. An average oversaturation throughput improvement of 1.1 ~ 8.1× over state-of-the-art approaches is achieved. In terms of fairness, the minimal oversaturation throughput is improved from near zero to half of the peak throughput.

Homomorphic encryption (HE) has broad application prospects in the cloud computing security field. Efficient homomorphic computations of primitive circuits are critical for the applications of HE. However, existing implementation methods over plaintext do not fit well with those over ciphertext. To address this issue, a concise evaluation model is proposed to compare different implementation methods, using weighted computational complexity (WCC). The number, depth, and distribution of homomorphic multiplications are considered together in the model for the first time. In addition, two primitive binary circuits on homomorphically encrypted data are optimized by using a unit called dot multiplication (DotMC). A novel comparison circuit based on DotMC is presented, and the number of homomorphic multiplications is reduced from O(n log n) to O(n) without increasing the multiplicative depth compared with the logarithm comparison, where n is the bit length of the operand. The WCC of comparison is reduced from O(n( log n)2) to O(n( log n)). The carry-lookahead adder is optimized by moving some DotMCs to levels with smaller weight, which reflects the effect of the distribution of homomorphic multiplications on performance. Finally, the proposed DotMC is accelerated with a single-instruction-multiple-data approach for even one operation; the number of homomorphic multiplication is reduced from O(n) to O( log n) compared with other comparison with the same strategy. Various circuits with the size from 4 to 2048 b are implemented with HElib to prove the optimization for comparison and addition, as well as the effectiveness of the proposed model.

The mnemonic descent method (MDM) algorithm is the first end-to-end recurrent convolutional system for high-accuracy face alignment. However, the heavy computational complexity and high memory access demands make it difficult to satisfy the requirements of real-time applications. To address this problem, an improved MDM (I-MDM) algorithm is proposed for efficient hardware implementation based on several hardware-oriented optimizations. First, a patch merging mechanism is introduced to dynamically cluster and eliminate redundant landmarks, which significantly reduces computational complexity with minimal accuracy loss. Second, a dedicated convolutional layer is inserted to halve the number of computations and memory access of the subsequent fully connected layer, yielding a 4.42% decrease in the failure rate. Third, a lightweight preprocessing method named dual regressors is proposed to reinitialize face images, which can greatly improve the overall accuracy. Moreover, compared with a similar method, the DR method can reduce computations and memory storage by nearly 99.9%. Overall and compared with the MDM algorithm, I-MDM not only reduces the number of computations by 23.5% but also decreases the failure rate by 17.9% on the 300 W test set. Based on the proposed I-MDM algorithm, an I-MDM-based hardwired accelerator is presented using the TSMC 65 nm CMOS process. First, compared with similar solutions, the gradient calculation operation is rearranged and loaded pixels are reused in the HoG feature extraction to eliminate all division operations and 25% off-chip memory access. Second, patch-independent central activations are used to enable patch-level pipelined operations, yielding a 2× acceleration in the overall process. This accelerator achieves 460 GOPS/W energy efficiency at 330 MHz, which is 38× higher than the most recent face alignment accelerator with the same process.

Coarse-grained reconfigurable architectures (CGRAs) are increasingly employed as domain-specific accelerators due to their efficiency and flexibility. A CGRA typically relies on compilers to perform task scheduling. The longstanding problem of static scheduling is that it suffers from insufficient parallelism in handling irregularities due to over-serialization and workload imbalance, which leads to severe resource underutilization and performance loss. To counteract the limitations of static scheduling in CGRAs, it is essential to exploit dynamic parallelism automatically and manage hardware resources adaptively. However, existing dynamic scheduling mechanisms, e.g., work stealing, often reschedule aggressively for instant performance but sacrifice efficiency, which is unfavorable to CGRAs that emphasize efficiency and fewer reconfigurations. This article proposes an elastic task scheduling scheme that enables lightweight dynamic scheduling in CGRAs. Tasks are rescheduled at runtime according to the classic tagged-token dataflow paradigm to enable dynamic task-level parallelism. Meanwhile, tasks are dynamically resized according to run-time throughputs via duplication, combination, and substitution operators for balanced multitask execution. We implement the elastic task scheduling scheme on a well-known reconfigurable architecture - triggered instruction architecture (TIA). Evaluation on the MachSuite benchmarks shows that the proposed scheme is effective in improving performance and energy efficiency. The average speedup is 2× over the baseline. Also, our design attains a 57 percent improvement in the area-normalized performance and a 49 percent better energy efficiency. Compared with a state-of-the-art dynamic scheduling method, our scheme achieves 1.6× speedup and 1.6× energy efficiency than work-stealing mechanism on the same substrate.

Coarse-grained reconfigurable architecture (CGRA) is an energy-efficient and processing-flexible parallel computing architecture. Efficiency of CGRA highly depends on how to map data dependencies using different CGRA resources. Previous works investigated different strategies for transferring data dependencies, using registers, processing elements (PEs) and memory. However, these works do not consider all those resources in CGRA and take a long time during compilation period. This paper proposes a Transfer-Aware Effective loop Mapping (TAEM) method for CGRA, which can efficiently utilize all those heterogeneous resources on CGRA and significantly accelerate the compilation time. Experimental results show that TAEM is able to reduce the compilation time by 11.1x over the state-of-the-art technique RAMP, while keeping the same or better performance of loop mapping results.

Deep convolutional neural networks (DCNNs), with extensive computation, require considerable external memory bandwidth and storage for intermediate feature maps. External memory accesses for feature maps become a significant energy bottleneck for DCNN accelerators. Many works have been done on quantizing feature maps into low precision to decrease the costs for computation and storage. There is an opportunity that the large amount of correlation among channels in feature maps can be exploited to further reduce external memory access. Towards this end, we propose a novel compression framework called Significance-aware Transform-based Codec (STC). In its compression process, significance-aware transform is introduced to obtain low-correlated feature maps in an orthogonal space, as the intrinsic representations of original feature maps. The transformed feature maps are quantized and encoded to compress external data transmission. For the next layer computation, the data will be reloaded with STC's reconstruction process. The STC framework can be supported with a small set of extensions to current DCNN accelerators. We implement STC extensions to the baseline TPU architecture for hardware evaluation. The strengthened TPU achieves average reduction of 2.57x in external memory access, 1.95x~2.78x improvement of system-level energy efficiency, with a negligible accuracy loss of only 0.5%.

Convolutional neural networks (CNNs) have emerged as the critical technology for deep learning with significantly growing computation and memory demands. Model compression has been widely acknowledged as an effective way to achieve acceleration on CNNs. However, most proposed architectures of FPGA are inefficient for compressed models that contain a large amount of zero operations. In this work, we propose a sparse CNNs inference accelerator on FPGA utilizing uniform sparsity introduced by pattern pruning to achieve high energy efficiency. Our architecture maintains the sparse weights in a compressed format to reduce the storage demands and displays a flexible kernel-stationary dataflow to enable the extensive data reusing. In addition, we design flexible computing arrays which can be dynamically reconfigured to balance workload with low overheads. Specially, the on-chip memory applies a novel data buffering structure with slightly rearranged sequences to address the challenge of access conflict. The experiments show that our accelerator can achieve $316 .4 {GOP/s }\sim 343 .5 {GOP/s }$ for VGG16 and ResNet-50.

Deep neural networks (DNNs) are becoming more and more popular in speech enhancement. Most of DNN-based speech enhancement approaches currently operate on magnitude spectra and ignore the phase mismatch between noisy and clean speech which greatly limits the speech enhancement performance. This paper presents a new approach to solve the phase mismatch problem by training traditional DNN adversarially with a time-domain discriminator. Instead of estimating a more accurate phase, the DNN is trained to be more adapted to noisy phase and able to minimize the influence brought by the phase mismatch. We also propose a new evaluation metric to judge the degree of adaptation to noisy phase. Experimental results show that adding of time-domain discriminator yields a more phase-adapted generator and significantly improves the speech enhancement performance.

Attention-based models have made tremendous progress on end-to-end automatic speech recognition(ASR) recently.However, the conventional transformer-based approaches usually generate the sequence results token by token from left to right, leaving the right-to-left contexts unexploited.In this work, we introduce a bidirectional speech transformer to utilize the different directional contexts simultaneously.Specifically, the outputs of our proposed transformer include a left-to-right target, and a right-to-left target.In inference stage, we use the introduced bidirectional beam search method, which can not only generate left-to-right candidates but also generate right-to-left candidates, and determine the best hypothesis by the score.To demonstrate our proposed speech transformer with a bidirectional decoder(STBD), we conduct extensive experiments on the AISHELL-1 dataset.The results of experiments show that STBD achieves a 3.6% relative CER reduction(CERR) over the unidirectional speech transformer baseline.Besides, the strongest model in this paper called STBD-Big can achieve 6.64% CER on the test set, without language model rescoring and any extra data augmentation strategies. 1

It is of vital importance to efficiently process large graphs for many data-intensive applications. As a result, a large collection of graph analytic frameworks has been proposed to improve the per-iteration performance on a single kind of computation resource. However, heavy coordination and synchronization overhead make it hard to scale out graph analytic frameworks from single platform to heterogeneous platforms. Furthermore, increasing the convergence rate, i.e. reducing the number of iterations, which is equally vital for improving the overall performance of iterative graph algorithms, receives much less attention. In this paper, we introduce the Block Coordinate Descent (BCD) view of graph algorithms and propose an asynchronous heterogeneous graph analytic framework, GraphABCD, using the BCD view. The BCD view offers key insights and trade-offs on achieving high convergence rate of iterative graph algorithms. GraphABCD features fast convergence under the algorithm design options suggested by BCD. GraphABCD offers algorithm and architectural supports for asynchronous execution, without undermining its fast convergence properties. With minimum synchronization overhead, GraphABCD is able to scale out to heterogeneous and distributed accelerators efficiently. To demonstrate GraphABCD, we prototype its whole system on Intel HARPv2 CPU-FPGA heterogeneous platform. Evaluations on HARPv2 show that GraphABCD achieves geo-mean speedups of 4.8x and 2.0x over GraphMat, a state-of-the-art framework in terms of convergence rate and execution time, respectively.

TCAM (Ternary Content-Addressable Memory) is the essential component for high-speed packet classification in modern hardware switches. However, due to its relatively slow update process, recent advances in Software-Defined Network (SDN) regard them as the bottleneck to the agile deployment of network services. Rule installation in commodity switches suffers from non-deterministic delays, ranging from a few milliseconds to nearly half a second. The crux of the problem is that TCAM prioritizes rules based on physical addresses. Corresponding entries have to be reallocated according to the priority of an incoming rule, such that the insertion delay grows linearly with the number of existing rules in a TCAM. In this paper, we present Constant-time Alteration Ternary CAM (CATCAM) that can accomplish both lookup queries and update requests for packet classification in a few nanoseconds. The key to fast update is to decouple rule priorities from physical addresses. We propose a matrix-based priority encoding scheme that records the priority relation between rules and can be implemented in 8T SRAM arrays with the emerging Processing In-Memory (PIM) technique. CATCAM also comes with a hierarchical architecture to scale out, its interval-based scheduling scheme guarantees deterministic update performance in all scenarios. CATCAM is developed under full-custom design in the 28 nm process. Evaluation across benchmark workloads shows that CATCAM provides at least three orders of magnitude speedup over state-of-the-art TCAM update algorithms and offers equivalent search capability to conventional TCAM while incurring 0.3% power and 20% area overhead.

Although convolutional neural network (CNN) models have greatly enhanced the development of many fields, the untenable number of parameters and computations in these models yield significant performance and energy challenges in hardware implementations. Transferred filter-based methods, as very promising techniques that have not yet been explored in the architecture domain, can substantially compress CNN models. However, their straightforward hardware implementation inherently incurs massive redundant computations, causing significant energy and time consumption. In this work, a highly efficient transferred filter-based engine (TFE) is developed to alleviate this deficiency, with CNN models compressed and accelerated. First, the filters of CNN models are flexibly transferred according to specific tasks to reduce the model size. Then, two hardware-friendly mechanisms are proposed in the TFE to remove duplicate computations caused by transferred filters, which can further accelerate transferred CNN models. The first mechanism exploits the shared weights hidden in each row of transferred filters and reuses the corresponding same partial sums, reducing at least 25% of repetitive computations in each row. The second mechanism can intelligently schedule and access the memory system to reuse the repetitive partial sums among different rows of the transferred filters with at least 25% of computations eliminated. Furthermore, an efficient hardware architecture is proposed in the TFE to fully reap the benefits of the two proposed mechanisms such that different types of networks are flexibly supported. To achieve high energy efficiency, the sub-array-based filter mapping method (SAFM) is proposed, where the process element (PE) subarray is used as the elementary computational unit to support various filters. Therein, input data can be efficiently broadcast in each PE sub-array and the load can be stripped from each PE and intensively alleviated, which can dramatically reduce the area and power consumption. Excluding MobileNet-like networks that adopt depth-wise convolution, most mainstream networks can be compressed and accelerated by the proposed TFE. Two state-of-the-art transferred filter-based methods, i.e., doubly CNN and symmetry CNN are implemented by exploiting the TFE. Compared with Eyeriss, average speedup improvements of 2.93× and 3.17× are achieved in the convolutional layers of various modern CNNs. The overall energy efficiency can be improved by 12.66× and 13.31× on average. Compared with other state-of-the-art related works, the TFE can maximally achieve a parameter reduction of 4.0×, a speedup of 2.72× and an energy efficiency improvement of 10.74× on VGGNet.

Large integer multiplication, or large degree polynomial multiplication, is the most time-consuming operation in fully homomorphic encryption (FHE). Low area and power consumption are difficult to maintain while achieving high performance for a large size multiplier. To address this issue, an area-efficient low-power architecture for multiplication, named NTTU, is proposed in this article. First, a combined number theoretic transform (NTT) method consisting of decimation-in-time (DIT) NTT for input in natural order and bit-reversed order is proposed to eliminate the steps of zero padding, scramble, and the first stage in NTT, thereby achieving a reduction of 7N/2 clock cycles compared with the single-type NTT method. Second, the NTT-uncoupled architecture is proposed to uncouple the multiplication components, decreasing the storage space for coefficients by 1/2 compared with state-of-the-art designs. Third, a parallel computing architecture based on a crossed memory access scheme is proposed, therein reducing the corresponding execution time by one-half compared with serial execution. Synthesized using 65 nm technology, the proposed architecture can multiply two 1024k/768k integers in 1.7 ms at 500 MHz at a cost of 13.66/7.67 million gates and 726.7/550.2 mW, and a 71.17 percent/ 30.37 percent area time product (ATP) reduction is achieved compared with the state-of-the-art ASIC designs.

Nowadays, driven by the needs of autonomous driving and edge intelligence, integrated CPU/GPU heterogeneous platform has gained significant attention from both academia and industry. As the representative series, NVIDIA Jetson family perform well in terms of computation capability, power consumption, and mobile size. Even so, the integrated heterogeneous platform only contains one limited physical memory, which is shared by the CPU and GPU cores and can be the performance bottleneck of the mobile/edge applications. On the other hand, with the unified memory (UM) model introduced in GPU programming, not only the memory allocation is significantly reduced, which mitigates the memory bottleneck of the integrated platforms but also the memory management and programming are simplified. However, as a programming legacy, the UM model still follows the conventional copy-then-execute model, initializing data on the CPU side after allocating memory. This legacy programming mode not only causes significant initialization latency but also slows the execution of the following kernel. In this article, we propose a framework to enable the latency-aware data initialization on the integrated heterogeneous platform. The framework not only includes three data initialization modes, the CPU initialization, GPU initialization, and hybrid initialization, but also utilizes an affinity estimation model to wisely decide the best initialization mode for an application such that the initialization latency performance of the application can be optimized. We evaluate our design on NVIDIA TX2 and AGX platforms. The results demonstrate that the framework can accurately select a data initialization mode for a given application to significantly reduce the initialization latency. We envision this latency-aware data initialization framework being adopted in a full-version of autonomous solution (e.g., Autoware) in the future.

The state-of-the-art convolutional neural network (CNN) structures present growing irregularity in the sense of layer connections, which derives from the innovative manual designs and the recently proposed neural architecture searching approaches. Such irregular structures improve recognition accuracy, but also bring challenges for hardware deployment, especially on CNN accelerators with regular architectures: 1) the complicated data dependency makes it nontrivial to decide the data reuse strategy between layers and 2) since the execution order of each network is not unique, the choice of layer scheduling, memory allocating, and loop tiling strategies greatly impact the hardware performance. These challenges cannot be solved by the existing CNN schedulers, which mainly focuses on the dataflow of a single layer. In this work, we propose a comprehensive framework to analyze and solve the mapping of an arbitrarily connected CNN network to specific hardware accelerators. We propose: 1) a dynamic programming and node-clustering-based DAG partitioning approach to efficiently exploit interlayer data reuse and 2) a subgraph scheduling and on-chip memory allocating strategy to find the optimal execution order. With the modeling of CNN accelerators, we also propose a loop tiling approach for fused layers. An automated framework is established to generate binary machine codes from original CNN models produced by mainstream deep learning frameworks, which can process large-scale CNNs with more than 1000 layers in only a few minutes. Experiments based on state-of-the-art accelerators (e.g., NVDLA) show that our techniques greatly reduce the external data transfer of interlayer dependencies and bring significant performance improvement over existing approaches.

Face detection and alignment are two fundamental tasks for facial applications and the corresponding accelerators have been designed to enable energy-efficient acceleration. However, these dedicated accelerators are always designed separately, thereby ignoring the inherent correlation between face detection and alignment and causing additional communication and area overhead. Based on this motivation, a multi-task cascaded convolutional networks (MTCNN) algorithm-based accelerator is presented in this work to support both face detection and alignment for multiple faces. First, multiply-accumulate (MAC) operations and memory access of the magnification process in the resize module are reduced by 22.8% and 24.8% on average, respectively, when compared with those of similar methods. Second, clustering non-maximum suppression (C-NMS) is proposed to significantly reduce the intersection over union computation and eliminate the hardware-inference sorting process in NMS, yielding a 16.0% speedup in the overall process. Third, an efficient pipeline architecture is proposed to implement a complexity- and memory-intensive proposal network of MTCNN in a more computationally efficient manner, with 38.3% less memory capacity than a similar solution. Meanwhile, only approximately half of the multipliers are needed to achieve the same throughput with high pipeline utilization. Fourth, considering the variable number of faces in each input, a batch schedule mechanism is proposed to improve the fully-connected layer hardware utilization by 16.7% on average in the batch process. Based on a simulation with the TSMC 28 nm CMOS process, this accelerator consumes only 10.9ms at 400 MHz to simultaneously process 5 faces. The power efficiency reaches 4.80 TOPS/W, which is$227.4\times $higher than that of the state-of-the-art solution.

Prevailing data-intensive applications, such as artificial intelligence and internet of things, demand considerable compute capability. Coarse-grained reconfigurable architectures (CGRAs) can meet this demand via providing abundant compute resources. However, compilation has become an essential problem because the increasing resources need to be orchestrated efficiently. Static compilation is insufficient due to conservative resource allocation and exponentially increasing time cost while state-of-the-art dynamic compilation still performs poorly in both generality and efficiency. This article proposes a dynamic compilation system for CGRAs through online pattern-based configuration transformation, which enables virtualization to improve resource utilization and flexibility. It utilizes statically-generated patterns to straightforwardly determine dynamic placement of registers and operations so that the transformation algorithm has a low complexity. Domain-specific features are extracted by a k-means clustering algorithm to help improve the quality of patterns. The experimental results show that statically compiled applications can be transformed onto arbitrary resources at runtime, reserving 73.5 (22.8-163.3 percent) of the original performance/resource on average, 9.1 (0-52.9 percent) better than the state-of-theart non-general methods.

The communication behaviors in NoCs of chip-multiprocessors exhibit great spatial and temporal variations, which introduce significant challenges for the reconfiguration in NoCs. Existing reconfigurable NoCs are still far from ideal reconfiguration scenarios, in which globally reconfigurable interconnects can be immediately reconfigured to provide bandwidths on demand for varying traffic flows. In this paper, we propose a hybrid NoC architecture that globally reconfigures the ring-based interconnect to adapt to the varying traffic flows with a high flexibility. The ring-based interconnect has the following advantages. First, it includes horizontal rings and vertical rings, which can be dynamically combined or split to provide low-latency channels for heavy traffic flows. Second, each combined ring connects a number of nodes, thereby improving both the utilization of each ring and the probability to reuse previous reconfigurable interconnects. Finally, the reconfiguration algorithm has a linear-time complexity and can be implemented using a low-overhead hardware design, making it possible to achieve a fast reconfiguration in NoCs. The experimental results show that compared to recent reconfigurable NoCs, the proposed NoC architecture can greatly improve the saturation throughput for synthetic traffic patterns, and reduce the packet latency over 40 percent for realistic benchmarks without incurring significant area and power overhead.

Multiple-input multiple-output (MIMO) and sparse code multiple access (SCMA) can be combined to achieve higher spectrum efficiency and more access for users, which also introduces more difficulties in signal detection. This paper explores low-complexity and low-latency iterative algorithms for soft symbol detection in an uplink MIMO-SCMA system over Rayleigh flat-fading channels. An expectation propagation framework (EPA) based on the extended factor graph is developed for MIMO-SCMA with multiantenna users. A new initialization method is proposed to accelerate convergence. Moreover, the SC-EPA with lower complexity is proposed by introducing QR decomposition and RE cluster-based decentralized factor node (FN) processing. Furthermore, new approaches for message passing between variable nodes (VNs) and FNs are proposed to improve the parallelism and reduce the complexity of the algorithm. The complexity of SC-EPA scales linearly with constellation size Ω (Ω <; M) and is independent of the receiving antenna Nr without any performance penalties. The robustness of the proposed algorithm in imperfect channels is evaluated, and the state evolution (SE) of the SC-EPA is derived. The link-level simulation results demonstrate that the EPA and SC-EPA receivers can achieve nearly the same performance as state of-the-art methods but with much lower complexity.

Despite the recent successes of deep neural networks, it remains challenging to achieve high precision keyword spotting task (KWS) on resource-constrained devices. In this study, we propose a novel context-aware and compact architecture for keyword spotting task. Based on residual connection and bottleneck structure, we design a compact and efficient network for KWS task. To leverage the long range dependencies and global context of the convolutional feature maps, the graph convolutional network is introduced to encode the nonlocal relations. By evaluated on the Google Speech Command Dataset, the proposed method achieves state-of-the-art performance and outperforms the prior works by a large margin with lower computational cost.

The performance of High-Level Synthesis (HLS) applications with irregular data structures is limited by its imperative programming paradigm like C/C++. In this paper, we show that constructing concurrent data structures with channels, a programming construct derived from CSP (communicating sequential processes) paradigm, is an effective approach to improve the performance of these applications. We evaluate concurrent data structure for FPGA by synthesizing a K-means clustering algorithm on the Intel HARP2 platform. A fully pipelined KMC processing element can be synthesized from OpenCL with the help of a SPSC (single-producer-single-consumer) queue and stack built from channels, achieving 15.2x speedup over a sequential baseline. The number of processing element can be scaled up by leveraging a MPMC (multiple-producer-multiple-consumer) stack with work distribution for dynamic load balance. Evaluation shows that an additional 3.5x speedup can be achieved when 4 processing element is instantiated. These results show that the concurrent data structure built with channels has great potential for improving the parallelism of HLS applications. We hope that our study will stimulate further research into the potential of channel-based HLS.

A Skyrmion Racetrack Memory (SRM) based Computing In-Memory Architecture (SRM-CIM) was proposed in this paper. Both data and computing operation can be achieved in SRM-CIM. SRM-CIM is used to support convolutional computing in Binary Convolutional Neural Network (BCNN). Experimental results show that SRM-CIM achieves 98.7% and 82% energy reduction when compared with RRAM and SOT-MRAM based counterparts.

This article consists of a collection of slides from the author's conference presentation.

Sparse Matrix-Vector Multiplication (SpMV) is the most important sparse linear algebra kernel in both scientific and engineering applications. Due to its irregular control flow and data access pattern, Von Neumann architectures like CPUs and GPUs cannot fully exploit the inherent parallelism of S pMV. Although FPGAs can efficiently accelerate SpMV in a dataflow manner, their performance is degraded in face of large matrices that exceed the capacity of on-chip memory because of excessive rescheduling of data. In this paper we propose ReDESK, a Reconfigurable Dataflow Engine for Sparse Kernels, for emerging tightly-coupled CPU-FPGA heterogeneous platforms. To fully exploit the heterogeneity, we design a novel representation of sparse matrix that is tailored for data prefetching on CPU-side and streaming processing on FPGA-side. In this way ReDESK can fully utilize the memory bandwidth regardless of the scale of SpMV problem. We evaluate ReDESK on Intel HARP-2 platform with a set of matrices from the University of Florida sparse matrix collection. The result demonstrates an average bandwidth utilization of 0.094 GFLOP/GB, which is 1.6-4.3x more efficient than previous SpMV on FPGAs.

Physical Unclonable Function (PUF) is an emerging security primitive for cryptography applications. However, achieving a very high reliability against the environmental variations remains a main challenge in PUF design and a key barrier for its commercialization. This paper presents a new PUF design based on the charging of a symmetric MOS capacitor pair by constant current with cross-coupled positive feedback inverters. The proposed weak PUF features high raw response reliability against variations in power supply and temperature without power-up reset noise and other issues due to the power-down and up of an array of cells. Extensive Monte-Carlo simulations have been performed using a standard 110nm CMOS process technology. The simulated results show an almost ideal uniqueness of 50.03% and superior reliability of 97.70% over a temperature range from 0 °C to 80 °C, and 96.20% with the supply voltage varies from 1.2 V to 1.8 V. The response bit can be generated at a rate of 27.78 Mbps with an average power consumption of 20.86 μW at 1.5V, and the energy consumption is only 750 fJ/bit.

Transactional Memory (TM) has been considered as a promising alternative to existing synchronization operations, which are often the largest stumbling block to unleashing parallelism of applications. Efficient implementations of TM, however, are challenging due to the tension between lowering performance overhead and avoiding unnecessary aborts.

The integral histogram image plays an important role in accelerating the feature computation in vision algorithms. However, the computational process of the integral histogram, called integral computation, has high computational complexity and numerous memory access operations, which limit its wide application. This brief proposes an in-memory computational architecture based on Spin Transfer Torque Magnetic Random Access Memory (STT-MRAM) to solve these problems. The architecture can work in two different modes depending on the requirements: the integral computation mode and the memory mode. The architecture can figure out the integral histogram when in the integral computation mode, and just store the data directly when in the memory mode. Utilizing the non-volatile, high density and low power characteristics of STT-MRAM, we integrate the computational units into the memory array to achieve parallel computation. Reduced number of data transmission between storage units and computation units contributes to cut down the latency and energy consumption. The evaluation results show that, comparing with the state-of-the-art work, our architecture provides$1.1\times \sim 9\times$performance improvements and reduces 87.4$\sim$97.3 percent energy consumption for$64\times 64\sim 512\times 512$size images, just with a 8 percent area overhead.

The low-entropy masking scheme (LEMS) is a costsecurity tradeoff solution that ensures a certain level of security with much lower overheads than a full-entropy masking scheme (FEMS). However, most existing LEMSs are based on a look-up-table (LUT) and limited to the first-order, which is vulnerable to classical higher-order correlation power analysis (CPA) attack and other special types of attack (e.g., collision attack). This paper proposes a new type of LEMS for a block cipher in which the S-box consists of power functions and an affine function. First, a low masking-complexity algorithm for evaluating S-boxes is developed by fully utilizing the property of a hybrid addition-chain (AC) named LUT-AC. Next, an LEMS for block ciphers is proposed. This LEMS provides two different masking modes to realize various cost-security tradeoff schemes. Due to the “masked invariant property” of the LUTAC, the masking complexity of the proposed LEMS is equal to O(d), whereas under FEMS it is equal to O(d2). Compared with existing LEMSs, the proposed LEMS has following advantages: higher security in terms of the masking entropy; resistance against collision attacks; and scalability to higher-order schemes. Per the proposed algorithm, an architecture without any nonlinear multiplication for evaluating AES is developed by replacing the LUT with seven scalar multiplications. The different LEMSs based on this architecture are developed. Their area overheads are evaluated by implementing different schemes in 65 nm CMOS process. The security of the first-order LEMS with rotation mode is verified by performing CPA on the SAKURA-G FPGA board. From the experimental success rates, it shows that the proposed first-order LEMS can resist CPA without revealing the correct subkey for up to 100 000 power traces, whereas the unprotected scheme is broken at 1100 traces.

Coarse-grained reconfigurable architectures (CGRAs) have drawn increasing attention due to their flexibility and energy efficiency. Data flow graphs (DFGs) are often mapped onto CGRAs for acceleration. The problem of DFG mapping is challenging due to the diverse structures from DFGs and constrained hardware from CGRAs. Consequently, it is difficult to find a valid and high quality solution simultaneously. Inspired from the great progress in deep reinforcement learning (RL) for AI problems, we consider building methods that learn to map DFGs onto spatially programmed CGRAs directly from experiences. We propose RLMap, a solution that formulates DFG mapping on CGRA as an agent in RL, which unifies placement, routing and processing element insertion by interchange actions of the agent. Experimental results show that RLMap performs comparably to state-of-the-art heuristics in mapping quality, adapts to different architecture, and converges quickly.

Considering that the binary-feature-based approximate nearest neighbor (ANN) search technique has not been fully exploited to date, a multisegment binary feature-based hierarchical clustering tree model is proposed to achieve fast binary feature matching (FM). In addition, the multisegment vocabulary forest, is developed for the ease of hardware-oriented implementation. During the ANN searching process, the corresponding leaf nodes of each segment of the query feature are returned simultaneously to improve processing speed and accuracy. Furthermore, a hierarchical decomposition based on the term frequency-inverse document frequency is used to reduce the run-time search space and total memory footprint for object database storage. Finally, a fine-grained feature-level fully pipelined object recognition accelerator is implemented based on a dedicated design between FM and object scoring. The performance of the proposed object recognition accelerator is evaluated based on TSMC 65 nm CMOS technology. The accelerator achieves 22 M-vec/s and 6.8 × 108vec/J in throughput and energy efficiency for full-HD resolution, respectively; these results represent a 10.6× and 9× improvement, respectively, relative to current state-of-the-art solutions. The average power consumption is 32.6 mW when operating at 200 MHz.

Due to technology scaling, lifetime reliability is becoming one of the major design constraints in the performance optimization of future many-core systems. Given a lifetime reliability constraint, the existing lifetime-constrained runtime mapping schemes often lead to low throughput because of the requirement to map all applications to compact regions. In this paper, we propose a runtime application mapping scheme that exploits a borrowing strategy to improve the throughput of many-core systems given a lifetime constraint. First, we propose using different strategies for mapping communication-intensive applications and computation-intensive applications. The lifetime reliability constraint can be relaxed in the local time scale when the communication requirement is high. The throughput is improved because the communication distance of communication-intensive applications is optimized while the waiting time of computation-intensive application is reduced. Then, we propose a method to effectively classify applications depending on the communication-to-computation ratio. A dynamic threshold is determined according to the current locations of available cores. Finally, we propose an improved neighborhood allocation scheme to reduce the communication cost in the task mapping. The experimental results show that compared to the state-of-the-art lifetime-constrained mapping, the proposed mapping scheme improves the throughput of many-core systems by 26% on average for synthetic task graphs and by 20% on average for realistic task graphs while the lifetime reliability is maintained within a constraint.

Deep learning is the amazing technology which has promoted the development of artificial intelligence and achieved many amazing successes in intelligent fields. Convolution-based layers (CLs), fully connected layers (FLs) and recurrent layers (RLs) are three types of layers in classic neural networks. Most intelligent tasks are implemented by the hybrid neural networks (hybrid-NNs), which are commonly composed of different layer-blocks (LBs) of CLs, FLs, and RLs. Because the CLs require the most computation in hybrid-NNs, many field-programmable gate array (FPGA)-based accelerators focus on CLs acceleration and have demonstrated great performance. However, the CLs accelerators lead to an underutilization of FPGA resources in the acceleration of the whole hybrid-NN. To fully exploit the logic resources and the memory bandwidth in the acceleration of CLs/FLs/RLs, we propose an FPGA resource efficient mapping mechanism for hybrid-NNs. The mechanism first improves the utilization of DSPs by integrating multiple small bit-width operations on one DSP. Then the LB-level spatial mapping is used to exploit the complementary features between different neural networks in the hybrid-NN. We evaluate the mapping mechanism by implementing four hybrid-NNs on Xilinx Virtex7 690T FPGA. The proposed mechanism achieves a peak performance of 1805.8 giga operations per second (GOPs). With the analysis on resource utilization and throughput, the proposed method exploits more computing power in FPGA and achieves up to $4.13 \times$ higher throughput than the state-of-the-art acceleration.

The coarse-to-fine shape searching (CFSS) framework is a recently developed algorithm that achieves relatively high accuracy in face alignment by alleviating the poor initialization problem facing traditional cascaded regression approaches. However, its high computational complexity and memory access demands make it difficult for CFSS to satisfy the requirements of real-time processing. To address this issue, a fast shape searching face alignment (F-SSFA) accelerator is presented based on the optimization of the CFSS algorithm and an efficient hardware implementation. First, the learning-based low-dimensional speeded-up robust features method, based on the correlations between the SURF features and the regression targets, is introduced to distill the feature set down to the only most distinct features to reduce the computing load. Second, the partial keypoints Euclidean distance and shape affine transformation are introduced to replace feature extraction and support vector machine classification, thereby accelerating the shape searching process. Compared with CFSS, F-SSFA achieves a $5.8\times $ speedup while achieving similar accuracy. Moreover, a VLSI architecture is proposed to realize the fixed-point F-SSFA algorithm. Multiple descriptors located in adjacent regions are simultaneously generated in a single access to the corresponding image data. Therefore, repeated memory access operations are avoided. The optimal parameter configuration for hardware implementation is also exploited based on a tradeoff between accuracy and hardware performance. Simulated with TSMC 65-nm 1P8M technology within a 3.6 mm2area, a post-layout simulation shows that 700 fps can be achieved while consuming 300 mW at 200 MHz.

Due to inherent error resiliency, many high performance multimedia applications can be approximated by multilayer perceptrons (MLPs), with little quality loss. An MLP accelerator can be designed to improve the power efficiency of multimedia systems. However, previous MLP accelerators' fixed computational pattern lowers the performance when the MLP topology varies for different applications. In this paper, we propose a scheduling framework to guide mapping MLPs onto limited hardware resources. The scheduling framework adjusts the computational patterns for various MLP topologies, obtaining 30% higher performance than the conventional scheduling. We implement a reconfigurable neural architecture (RNA) to support different patterns in the framework and further improve the performance and efficiency. RNA achieves a speedup of 572× on the approximable part, whole application speedup of 7.9× and energy savings of 6.3×, with little quality loss on the benchmarks.

Face alignment is a critical task in many multimedia and vision applications that use face-based algorithms. Recent research has focused on achieving efficient initialization to improve performance; however, the use of facial attributes and the extent of their correlation with initialization have not been fully exploited. This paper presents a lightweight method called expression- and pose-based adaptive initialization (EXPAI), in which facial attributes, that is, expression and pose information, are used as priors. This approach can significantly improve the face alignment performance. In addition, reliable expression and head pose information can be derived simultaneously in the same framework. First, an expression- and pose-based template dictionary is formed by augmenting the mean shape across three degrees of freedom, thereby substantially improving the robustness of the initial shape with respect to large head pose variations. Second, each the template corresponds to an image of interest, which is jointly determined using a shape-constrained multiclass classifier and binary classifiers, and is assigned a pretrained confidence coefficient. The initial shape that is thus generated for subsequent cascaded regression is more adaptive and enables higher accuracy. Furthermore, EXPAI enables initialization with significantly increased computational efficiency because of its independence from the original dataset. The experimental results obtained on the widely used 300-W dataset show that our method achieves very competitive performance compared with that of state-of-the-art methods. In particular, for the challenging subset of 300-W, EXPAI reduces errors by more than 14% compared with coarse-to-fine shape searching (CFSS), which currently exhibits the best performance among regression-based approaches. Furthermore, a speed increase of more than 10 times compared with CFSS is achieved.

Recent advances in deep learning (DL) have stimulated increasing interests in neural networks (NN). From the perspective of operation type and network architecture, deep neural networks can be categorized into full convolution-based neural network (ConvNet), recurrent neural network (RNN), and fully-connected neural network (FCNet). Different types of neural networks are usually cascaded and combined as a hybrid neural network (Hybrid-NN) to complete real-life cognitive tasks. Such hybrid-NN implementation is memory-intensive with large number of memory accesses, hence the performance of hybrid-NN is often limited by the insufficient memory bandwidth. A “3D + 2.5D” integration system, which integrates a high-bandwidth 3D stacked DRAM side-by-side with a highly-parallel neural processing unit (NPU) on a silicon interposer, overcomes the bandwidth bottleneck in hybrid-NN acceleration. However, intensive concurrent 3D DRAM accesses produced by the NPU lead to a serious thermal problem in 3D DRAM. In this paper, we propose a neural processor calledParanafor hybrid-NN acceleration in consideration of thermal problem of 3D DRAM. Parana solves the thermal problem of 3D memory by optimizing both the total number of memory accesses and memory accessing behaviors. For memory accessing behaviors, Parana balances the memory bandwidth by spatial division mapping hybrid-NN onto computing resources, which efficiently avoids that masses of memory accesses are issued in a short time period. To reduce the total number of memory accesses, we design a new NPU architecture and propose a memory-oriented tiling and scheduling mechanism to exploit the maximum utilization of on-chip buffer. Experimental results show that Parana reduces the peak temperature by up to 54.72$^\circ$C and the steady temperature by up to 32.27$^\circ$C over state-of-the-art accelerators with 3D memory without performance degradation.

Deep convolutional neural networks (DCNNs) have been widely used in various AI applications. Inception and Residual are two promising structures adopted in many important modern DCNN models, including AlphaGo Zero's model. These structures allow considerably increasing the depth and width of the network to improve accuracy, without increasing the computational budget or the difficulty of convergence. Various accelerators for DCNNs have been proposed based on FPGA platform because it has advantages of high performance, good power efficiency, and fast development round, etc. However, previous FPGA mapping methods cannot fully adapt to the different data localities among layers and other characteristics of Inception and Residual, which leads to a under-utilization of FPGA resources. We propose LCP, a Layer Clusters Paralleling mapping method to classify the layers into clusters based on their differences of parameters and data localities, and then accelerate them in different partitions of FPGA. We evaluate our mapping method by implementing Inception/Residual modules from GoogLeNet [8] and ResNet-50 [4] on Xilinx VC709 (Virtex 690T) FPGA. The results show that the proposed method fully utilizes resources and achieves up to 4.03× performance than the baseline and 2.00× performance than the state-of-the-art methods.

While deep convolutional neural networks (CNNs) have emerged as the driving force of a wide range of domains, their computationally and memory intensive natures hinder the further deployment in mobile and embedded applications. Recently, CNNs with low-precision parameters have attracted much research attention. Among them, multiplier-free binary- and ternary-weight CNNs are reported to be of comparable recognition accuracy with full-precision networks, and have been employed to improve the hardware efficiency. However, even with the weights constrained to binary and ternary values, large-scale CNNs still require billions of operations in a single forward propagation pass.

The growing size of convolutional neural networks (CNNs) requires large amounts of on-chip storage. In many CNN accelerators, their limited on-chip memory capacity causes massive off-chip memory access and leads to very high system energy consumption. Embedded DRAM (eDRAM), with higher density than SRAM, can be used to improve on-chip buffer capacity and reduce off-chip access. However, eDRAM requires periodic refresh to maintain data retention, which costs much energy consumption. Refresh is unnecessary if the data's lifetime in eDRAM is shorter than the eDRAM's retention time. Based on this principle, we propose a Retention-Aware Neural Acceleration (RANA) framework for CNN accelerators to save total system energy consumption with refresh-optimized eDRAM. The RANA framework includes three levels of techniques: a retention-aware training method, a hybrid computation pattern and a refresh-optimized eDRAM controller. At the training level, CNN's error resilience is exploited in training to improve eDRAM's tolerable retention time. At the scheduling level, RANA assigns each CNN layer with a computation pattern that consumes the lowest energy. At the architecture level, a refresh-optimized eDRAM controller is proposed to alleviate unnecessary refresh operations. We implement an evaluation platform to verify RANA. Owing to the RANA framework, 99.7% eDRAM refresh operations can be removed with negligible performance and accuracy loss. Compared with the conventional SRAM-based CNN accelerator, an eDRAM-based CNN accelerator strengthened by RANA can save 41.7% off-chip memory access and 66.2% system energy consumption, with the same area cost.

Convolutional neural networks (CNNs) have achieved great success in many applications. Recently, various FPGA-based accelerators have been proposed to improve the performance of CNNs. However, current most FPGA-based methods only use the same bit-width selection for all CNN layers which lead to very low resource utilization and difficulty in further performance improvement. In this paper, we propose a bit-width adaptive accelerator design approach which can adapt to the CNN layers with various bit-width requirements in a same network. We construct multiple different bit-width convolutional processors to compute the CNN layers in parallel way. We partition the FPGA DSP resources and use our optimization approach to find the optimal resource allocation. On a Xilinx Virtex-7 FPGA, our design approach achieves higher throughput than the state-of-the-art FPGA-based CNN accelerators from 5.48× to 7.25× and by 6.20× on average, when we evaluate the convolutional layers of AlexNet and deeper VGG CNNs.

JPEG compression is an important part in low-power multimedia applications. This paper proposes an approach that leverages the error resilience of JPEG for different energy budgets. We select and train neural networks to approximate DCT and quantization code regions in JPEG. Then we design an architecture called reconfigurable neural unit (RNU) to accelerate trained neural networks which replace original codes. In addition, some architecture innovations are proposed to make our JPEG encoder works efficiently in near-threshold voltage region. This JPEG encoder synthesized with a 40nm CMOS technology, is able to operate at 40MHz for a 0.6V supply voltage. Results show up to 5.0 × energy reduction with 2.5 × performance degradation when compared to using a 1.0V nominal supply voltage.

This paper presents a dynamically reconfigurable processing array named Anole for symmetric-key algorithms. Processing elements and the interconnections between them are designed to support various block and stream ciphers. Without affecting flexibility, three key techniques are presented to increase energy efficiency (throughput/power, the number of operations per unit energy consumption) and area efficiency (throughput/area). First, the distributed control network supports multithreading on reconfigurable fabrics at a low cost, thereby maximizing the utility of computing resources in the space domain. Second, the concurrent computation and reconfiguration scheme integrates configuration contexts with processing data to simultaneously execute in the data-path. The resulted immediate switching between different configurations increases the utilization rate of hardware resources in the temporal domain. Third, under configuration context compression and organization, the context memory size and configuration time are further minimized. Anole is implemented on a 7.75 mm2silicon square with TSMC 65-nm technology at 400 MHz. Experiments show that Anole significantly outperforms field programmable gate array and general purpose processor by more than two orders of magnitude in energy and area efficiencies. Compared with state-of-the-art reconfigurable solutions, Anole achieves (average) 16.5× higher energy efficiency and 9.4× higher area efficiency.

Coarse-grained reconfigurable arrays (CGRAs) can be dynamically programmed by configuration contexts to concurrently run multiple operations on a processing elements array. This further widens the gap between off-chip memory bandwidth demand and the limited speed of off-chip memory access. Cache prefetching is widely used for mitigating off-chip memory latency. However, straightforwardly applying existing prefetching techniques (primarily focusing on instruction driven processors) to CGRA may induce inaccurate prefetching, thereby crippling CGRA performance. Based on repetitively executed context in CGRA computing, this paper proposes a context-directed pattern matching (CDPM) mechanism to improve prefetching accuracy for CGRAs. CDPM generates a prefetch pattern for an initially executed context, and then reuses the pattern to issue prefetch requests when the context is re-executed. In order to eliminate the outdated prefetch pattern, CDPM also evaluates the prefetching accuracy of the prefetch pattern at run-time by adding prefetch addresses to a Bloom filter. The distinguishing feature of CDPM is the employment of the CGRA configuration context as a guide to improving prefetching accuracy. Experimental results showed that CDPM prefetching averagely improved performance by 31.1% compared to tests without prefetching and by 7.7% compared to state-of-the-art cache prefetching techniques, while only incurring slight area and power overheads.

Coarse grained reconfigurable architectures (CGRA) have been applied to numerous fields of computing- and data-intensive applications, such as computer vision, baseband communication, and cipher processing. A CGRA usually comprises hundreds of reconfigurable computing-cells (RCC), which account for a majority of the die area. As such, RCCs have a higher probability of being attacked by hardware Trojans, which seriously affects CGRA behavior. However, a CGRA can be dynamically and partially reconfigured via configuration contexts at runtime; this property could be utilized as an effective countermeasure against malicious hardware. This particular topic has yet to undergo significant research. This paper proposes a secure mapping approach called dynamic resource management based on security value (DRMaSV) to enhance CGRA capability against hardware Trojans by selectively protecting RCCs. DRMaSV realizes run-time monitoring based on an adapted triple modular redundancy mechanism under hardware resource constraints (i.e., area constraints). First, in order to measure the capability against hardware Trojans, a security capability metric called “security value” (SV) is defined, with measurements categorized as “Influence” and “Unreliability.” Here, both the circuit architecture and the level of Unreliability for modules used in the circuit are considered. Next, a DRM strategy to maximize the SV under hardware resource constraints is introduced. This strategy is described by the dynamic programming model (i.e., 0/1 knapsack problem), which can obtain an optimal solution. Finally, a mapping approach for CGRAs is derived by attaching the DRM strategy to a generic mapping flow. Simulations show that the proposed secure mapping approach ensures a given number of correct outputs, which then allows the number of outputs affected by activated Trojans under any given hardware resource constraint (area constraint) or overhead (area overhead) to be minimized. The results of actual chip design experiments are in agreement with the simulation results, indicating that the proposed secure mapping approach is effective.

Generative networks have become ubiquitous in image generation applications like image super-resolution, image to image translation, and text to image synthesis. They are usually composed of convolutional (CONV) layers, convolution-based residual blocks, and deconvolutional (DeCONV) layers. Previous works on neural network acceleration focus too much on optimizing CONV layers computation such as data-reuse or parallel computation, but have low processing element (PE) utilization in computing residual blocks and DeCONV layers: residual blocks require very high memory bandwidth when performing elementwise additions on residual paths; DeCONV layers have imbalanced operation counts for different outputs. In this paper, we propose a dual convolution mapping method for CONV and DeCONV layers to make full use of the available PE resources. A cross-layer scheduling method is also proposed to avoid extra off-chip memory access in residual block processing. Precision-adaptive PEs and buffer bandwidth reconfiguration are used to support flexible bitwidths for both inputs and weights in deep neural networks. We implement a generative network accelerator (GNA) based on intra-PE processing, inter-PE processing, and cross-layer scheduling techniques. Owing to the proposed optimization techniques, GNA achieves energy efficiency of 2.05 TOPS/W with 61% higher PE utilization than traditional methods in generative network acceleration.

Memory bandwidth bottlenecks severely restrict parallel access of data elements from data arrays. To realize high throughput out of a relatively low bandwidth, memory partitioning algorithms have been proposed to separate data arrays into multiple memory banks, from which multiple data can be accessed in parallel. However, previous partitioning schemes only considered the case of single-pattern and single-array. In the case of multipattern and multiarray, the previous partitioning schemes will use too much time to find a partition solution and cause excessively high storage overhead. In this paper, we propose an efficient two-step memory partitioning strategy for multipattern data access in multiple arrays. First, a fast, low complexity and low overhead difference-based data splitting algorithm provides a multibank solution for multiple patterns access. Then an area-efficient bank merging algorithm merges those partitioned banks from different arrays which satisfy conflict-free requirement in order to reduce the area overhead caused by partitioning. Experimental results show that our data splitting algorithm saves up to 83.0% in searching time and reduces 39.4% storage overhead compared to the state-of-the-art approaches. With the further optimization of area-efficient bank merging, the memory area overhead are saved up to 18.9% and the total partitioning time are saved up to 45.6%.

With VLSI process technology scaling into nano-scale, the increasingly serious aging issues (e.g., NBTI and HCI aging effects) have brought a significant threat to system reliability. Coarse-grained reconfigurable architectures (CGRAs) exhibit the feature to reconfigure and execute different mapping schemes (Maps) dynamically, compensating for each other to mitigate aging issues effectively. In this paper, a two-stage stress-aware loops mapping algorithm is first proposed for the CGRA-mapped designs by jointing the intra-kernel and inter-kernel stress optimizations. With pipelining techniques, the intra-kernel stress optimization employs the stress-aware force-directed and effective MCC (Maximal Compatibility Class) methods to optimize operations' placement and mapping distribution on processing elements (PEs), which helps to avoid overmany operations to be mapped on the same PEs and reduce the accumulated stresses. By leveraging the dynamic reconfiguration feature, the inter-kernel stress optimization develops a multi-map scheduling method to reconfigure a set of ordered maps on CGRA dynamically, which diversifies the PEs' usage and compensates for the stresses on different PEs among them. Experimental results show that our approach can reduce the maximum stress by 82.0% for NBTI and 70.4% for HCI, and improve the aging efficiency by 6.01X and MTTF by 3.16X averagely, while keeping the optimized performance.

Distributed controlled coarse-grained reconfigurable arrays (CGRAs) enable efficient execution of irregular control flows by reconciling divergence in the processing elements (PEs). To further improve performance by better exploiting spatial parallelism, the triggered instruction architecture (TIA) eliminates the program counter and branch instructions by converting control flows into predicate dependencies as triggers. However, pipeline stalls, which occur in pipelines composed of both intra and inter-PEs, remain a major obstacle to the overall performance. In fact, the stalls in distributed controlled CGRAs pose a unique problem that is difficult to resolve by previous techniques. This work presents a triggered-issuance and triggered-execution (TITE) paradigm in which the issuance and execution of instructions are separately triggered to further relax the predicate dependencies in TIA. In this paradigm, instructions are paired as dual instructions to eliminate stalls caused by control divergence. Tags that identify the data transmitted between PEs are forwarded for acceleration. As a result, pipeline stalls of both intra- and inter-PEs can be significantly minimized. Experiments show that TITE improves performance by 21 percent, energy efficiency by 17 percent, and area efficiency by 12 percent compared with a baseline TIA.

Coarse Grained Reconfigurable Architectures (C-GRAs) have been paid an increasing attention due to their inherent advantages of high performance and energy efficiency. As we know, multi-Vddtechnique is popularly used to reduce energy consumption, and modulo scheduling is one of widely-used pipeline techniques to improve performance. To achieve both high performance and energy-efficiency simultaneously, this paper proposes an energy-aware mapping algorithm integrating multi-Vddassignment into the scheduling and mapping procedures of loop applications. Also, an energy-aware FDS (eFDS) algorithm and a rapid MCC searching method based on compatibility concept are successfully adopted to solve the bi-objective optimization problem. The experimental results show that the proposed approach brings 18.7% energy reduction and 1.44X energy-efficiency improvement while keeping optimized performance.

With the process scaling into nano-scale VLSI technology, the increasingly serious aging issues (e.g. NBTI aging effect) bring a significant threat to system reliability. Coarsegrained reconfigurable architectures (CGRAs) exhibit the feature to reconfigure different mapping schemes (Maps) dynamically during loops execution, which can mitigate the aging issues on CGRAs effectively. In this paper, we propose a stress-aware loops mapping algorithm by jointing intra-kernel and inter-kernel stress optimizations strategies in the early phase of CGRA-mapped designs. With the pipelining technique, a stress-aware force-directed method is introduced in the intra-kernel optimization, avoiding many operations to be mapped on some certain PEs and reducing the stresses accumulated on them. By leveraging the dynamic reconfiguration, a multi-map scheduling method is proposed in the inter-kernel stress optimization to find a set of ordered maps to reconfigure dynamically, which diversifies PE usages and compensates for the accumulated stresses on different PEs among them. Experimental results show our proposed approach enlarges the maximum stress reduction up to 78.9% and improves the MTTF by 340.3% on average while keeping the optimized performance.

Long Short-Term Memory (LSTM) based Recurrent Neural Networks (RNNs) are promising for cognitive intelligence applications like speech recognition, image caption and nature language processing, etc. However, the cascade dependent structure in RNN with huge amount of power inefficient operations like multiplication, memory accessing and nonlinear transformation, could not guarantee high computing speed and low power consumption. In this work, by exploiting semantic correlation, we propose a semantic correlation based data pre-fetch method to break the dependency and achieve parallel processing. Based on this method, a full parallel and pipeline architecture that tackles huge amount operations is designed. Experiments on benchmarks of image caption, speech recognition and language processing show that, this work improves computing speed by 5.1 times, 44.9 times and 1.53 times, respectively, and power efficiency by 1885.7 times, 4061.5 times and 127.5 times, respectively, when compared with state-of-the-art works.

Spin-transfer torque random access memory (STT-RAM) has been proposed to be an excellent candidate for substituting traditional memory due to its fascinating features such as high density and low power. Memory partitioning is an efficient strategy to overcome the obstacle of memory bandwidth limiting speed of parallel data access. However, the performance is unsatisfactory, while previous memory partitioning methods are applied to STT-RAM, since they have no regard for the problem of read disturbance. In this paper, a disturbance aware memory partitioning (DaMP) method for STT-RAM is proposed. The experimental results show DaMP outperforms state-of-the-art method in terms of bank number, storage overhead, performance and searching speed.

Convolutional neural networks (CNNs) have achieved great success in many applications. Recently, various FPGA-based accelerators have been proposed to improve the performance of CNNs. However, current most FPGA-based methods use single bit-width selection for all CNN layers, which lead to very low resource utilization efficiency and difficulty in further performance improvement. In this paper, we propose a new approach utilizing bit-width partitioning of FPGA DSP resources to improve the performance and resource utilization efficiency of CNN accelerator. Moreover, we use optimization approach to find the optimal allocation plan for DSP resources. On a Xilinx Virtex-7 FPGA, our design approach achieves performance over the state-of-the-art FPGA-based CNN accelerators from 5.48x to 7.25x and by 6.21x on average, when we evaluate the popular CNNs.

Visual feature detection has been widely used in many computer vision applications. In comparison with popular feature detection algorithms, AFFINE-SIFT (ASIFT) achieves the strongest robustness on image illumination, image rotation and image scale transformation, etc., however, since feature vectors of different feature points need to be computed, feature generation of ASIFT exhibits high computing complexity. Meanwhile, it also shows low efficiency on parallel memory accessing since multiple computing processes of feature points need to access the same memory concurrently, which causes memory accessing conflicts and incurs high power consumption. In this work, we improve the feature generation based on a rotation invariant block binary pattern(RIBBP) to reduce computation and design a parallel processing architecture. Meanwhile, a spatial relevance based memory accessing(SRBMA) optimization is performed to improve data reuse and enhance parallel memory accessing. Evaluations using TSMC 65 nm 1P9M LP process show that this work improves the processing speed and energy efficiency of feature generation by 17% and 37.9%, respectively, when compared with state-of-the art work.

High-Level Synthesis (HLS) has been widely recognized as an efficient compilation process targeting FPGAs for algorithm evaluation and product prototyping. However, the massively parallel memory access demands and the extremely expensive cost of single-bank memory with multi-port have impeded loop pipelining performance. Thus, based on an alternative multi-bank memory architecture, a joint approach that employs memory-aware force directed scheduling and multi-cycle memory partitioning is formally proposed to achieve legitimate pipelining kernel and valid bank mapping with less resource consumption and optimal pipelining performance. The experimental results over a variety of benchmarks show that our approach can achieve the optimal pipelining performance and meanwhile reduce the number of multiple independent memory banks by 55.1% on average, compared with the state-of-the-art approaches.

The coarse-grained reconfigurable architecture (C-GRA) is a promising platform that provides both high performance and high power-efficiency. Dataflow graph (DFG) mapping is critical to tap the potentials of CGRAs. Inspired from the great progress made in tree search game using deep neural network, we proposed a frame work for learning convolutional neural network for mapping DFGs onto spatial programmable CGRAs. Considering the mapping process, we present a dual-input neural network capturing the features from both DFGs in applications and Process Element Array (PEA) in CGRA. In order to train the neural network, algorithms are designed to automatically generate a data set from PEA intermediate states of preprocessed DFG. Finally, experimental results demonstrate that our proposed mapping approach is competitive with state-of-the-art DFG mapping algorithms in performance while the compilation time is greatly reduced.

The in‐loop filter comprises deblocking filter and sample adaptive offset filter, which is an important module for improving image quality in a high‐efficiency video coding (HEVC) decoder. The in‐loop filter has a high computational complexity that accounts for ∼20% of the HEVC decoding computing load. Furthermore, it is difficult to implement a high‐performing in‐loop filter due to its large conditional processing requirement. First, this study presents a novel reconfigurable HEVC in‐loop filter implementation on a coarse‐grained dynamically reconfigurable processing unit. Next, a repartition scheme is presented that allows the in‐loop filter implementation at a coding tree unit along with the other decoding modules in the HEVC decoder, which satisfies requirements of low latency applications. Finally, a hierarchised‐pipeline and synchronised‐parallel technique is used to improve performance by eliminating data hazards in pipeline techniques and synchronisation problems in parallel techniques. Implementation results show that the presented HEVC in‐loop filter performs up to 1920 × 1080@52 frames per second at 250 MHz. The throughput is 67.5 × 9 × more than solutions based on digital signal processor and general‐purpose processor, respectively.

Traditional detection countermeasures against fault attacks have been criticized as insecure because of the fragile comparison operation that can be maliciously bypassed. In order to avoid the comparison, infection countermeasures have been designed to confuse the faulty ciphertexts so that the output cannot be further explored. This paper presents an infection method that resists fault attacks using the existing Benes network module in high-performance crypto processors. The Benes network is originally used to accelerate permutation operations in block ciphers. The hamming weight of the differential results is balanced by modifying specific network switches, without changing the network topology. A further confusion is performed to destroy the determinacy by configuring part of the network with a random bit-stream. Furthermore, a statistical evaluation method is presented to quantitatively verify the proposed countermeasure in addition to a formal proof of security. This also provides a new concept for the evaluation of future random-enhanced infection methods. Experiments are carried out using Advanced Encryption Standard (AES), triple Data Encryption Standard (DES), and Camellia as examples. Under statistical evaluation, the results show that the proposed countermeasure improves the fault resistance by over four orders of magnitude compared with the unprotected case. Also, the performance and the area overhead are within 10% compared with the original Benes network.

In this paper, a multi-objective, i.e., reliability, communication energy, performance, co-optimization model oriented mapping approach is proposed to find optimal mappings when applications are mapped onto network-on-chip (NoC) based reconfigurable architectures. A co-optimization model, defined as reliability efficiency model (REM), is developed to evaluate the overall reliability efficiency of a mapping. In REM, reliability efficiency is defined as the reliability profit at the same energy latency product. Based on REM, a mapping approach, referred to as priority and compensation factor oriented branch and bound (PCBB), is introduced to figure out the best mapping pattern. Two techniques, priority allocation and compensation factor utilization, are adopted to make a tradeoff between search efficiency and accuracy. Experimental results show that the proposed approach has three major contributions compared to state-of-the-art approaches. (1) PCBB is highly efficient in finding best mappings, with a 3x and 720x speedup compared to branch and bound (BB) and simulated annealing (SA). (2) PCBB is able to dynamically remap after the reconfiguration of the architecture. (3) General quantitative evaluation for reliability, communication energy and performance are made respectively before integrated into the unified model REM, whereas other similar models only touch upon two of them quantitatively.

Multiple coarse-grained reconfigurable arrays (CGRA), which are organized in parallel or pipeline to complete applications, have become a productive solution to balance the performance with the flexibility. One of the keys to obtain high performance from multiple CGRAs is to manage the shared on-chip cache efficiently to reduce off-chip memory bandwidth requirements. Cache partitioning has been viewed as a promising technique to enhance the efficiency of a shared cache. However, the majority of prior partitioning techniques were developed for multi-core platform and aimed at multi-programmed workloads. They cannot directly address the adverse impacts of data correlation and computation imbalance among competing CGRAs in multi-CGRA platform. This paper proposes a correlation- and iteration- aware cache partitioning (CIACP) mechanism for shared cache partitioning in multiple CGRAs systems. This mechanism employs correlation monitors (CMONs) to trace the amount of overlapping data among parallel CGRAs, and iteration monitors (IMONs) to track the computation load of each CGRA. Using the information collected by CMONs and IMONs, the CIACP mechanism can eliminate redundant cache utilization of the overlapping data and can also shorten the total execution time of pipelined CGRAs. Experimental results showed that CIACP outperformed state-of-the-art utility-based cache partitioning techniques by up to 16 percent in performance.

Coarse-grained reconfigurable architecture (CGRA) is a promising architecture with high performance, high power-efficiency and attraction of flexibility. The computation-intensive parts of an application (e.g., loops) are often mapped on CGRA for acceleration. Due to the high parallel data access demands, the architecture with multi-bank memory is proposed to improve parallelism. For CGRA with multi-bank memory, a joint solution, which simultaneously considers the memory partitioning and modulo scheduling, is proposed to achieve a valid mapping with better performance. In this solution, the modulo scheduling and operator scheduling are used to achieve a valid loop mapping and a valid data placement without any memory access conflicts. By avoiding the pipelining stalls caused by conflicts, the performance of loop mapping is greatly improved. The experimental results on benchmarks of the Livermore, Polybench and Mediabench show that our approach can improve the performance of loops on CGRA to 1.89×, 1.49× and 1.37× compared with REGIMap, HTDM and REGIMap with memory partitioning, at cost of an acceptable increase in compilation time.

Deep convolutional neural networks (DCNNs) have been successfully used in many computer vision tasks. Previous works on DCNN acceleration usually use a fixed computation pattern for diverse DCNN models, leading to imbalance between power efficiency and performance. We solve this problem by designing a DCNN acceleration architecture called deep neural architecture (DNA), with reconfigurable computation patterns for different models. The computation pattern comprises a data reuse pattern and a convolution mapping method. For massive and different layer sizes, DNA reconfigures its data paths to support a hybrid data reuse pattern, which reduces total energy consumption by 5.9~8.4 times over conventional methods. For various convolution parameters, DNA reconfigures its computing resources to support a highly scalable convolution mapping method, which obtains 93% computing resource utilization on modern DCNNs. Finally, a layer-based scheduling framework is proposed to balance DNA's power efficiency and performance for different DCNNs. DNA is implemented in the area of 16 mm2at 65 nm. On the benchmarks, it achieves 194.4 GOPS at 200 MHz and consumes only 479 mW. The system-level power efficiency is 152.9 GOPS/W (considering DRAM access power), which outperforms the state-of-the-art designs by one to two orders.

Coarse-grained reconfigurable architecture (CGRA) is a promising platform for loop acceleration, but existing software pipelining methods cannot achieve satisfactory performance on a fair number of imperfect nested loops, especially those with sibling inner loops. To tackle this problem, this paper makes 2 contributions: 1) a 2-level pipelining method with an effective II optimization strategy for the imperfect loops with sibling inner loops; 2) a novel kernel compression method to reduce oversize kernel. Experiment results show that our approach can achieve much higher performance than the state-of-the-art approaches at acceptable costs.

This paper proposes a context directed pattern matching (CDPM) mechanism, which employs the context of the coarse-grained reconfigurable arrays (CGRAs) as a guide to improve cache prefetching accuracy. CDPM generates a prefetch pattern for an initially executed context, and reuses the pattern to issue prefetch requests when the context is again executed on CGRA. To eliminate the outdated prefetch pattern, CDPM also evaluates the prefetching accuracy of the prefetch pattern at run-time. Experiments showed that CDPM averagely improved performance by 31.1% compared to tests without any prefetching and by 7.7% compared to state-of-the-art prefetching techniques.

To realize high throughput out of a relatively low bandwidth, memory partitioning algorithms have been proposed to separate data arrays into multiple memory banks, from which multiple data can be accessed in parallel. However, previous partitioning schemes only considered the case of single-pattern and single-array. In this paper, we propose an efficient two-step memory partitioning strategy for multi-pattern access in multiple multidimensional arrays. First, a fast, low complexity and low difference-based data splitting algorithm provides a multi-bank solution for multiple patterns access. Then an area-efficiency bank merging algorithm reduce the area overhead caused by partitioning. Experimental results show that our memory splitting algorithm saves up to 83.0% in searching time finding a multi-bank solution, compared to the state-of-the-art approach and the storage overhead can be reduced by 34.5%. Meanwhile the area overheads are saved up to 18.86% and the whole partition time are saved up to 45.6% through our entire algorithm.

Coarse-Grained Reconfigurable Architecture (CGRA) is a promising architecture with high performance, high power-efficiency and attraction of flexibility. The compute-intensive parts of an application (e.g. loops) are often mapped onto CGRA for acceleration. Since the high-parallel demands of PEs and the extremely expensive cost of single-bank memory with multi-port, the architecture with multi-bank memory is favored increasingly. Based on this purpose, a joint solution, which simultaneously considers modulo scheduling and data placement, is proposed to achieve a valid mapping with better performance. The experimental results on loops from Livermore, Polybench and Mediabench show that our approach can significantly improve the performance of the kernels on CGRA compared with REGIMap, HTDM and REGIMap+MP, with an acceptable increase in compilation time.

Coarse-grained reconfigurable architecture (CGRA) is becoming an increasingly attractive platform because of its high performance and power (or energy) efficiency. To reduce energy consumption, the dual-Vddtechnique has been employed in CGRAs, and the modulo scheduling technique is widely used to improve performance of applications. To achieve both high performance and energy-efficiency simultaneously, this paper formulates the solution as a biobjective optimization problem of energy consumption and initiation interval of loop pipelines on CGRAs, and proposes a joint modulo scheduling and dual-Vddassignment approach. The experimental results show that the proposed approach can bring a significant energy reduction of 24.8% and kernel energy efficiency acceleration of 1.41× on average, while the performance is maintained.

Binary image descriptors, which derive image feature description from the local image patches directly, are widely adopted in the mobile and embedded applications due to lower computational complexity and memory requirement. With the aim of improving the computation efficiency without degrading recognition performance, a lightweight binary robust descriptor is proposed based on the analysis of the state-of-the-art binary descriptors in this paper. A directional edge detection and optimized keypoint score function are developed to refine the keypoints. In addition, rotation invariance is achieved by executing circular symmetric-based descriptor generation and a coarse-grained orientation calculation method concurrently. The experimental results demonstrate that the proposed keypoint detector and binary descriptor achieve more than two times speedup and at least 23.6% improvement in processing speed with comparable performance, respectively. Furthermore, a very large scale integration architecture is also designed based on in-depth exploration of bit-level and task-level parallelism. Based on the postlayout simulation in a TSMC 65-nm CMOS process, the accelerator can achieve 135 frames/s on 1080p image while only consuming 87.5 mW at a 200-MHz operating frequency.

With the increasing accuracy of fault injections, it has become possible to inject two faults into specific circuit regions precisely at a certain time. Unfortunately, most existing fault attack countermeasures are based on the single fault assumption, and it is, therefore, very difficult to resist double fault attacks. Reconfigurable array architecture (RAA) has the ability to introduce spatial and time randomness by dynamic reconfiguration, which can alleviate the threat of double fault attacks. This paper, for the first time, analyzes the double fault attack issues in the fault injection phase systematically. An evaluation model, named injection effort model (IEM), is proposed to quantify the efforts of a successful fault injection. In IEM, the real injection process is described mathematically using the probability method, so that a theoretical basis can be provided for the corresponding countermeasure design. Based on the concept of spatial and time randomization, three countermeasures are implemented on RAA for the purpose of decreasing the implementation overhead under the premise of ensuring the security. When these countermeasures are adopted, tradeoffs can be made between the double fault resistance and the extra overhead through changing the degree of randomness. Experiments are carried out to analyze the relationship between the resistance and the overhead using Advanced Encryption Standard (AES), Data Encryption Standard (DES), and Camellia. When the overhead constraints in terms of throughput, hardware resources, and energy are 5%, 35%, and 10% respectively, the double fault resistance can increase by two to four orders of magnitude (ranging from 824 to 10 149 for different algorithms).

Coarse-Grained Reconfigurable Architectures (CGRAs), which provide high performance, low power and flexibility, is viewed as a promising trend for computing. CGRAs are mostly employed to process compute-intensive kernels because of their inefficiency for control flows. Various methods have been proposed to alleviate this problem, and triggered instruction is one of the state-of-the-art techniques. In this paper, a reconfigurable architecture called Triggered-Long-Instruction Architecture (TLIA) is proposed to enhance the triggered instructions with parallel condition method. In the proposed architecture, triggered instruction set is employed on processing elements (PEs). In this way, over-serialized execution and branch instructions are both eliminated. In the meanwhile, each PE has an improved data-path with three ALUs which is inspired by the parallel condition method. In this way, the amount of parallelism inside each control flow is increased by paralleling predicate computations and predicated operations. Moreover, multiple triggered instructions, which may have internal control dependence, can be executed on PEs in parallel. The strategy of issuing instructions is implemented in hardware, and verified by FPGA. Experimental results show that the performance is improved by 20.9 to 140.0 percent, the area is reduced by 24.5 percent, and the power is reduced by 32.5 percent over the equivalent Triggered Instruction Architecture (TIA).

Coarse-grained reconfigurable architecture (CGRA) is a promising parallel computing platform that provides high performance, high power efficiency and flexibility. However, for imperfect nested loops, the existing loop mapping methods often result in low execution performance and poor hardware utilization. To tackle this problem, this paper makes three contributions:1)a highly effective and general approach to map imperfect loops on CGRA;2)a global optimization strategy to search the optimal initiation intervals (IIs);3)a powerful kernel compression method to reduce the oversized kernel. Experiment results show that our approach can reduce the total computing latency by 20.5, 58.5 and 73.2 percent compared to the state-of-the-art approaches on$2 \times 2$,$4 \times 4$and$8 \times 8$CGRA respectively. Moreover, the compilation time and configuration context size is acceptable in practice.

Coarse-grained reconfigurable architecture (CGRA) is a promising architecture with high performance, high power efficiency, and attraction of flexibility. The computation-intensive portions of applications, i.e., loops, are often implemented on CGRAs for acceleration. The loop pipelining techniques are usually used to exploit the parallelism of loops. However, for nested loops, the existing loop pipelining methods often result in poor hardware utilization and low execution performance. To tackle this problem, this paper makes three contributions: 1) we propose the use of affine transformation to facilitate nested loop pipelining; 2) based on polyhedral model, we present a precise and general formulation of the nested loop pipelining problem on a CGRA; and 3) using the insights from problem formulation, we design a joint affine transformation and multipipeline merging approach to improve the performance of nested loop on CGRA. The experimental results show that our approach can improve the performance of nested loops up to 35% on average, compared with the state-of-the-art techniques.

Integral histogram image can accelerate the computing process of feature algorithm in computer vision, but exhibits high computation complexity and inefficient memory access. In this paper, we propose a configurable parallel architecture to improve the computing efficiency of integral histogram. Based on the configurable design in the architecture, multiple integral objects for integral histogram image, such as image intensity, image gradient, and local binary pattern, are well supported. Meanwhile, by means of the proposed strip-based memory partitioning mechanism, this architecture processes the integral histogram quickly with maximal parallelism in a pipeline manner. Besides, in this architecture, the proposed data correlation memory compression mechanism effectively solves the expansion problem of integral histogram memory caused by storing the histogram data. It fully reduces the data redundancy in the integral histograms, and saves a lot of memory resources. Experiments using Cyclone IV-based field-programmable gate array platform and 65-nm technology-based postsynthesis show that our architecture improves the average computing speed by 8.6 times with high power efficiency compared with the state-of-the-art works.

Stacked DRAM used as the last-level caches (LLCs) in multicore systems delivers performance enhancement due to its capacity benefit. While the performance of LLC depends heavily upon its block replacement policy, the conventional replacement policy needs redesigning to exploit the best of DRAM cache and avoid its drawbacks. The existing DRAM cache insertion policy blindly forwards victim lines replaced from L3 to the off-chip memory, regardless of the potential for increased hits by placing a fraction of them in the DRAM cache. Nevertheless, a naïve design that steers all dirty victims to the DRAM cache introduces excessive writeback traffic, which aggravates capacity misses and DRAM interference. To leverage insertions in terms of writeback or fill requests, we propose a collective writeback and fill policy that adapts to the distinct access patterns of heterogeneous workloads based on runtime misses and writeback efficiency. The synthesis result shows that the new policy has only a small hardware overhead. According to the experimental results on the gem5 simulator, the new policy increases harmonic instruction per cycle throughput by 18%, 11.6%, and 11.7%, respectively, compared with the Always Writeback (AW)-Always Fill policy, Never Writeback Adaptive DRAM Placement policy, and AW Adaptive DRAM Placement policy on 64-MB alloy cache, while the improvement is 19.3%, 13.7%, and 14.5% on 64-MB MissMap cache.

The coarse-grained reconfigurable architectures (CGRAs) are a promising class of architectures with the advantages of high performance and high power efficiency. The compute-intensive parts of an application (e.g., loops) are often mapped onto the CGRA for acceleration. Due to the extra overhead of memory access and the limited communication bandwidth between the processing element (PE) array and local memory, previous works trying to solve the routing problem are mainly confined in the internal resources of PE arrays (e.g., PEs and registers). Inevitably, routing with PEs or registers will consume a lot of computational resources and cause the increase of the initiation interval. To solve this problem, this paper makes two contributions: 1) establishing a precise formulation for the CGRA mapping problem while using shared local data memory as a routing resource and 2) extracting an effective approach for mapping loops to CGRAs. The experimental results on loops of the SPEC2006, Livermore, and MiBench show that our approach (called MEMMap) can improve the performance of the kernels on CGRA up to 1.62×, 1.58×, 1.28×, and 1.23× compared with the edge-centric modulo scheduling, EPIMap, REGIMap, and force-directed map, respectively, with an acceptable increase in compilation time.

A coarse-grained reconfigurable architecture (CGRA) is a promising platform based on considerations for both performance and power efficiency. One of the primary obstacles that CGRAs might face is how to accelerate loops with if-then-else (ITE) structures. A recent control paradigm for CGRAs named triggered instruction architecture (TIA) can provide an efficient scheme to accelerate loops with ITE structures. Yet common loop mapping frameworks cannot leverage this scheme autonomously. To this end, this brief makes two contributions: 1) identify and remove redundancy nodes from a data flow graph and 2) propose an integrated approach-TRMap, which consists of operations merging, Boolean operations offloading, and transformation of triggers. Our experimental results from some vital kernels extracted from SPEC2006 benchmarks and digital signal processing applications show that by using TIA scheme, TRMap is able to accelerate loops with ITE structures to an execution that is 1.38× and 1.64× faster than that achieved by a full predication scheme (FP-Choi) and a state-of-the-art method (BRMap).

An approach using a minimum cost maximum flow algorithm is proposed for fault-tolerant topology reconfiguration in a Network-on-Chip system. Topology reconfiguration is converted into a network flow problem by constructing a directed graph with capacity constraints. A cost factor is considered to differentiate between processing elements. This approach maximizes the use of spare cores to repair faulty systems, with minimal impact on area, throughput and delay. It also provides a transparent virtual topology to alleviate the burden for operating systems.

Coarse-grained Reconfigurable Architecture (CGRA) is a promising mobile computing platform that provides both high performance and high energy efficiency. Since loop nests are usually mapped onto CGRA for acceleration, optimizing the mapping is an important goal for design of CGRAs. Moreover, how to reduce energy consumption also becomes one of primary concerns in using CGRAs. This paper makes three contributions: a) Proposing an energy consumption model for CGRA; b) Formulating loop nests mapping problem to minimize the battery charge loss; c) Extract an efficient heuristic algorithm called BPMap. Experiment results show that our methods improve the performance of the kernels and lower the energy consumption.

Visual feature extraction is a fundamental technique in vision-based application. This paper proposes an effective and efficient VLSI architecture based on optimized accelerated KAZE (AKAZE) for real-time feature extraction. AKAZE is a new feature detection algorithm with strong robustness for object recognition. To extract feature more robustly and reduce hardware resource, a two-dimensional pipeline array named Loop-Snake Architecture is presented. It takes advantage of computational similarity in different octaves and provides flexibility in precision-speed tradeoff on the fly. Furthermore, Polar Local Difference Binary descriptor and the corresponding structure are proposed to greatly reduce the memory bandwidth requirement and improve the speed. The experimental results indicate the optimized algorithm keeps the same accuracy compared with the original algorithm. The whole hardware system achieves 127fps in 1080p resolution at 200 MHz frequency. The throughput is twice faster than the state-of-the-art solutions.

Memory bandwidth bottlenecks severely restrict parallel access of data from memory arrays. To increase bandwidth, memory partitioning algorithms have been proposed to access multiple memory banks simultaneously. However, previous partitioning schemes propose complex partitioning algorithms, which leads to non-optimal memory bank space utilization and unnecessary storage overhead. In this paper, we develop an efficient memory partitioning strategy with low time complexity and low storage overhead for data access in multidimensional arrays. Experimental results show that our memory partitioning algorithm saves up to 93.7% in the amount of arithmetic operations, 96.9% in execution time and 31.1% in storage overhead, compared to the state-of-the-art approach.

Control-intensive kernels are becoming the bottleneck that limits the performance of Coarse-Grained Reconfigurable Architecture. Some methods, such as predicated execution, speculative execution, and dual-issue-single-execution, have been proposed to alleviate this problem. But they cannot be always efficient for various control flows. This paper proposes a new architecture, which combines the techniques of triggered instruction and parallel condition, in order to solve the problem completely. The architecture utilizes the basic framework of the triggered instruction to avoid over-serialized execution and branch instruction. Meanwhile, it takes the mechanism of the parallel condition to explore the parallelism between predicate and compute instructions without reconciliation operations. The mechanism of executing multiple instructions that have internal control dependence in parallel is discussed as well. The experiment result shows that the proposed architecture can achieve 20.9% to 140.0% higher performance than that of triggered instruction architecture in terms of cycle count.

A mixed-grained reconfigurable computing platform targeting multiple-standard video decoding is proposed in this paper. The platform integrates eight coarse-grained Reconfigurable Processing Units (RPUs), each of which consists of 16×16 multi-functional Processing Elements (PEs) and are implemented in TSMC 65 nm technology and two Altera Stratix IV EP4SE820 FPGAs. By exploiting dynamic reconfiguration of the RPUs and static reconfiguration of the FPGAs, the proposed platform achieves scalable performances and cost trade-offs to support a variety of video coding standards, including H.264, MPEG-2, AVS and HEVC. Two types of platform configuration are tested in this work. One configuration utilizes two RPUs and targets multiple-standard high-definition (HD) video decoding, while the other utilizes only one RPU, which works under a lower frequency and targets at standard resolution (SD) decoding. The HD configuration can decode 1920×1080 H.264 video streams at 30 frames per second (fps) under 200 MHz and 1920×1080 HEVC video streams at 30 fps under 236 MHz. It achieves a 25% performance gain over an industrial coarse-grained reconfigurable processor for H.264 decoding, and a 3.85× performance boosts over the Intel i5 general-purpose CPU for HEVC decoding.

Reconfigurable Architecture provides a promising solution for embedded systems for high performance, low power and flexibility. Control dependence and control divergence are critical problems that impact the performance. Many methods were proposed to handle control flows efficiently, such as predicated execution and speculative execution. However, they exhibit different performances for different types of control flows, so composite methods are required to provide overall optimal performance. In this paper, a novel architecture is proposed which combines Triggered Instruction and parallel condition. It is designed on the basis of triggered instruction architecture (TIA) while each PE incorporates multiple arithmetic logic units with fast mutual control as in the technique of parallel condition. It can remove branch instructions as well as parallelize control and compute instructions without reconciliation operation, so it explores parallelism in branch level while avoids over-serialization execution in program-counter-based PE. The experiment was conducted on a model in C language and the result shows that the proposed architecture can achieve 80.0% higher performance on average than TIA.

The memory architecture has a significant effect on the flexibility and performance of a coarse-grained reconfigurable array (CGRA), which can be restrained due to configuration overhead and large latency of data transmission. Multi-context structure and data preloading method are widely used in popular CGRAs as a solution to bandwidth bottlenecks of context and data. However, these two schemes cannot balance the computing performance, area overhead, and flexibility. This paper proposed group-based context cache and multi-level data memory architectures to alleviate the bottleneck problems. The group-based context cache was designed to dynamically transfer and buffer context inside CGRA in order to relieve the off-chip memory access for contexts at runtime. The multi-level data memory was designed to add data memories to different CGRA hierarchies, which were used as data buffers for reused input data and intermediate data. The proposed memory architectures are efficient and cost-effective so that performance improvement can be achieved at the cost of minor area overhead. Experiments of H.264 video decoding program and scale invariant feature transform algorithm achieved performance improvements of 19% and 23%, respectively. Further, the complexity of the applications running on CGRA is no longer restricted by the capacity of the on-chip context memory, thereby achieving flexible configuration for CGRA. The memory architectures proposed in this paper were based on a generic CGRA architecture derived from the characteristics found in the majority of existing popular CGRAs. As such, they can be applied to universal CGRAs.

Coarse-Grained Reconfigurable Architecture (CGRA) is a promising accelerator when considering both high performance and high power-efficiency. One of the challenges that CGRAs are confronting is to accelerate loops with control flow (if-then-else structures). Existing techniques employ predication to accelerate the conditionals but cannot accelerate nested conditionals efficiently. The state-of-the-art method dual issue scheme issues instructions from both the branch paths and then executes only the instructions from the path chosen by a predicate. But it also cannot handle nested conditionals. In this paper, we propose a solution to map loops with nested conditionals on a CGRA for the Triggered Instruction Architecture (TIA) paradigm - in which lacks compiler support. Experimental results show:We can accelerate loop kernels with nested conditionals via trigger scheme average of 1.41×, 1.79× and 1.29× better performance compared to partial predication, full predication and dual issue scheme respectively.

Approximate computing emerges as a promising technique for high energy efficiency. Multi-layer perceptron (MLP) models can be used to approximate many modern applications, with little quality loss. However, the various MLP topologies limits the hardwares performance in all cases. In this paper, a scheduling framework is proposed to guide mapping MLPs onto limited hardware resources with high performance. We then design a reconfigurable neural architecture (RNA) to support the proposed scheduling framework. RNA can be reconfigured to accelerate different MLP topologies, and achieves higher performance than other MLP accelerators.

The research on depth map is becoming a focus of image understanding and computer vision. In this paper, depth map is introduced to enhance the performance of face recognition and a novel multi-modal 2D + 3D method is proposed. First of all, we propose a new local feature descriptor called Enhanced Local Mixed Derivative Pattern (ELMDP). Then, this feature descriptor is applied on the 2D intensity image and the depth map respectively. At last the two parts of extracted feature are combined together, multiplied by corresponding confidence weights. Experiments are conducted on 3 sub-databases of Curtin Faces database which contains variations in illumination, expression, pose and disguise. Our proposed method outmatches the other methods on recognition rate and the Receiver Operating Characteristic (ROC) curve is much gentler. All the results demonstrate that the proposed method is quite outstanding and robust.

In this paper, an efficient application mapping approach is proposed for the co-optimization of reliability, communication energy, and performance (CoREP) in network-on-chip (NoC)-based reconfigurable architectures. A cost model for the CoREP is developed to evaluate the overall cost of a mapping. In this model, communication energy and latency (as a measure of performance) are first considered in energy latency product (ELP), and then ELP is co-optimized with reliability by a weight parameter that defines the optimization priority. Both transient and intermittent errors in NoC are modeled in CoREP. Based on CoREP, a mapping approach, referred to as priority and ratio oriented branch and bound (PRBB), is proposed to derive the best mapping by enumerating all the candidate mappings organized in a search tree. Two techniques, branch node priority recognition and partial cost ratio utilization, are adopted to improve the search efficiency. Experimental results show that the proposed approach achieves significant improvements in reliability, energy, and performance. Compared with the state-of-the-art methods in the same scope, the proposed approach has the following distinctive advantages: 1) CoREP is highly flexible to address various NoC topologies and routing algorithms while others are limited to some specific topologies and/or routing algorithms; 2) general quantitative evaluation for reliability, energy, and performance are made, respectively, before being integrated into unified cost model in general context while other similar models only touch upon two of them; and 3) CoREP-based PRBB attains a competitive processing speed, which is faster than other mapping approaches.

A coarse-grained reconfigurable processing unit (RPU) consisting of 16 ×16 multi-functional processing elements (PEs) interconnected by an area-efficient line-switched mesh connect (LSMC) routing is implemented on a 5.4 mm ×3.1 mm die in TSMC 65 nm LP1P8M CMOS technology. A hierarchical configuration context (HCC) organization scheme is proposed to reduce the implementation overhead and the energy dissipation spent on fast reconfiguration. The proposed RPU is integrated into two system-on-a-chips (SoCs), targeting multiple-standard video decoding. The high-performance chip, comprising two RPU processors (named REMUS_HPP), can decode 1920 ×1080 H.264 video streams at 30 frames per second (fps) under 200 MHz. REMUS_HPP achieves a 25% performance gain over the XPP-III reconfigurable processor with only 280 mW power consumption, resulting in a 14.3 × improvement on energy efficiency. The other chip (named REMUS_LPP), targeting low power applications, integrates only one RPU processor. REMUS_LPP can decode 720 ×480 H.264 video streams at 35fps with 24.5 mW under 75 MHz, achieving a 76% reduction in power dissipation and a 3.96 × improvement on energy efficiency compared with the ADRES reconfigurable processor.

With an increasing number of processing elements (PEs) integrated on a single chip, fault-tolerant techniques are critical to ensure the reliability of such complex systems. In current reconfigurable architectures, redundant PEs are utilized for fault tolerance. In the presence of faulty PEs, the physical topologies of various chips may be different, so the concept of virtual topology from network embedding problem has been used to alleviate the burden for the operating systems. With limited hardware resources, how to reconfigure a system into the most effective virtual topology such that the maximum repair rate can be reached presents a significant challenge. In this article, a new approach using a maximum flow (MF) algorithm is proposed for an efficient topology reconfiguration in reconfigurable architectures. In this approach, topology reconfiguration is converted into a network flow problem by constructing a directed graph; the solution is then found by using the MF algorithm. This approach optimizes the use of spare PEs with minimal impacts on area, throughput, and delay, and thus it significantly improves the repair rate of faulty PEs. In addition, it achieves a polynomial reconfiguration time. Experimental results show that compared to previous methods, the MF approach increases the probability to repair faulty PEs by up to 50% using the same redundant resources. Compared to a fault-free system, the throughput only decreases by less than 2.5% and latency increases by less than 4%. To consider various types of PEs in a practical application, a cost factor is introduced into the MF algorithm. An enhanced approach using a minimum-cost MF algorithm is further shown to be efficient in the fault-tolerant reconfiguration of heterogeneous reconfigurable architectures.

This paper proposes a flexible energy- and reliability-aware application mapping approach for network-on-chip (NoC)-based reconfigurable architecture. A parameterized cost model is first developed by combining energy and reliability with a weight parameter that defines the optimization priority. Using this model, the overall mapping cost could be evaluated. Subsequently, a mapping method using branch and bound with a partial cost ratio is employed to find the best mapping by enumerating all the possible patterns organized in a search tree. To improve the search efficiency, nonoptimal mappings are discarded at early stages using the partial cost ratio. Using the proposed approach, applications can be mapped onto most NoC topologies and running with various routing algorithms when considering both energy and reliability. Other state-of-the-art works have also done substantial research for the same topic but only limited to a specific topology or routing algorithm. Even for the same topology and routing algorithm, the proposed approach still shows considerable advantages in many aspects. Experiments show that this approach gains not only significant reduction in energy but also improvement in reliability. It also outperforms other approaches in throughput and latency with competitive run time.

Coarse-grained reconfigurable architectures (CGRAs) have drawn increasing attention due to their flexibility and efficiency. Loops in applications are often mapped onto CGRAs for acceleration, and the mapping of loops onto CGRA is quite a challenging work due to the parallel execution paradigm and constrained hardware resource. To map loops onto CGRAs efficiently, it is important to transform loops into pieces that obey hardware resource constraints with less overhead (e.g., communication and configuration overhead). In this paper, we tackle this problem by establishing a performance optimization problem, including loop transformation and back- end placing and routing. A novel searching strategy is also designed to find the optimal result efficiently. Finally, we built a complete flow of mapping loop nests onto CGRA. Experiment results on most kernels of the Polybench show that our proposed approach can improve the performance of the kernels by 42% on average, as compared with the state-of-the-art methods. The runtime complexity of our approach is also acceptable.

Coarse-grained reconfigurable architecture (CGRA) can provide strong capability of parallel computation and flexibility; it is becoming a promising platform for mobile computing. As mobile platforms increasingly demand power, more and more mobile platforms adopt multibattery- or multicell-based power systems to extend battery runtime. This paper addresses energy management for the purpose of extending the lifetime of battery-powered reconfigurable computing platforms. Considering the nonlinear characteristics of batteries and working mechanism of the CGRA, a multiobjective optimization model with respect to the battery state and energy consumption is built for extending battery lifetime. Meanwhile, based on this optimization model, a joint task-mapping and battery-scheduling method is proposed to achieve a practical and efficient solution. The physical experiments show that this paper achieves higher improvement on battery runtime when compared with state-of-the-art works.

With the development of parallel computing, the compute-intensive part of an application could be accelerated so dramatically that the control intensive part, usually processed by a sequential processor, is becoming more and more critical in terms of performance and power consumption. To address this problem, this paper proposes a novel reconfigurable architecture to execute control-intensive kernels efficiently. The architecture applies three key design methods. The first one, parallel condition, exploits the instruction level parallelism of conditional branches with hardware design. The second one, configuration branch, enables the architecture to independently execute an entire application that has loops and other control flows. The third one, compound configuration, combines multiple configurations of low hardware utilization, which are common in sequential codes particularly, and thus reduces the reconfiguring times. Therefore, to offload control-intensive kernels onto the proposed architecture will speed up these workloads and boost the overall performance. The experiments were conducted on a benchmark that contains various branches, loops, and sequential codes. The results showed that the proposed architecture alone could implement the benchmark correctly. In addition, the proposed methods can improve performance by over 40% compared with the conventional techniques. The power efficiency is two orders larger than general purpose processors.

The coarse-grained reconfigurable architecture (CGRA) is a promising platform for mobile computing. In this paper, how to prolong the lifetime of battery-powered reconfigurable computing platform is addressed. Considering the nonlinear characteristics of battery, a multi-objective optimization model is built for extending the lifetime of battery. Based on this model, a joint task-mapping and battery-scheduling method is proposed. The experimental results show that the proposed method achieves 26.22% improvement of battery runtime on average comparing to the state-of-the-art methods.

A coarse-grained reconfigurable architecture is a promising architecture with high power efficiency, which is typically composed of a host controller and a processing element array (PEA). Loops are often mapped onto PEAs for acceleration. In previous work, innermost loop is pipelined, and the the maximal number of concurrently executable operators (CEOs) in the kernel is limited by the inner loop. The loop body DFG of the input 2D nested loop with a inner loop carried dependence ([0,1]) and outer loop carried dependence ([1,1]). We would map this loop onto a 4×4 PEA with pipelining. We assume that the latency of executing one loop iteration is Lb, and the number of iterations involved at one cycle in the kernel phase of pipelining is Wk. As there is a inner loop dependence ([0,1]), the initiation interval (IIi) of inner loop pipelining could be minimized to 1 and we get Wk= 4. We also note that the angle α is contained by two sides in Figure 1(b), which could be written as follow: tan(α) = Wk/Lb = 1/IIi.

With the prevalence of reconfigurable computing, many relevant courses are designed and taught to graduate students. Traditional Field Programmable Gate Arrays (FPGAs) based hardware platforms are far from satisfying to reflect the important criteria characterizing a general reconfigurable computing system. In order to provide students a comprehensive understanding of reconfigurable computing system in a broader way, this paper presents a mixed-grained educational hardware platform. Different from the traditional ones, the proposed hardware platform includes not only fine-grained reconfigurable fabrics (e.g. FPGAs), but also coarse-grained ones which makes it possible to reveal essential features and intrinsic mechanisms of reconfigurable computing system. Utilizing this hardware platform, a course including four hands-on laboratory projects is designed. The feedback from students and teachers confirms that with the help of the proposed hardware platform, a thorough understanding of reconfigurable computing systems is achieved in an intuitive way and the practical experience is also significantly enhanced.

This paper proposes three configuration approaches to improve computing efficiency of a coarse-grained reconfigurable array, including input data relocation, line-based context switching, and loop interval minimization. These proposed approaches fully exploit the parallelism and pipelining of the reconfigurable array, which reduce interval latency when switching the configuration contexts, and therefore greatly enhance computing efficiency. These proposed techniques are used in a coarse-grained reconfigurable multimedia system (REMUS). Measured results show that, owing to the proposed approaches, REMUS can achieve 1080p@30fps performance for H.264 high profile video decoding under 200MHz working frequency. When normalized to the same technology, REMUS outperforms XPP-III 6.98x in energy efficiency.

Robust feature plays an important role in many vision based applications. This paper proposes a fast extreme illumination robust feature in affine space. It inherits the techniques of extreme point location and main orientation computation from SIFT (Scale Invariant Feature Transform) algorithm, and adopts the rotation and scale invariant circular binary pattern based histograms in the affine space to generate feature vectors of the extreme points. Based on the binary pattern based histograms, this work maximally improves the illumination robustness in affine space and reduces the processing time. Comparing with the typical work-ASIFT(Affine SIFT) that is characterized by strong robustness on the aspects of viewpoint, scale, rotation and illumination, this work improves the robustness for the extreme illumination change in the affine space while maintains the comparable detection performance on the other aspects, and achieves the average 82.6 times improvement on the processing time.

Our work investigates how to map loops efficiently onto Coarse Grained Reconfigurable Architecture (CGRA). This paper examines the properties of CGRA and builds Map-Reduce inspired models for the loop parallelization problem. We solve our model using Geometric Programming methods to obtain best loop unrolling parameters. Those parameters are used in the Back-End process that followed. Experiment results show the proposed approach achieved up to 44% performance gain compared to a state-of-the-art loop unrolling scheme.

This paper proposes a method of fast integral image computing on hardware. We propose a high efficient hardware-based algorithm, and design a pipelined architecture suitable for our algorithm. Parallelism and time complexity of the algorithm are analyzed. And the hardware implementation of each operations of the algorithm is presented. Compared with two related works, we find that our architecture is of the highest efficiency, as it reaches the highest speed by consuming the comparatively lowest logic resources and power.

Multiple mobile device-based collaborative computing emerges with the rapid proliferation of various smart mobile devices such as smartphones and tablets, which provide always-on connectivity, information and communication. However, due to severe resource poverty and poor network connectivity, lots of traditional embedded electronic devices with attracting features cannot be incorporated into this computing paradigm conveniently. In this paper, an uneven-dual-core SoC, which integrates a CPU core and a MCU core on a single chip with multiple operating system support, is proposed to realize loosely-coupled multiple heterogeneous device collaboration. A network file system, MRFS (Multi-client Raindrop File System), and FAT-X (File Allocation Table eXtension) are also proposed to provide client-centric cross-device data consistency and virtual file access respectively. Comprehensive mobile services are enabled by offloading appropriate tasks from existing smart mobile devices to involved traditional embedded devices. The SoC is implemented onto a 16.65 mm2silicon with 65 nm CMOS technology. This paper also presents three typical applications to illustrate the universality and huge potential for innovative usage model of the proposed system.

To assist the system architects with fast exploration and performance evaluation of the reconfigurable software/hardware architectures, this paper presents a system-level simulator, named after SimRPU, for the reconfigurable processing unit (RPU), which is the major computing engine in reconfigurable processor. The proposed simulator consists of a simulation kernel, a software compiler, a system profiler providing performance, area and power information for the desired architectures, and a system debugger supporting inspecting and modification of the internal state of the RPU. Object-oriented hierarchical and parameterized architecture modeling techniques are proposed to satisfy the requirements for a fast and comprehensive evaluation. Cycle-accurate simulation mechanisms are developed to improve the accuracy of the profiled performance data. Compared with the traditional register transfer level (RTL) based simulation scheme, the proposed simulator could achieve an average speedup of 18.5× with only 3.5% reduction on performance estimation accuracy. One reconfigurable processor targeted at high-definition multimedia decoding applications (such as H.264, MPEG2, AVS, etc.) is implemented with Taiwan Semiconductor Manufacturing Company 65-nm process using the proposed exploration and design flow. The measured results show that the implemented architecture has obvious advantages in terms of both performance and power consumption than the reference designs in multimedia decoding applications.

The coarse-grained reconfigurable architecture (CGRA) is proven to be energy efficient in several specific domains. In CGRAs, the on-chip memory hierarchy, which contains the context memory and the data memory organizations, should be well considered to achieve appropriate tradeoffs among three aspects: 1) performance; 2) area; and 3) power. In this paper, two techniques called the hierarchical configuration context (HCC) and the lifetime-based data-memory organization (LDO) focusing on the context memory and the data memory organizations are proposed to compress the on-chip memory space and to reduce the reconfiguration time and the data-reference time. In the HCC, the contexts are constructed in a hierarchical fashion to completely eliminate the repetitive portions of the contexts, not only reducing the overall context storage, but also alleviating the context transportation overhead. A fast context-indexing mechanism in the HCC is proposed to achieve fast reconfiguration, as the hierarchically organized contexts can be located and accessed conveniently. In the LDO, the on-chip data are classified into two types, based on the lifetime of data. The short-lifetime data are stored in the first in first out to increase the reuse ratio of memory space automatically, whereas the long-lifetime data are stored in the radom access memory for several time references. The HCC and the LDO are used in a CGRA core called as reconfigurable processing unit (RPU). Two RPUs are integrated in a reconfigurable computing processor (RCP) called as REconfigurable MUlti-media System, High-Performance Processor (REMUS_HPP). Because of the HCC, compared with a traditional nonhierarchical system, the total context storage required in H.264 decoding is reduced by 77%. Because of the LDO, the normalized on-chip data memory size at same performance level in the REMUS_HPP is only 23.8% and 14.8% of those in XPP-III (a high-performance RCP) and ADRES (a low-power RCP). REMUS_HPP is implemented on a 48.9-mm2silicon with TSMC 65-nm technology, using a 200-MHz working frequency to achieve 1920 × 1088 at 30 fps H.264 high-profile decoding. Compared with XPP-III, the performance of the REMUS_HPP is 1.81× boosted, whereas the energy efficiency is 4.75× higher.

The coarse-grained reconfigurable architecture (CGRA) is a promising platform that provides both high performance and high power-efficiency. The compute-intensive portions of an application (e.g. loops) are often mapped onto CGRA for acceleration. To optimize the mapping of loop nests to CGRA, this paper makes two contributions: i) Establishing a precise CGRA performance model and formulating the loop nests mapping as a nonlinear optimization problem based on polyhedral model, ii) Extracting an efficient heuristic loop transformation and mapping algorithm (PolyMAP) to improve mapping performance. Experiment results on most kernels of the PolyBench and real-life applications show that our proposed approach can improve the performance of the kernels by 21% on average, as compared to one of the best existing mapping algorithm, EPIMap. The runtime complexity of PolyMAP is also acceptable.

This paper proposed a THPHP (Task-based Hybrid Parallels and Hybrid Pipelines) scheme to implement multistandard video decoding algorithms, i.e. MPEG-2, H.264 and AVS (Audio Video coding Standard), on a heterogeneous coarsegrained reconfigurable multimedia processor called REMUS (REconfigurable MUltimedia System). Multiple level parallelism and multiple level pipeline techniques are proposed in this scheme. Simulation results show that the video decoder can support H.264 HP (High Profile) 1920×1080@30fps (frame per second) streams, AVS JP (Jizhun Profile) 1920×1080@39fps streams, and MPEG-2 MP (Main Profile) 1920×1080@41fps streams when exploiting a 200MHz working frequency.

A coarse-grained reconfigurable architecture (CGRA) is typically a hybrid architecture, which is composed of a reconfigurable processing unit (RPU) and a host microprocessor. Many compute-intensive applications (e.g., loop nests) are often mapped onto RPUs to speed up the execution of programs. However, communication volume and reconfiguration cost are two bottlenecks for the performance of RPUs. Therefore, loop transformations to break through the bottlenecks and tap the potentials of RPU would be of much significance. In this paper, an automatic loop transformation approach for RPUs is proposed, where the communication cost and reconfiguration cost are under a joint consideration. Experimental results show that our scheme can save up to 22.7% of execution time on average on partial differential equation (PDE) solver kernels compared with the approach just considering communication cost, and performs much better than the loop unrolling scheme on a great majority of loop kernels. Also, run-time complexity is acceptable for the practical cases.

Effective fault tolerant techniques are crucial for a Network-on-Chip (NoC) to achieve reliable communication. In this paper, a novel VLSI architecture employing redundant routers is proposed to enhance the fault tolerance of an NoC. The NoC mesh is divided into blocks of 2×2 routers with a spare router placed in the center. The proposed fault-tolerant architecture, referred to as a quad-spare mesh, can be dynamically reconfigured by changing control signals without altering the underlying topology. This dynamic reconfiguration and its corresponding routing algorithm are demonstrated in detail. Experimental results show that the proposed design achieves significant improvements on reliability compared with those reported in the literature.

To deal with interconnect delay problem, through-silicon-via (TSV) based 3D interconnect is widely used. However, TSV-interconnected 3D chips face problems such as high cost, low yield and large power dissipation. In this paper, we propose a wireless 3D on-chip-network architecture for application-specific SoC design, using inductive-coupling interconnect instead of TSV for inter-layer communication, which cuts down manufacture cost, improves interconnect performance, reduces power consumption and provides larger design space. The key to this design is allocating wireless links in the 3D on-chip network effectively while maintaining signal integrity. We develop a design flow which can fully exploit the design space brought by wireless links and provide flexible tradeoff for user's choice. Experimental results show that our architecture brings great improvement on both performance and power consumption.

Emerging techniques and systems for Wireless Sensor Network (WSN) are developed in the last decade for various application fields. In WSN, the sensor nodes are usually distributed over a large area and are powered by batteries with limited energy, maintaining a long service lifetime for the entire network becomes a challenging task. In this paper, a novel battery aware MAC analytical model is proposed for low duty-cycled WSN. The proposed analytical model takes the characteristics of actual battery into account and targets the optimal sleep interval with a reasonable trade-offs between the energy dissipation on sending the preamble and idle listening. The simulation results demonstrate that the proposed approach can improve the energy efficiency as well as guarantee low latency and high reliability.

How to map IP cores onto NoC architectures is a significant issue (application mapping) in multi-core system design. Many mapping algorithms which aim at optimizing cost metrics(e.g. energy consumption) in the mapping procedure are proposed. Some of those algorithms consider satisfying performance metrics (e.g. latency) constraints. This paper analyses the mechanism leading to performance decreases and furthermore proposes a new approach SPC (satisfy processing capacity) to guarantee performance metrics for cost oriented application mapping algorithms. Using this approach in one of those algorithms, performance metrics constraints are well ensured and cost metrics are hardly affected. In fact, one random experiment shows that performance is enhanced at most 10% even if cost is reduced by 6%.

In this paper, a configuration context reduction method for coarse-grained reconfigurable architecture (CGRA) is proposed. The proposed method exploits the structure correlation of computation tasks that are mapped onto CGRA and reduce the redundancies in configuration context. Experimental results show that the proposed method can averagely reduce the configuration context size up to 57% and speed up the execution up to 28.7%.

Computing-intensive algorithms which occupy most of executing time are always the main bottleneck in real-time or high quality video applications. In this paper, the optimization methods of the computing-intensive decoding algorithms of H.264, including MC (Motion Compensation), Deblocking and IDCT-IQ (Inverse Discrete Cosine Transform-Inverse Quantization), are proposed firstly, and then implemented on the REMUS (REconfigurable MUltimedia System) which is an embedded coarse-grain reconfigurable multimedia system. Tests show that the efficiency of MC is improved by 32.5%, Deblocking by 69% and IDCT-IQ by 88.5% compared with XPP PACT(a commercial reconfigurable processor). Compared with typical ASIC solutions, execution performance of MC and IDCT improved by 49% and 17%, respectively, while that of Deblocking remained about the same.

This paper presents a single chip VLSI architecture of wireless image sensor node, which is constituted by an enhanced embedded 8051 microcontroller, a CMOS camera interface and hardware accelerators. The algorithms and control flows of the IEEE 802.15.4 MAC layer are accelerated by hardware, results in 45% less code size compared with the conventional software stack. An innovated CFA preprocessing algorithm and JPEG-LS compressing method is adopted and implemented by hardware, which has a minimal 46.3dB PSNR, an average compression ratio of about 3.0bit/pixel and an approximately 5fps at 16MHz system clock. Furthermore, low power design and techniques are employed to extend battery life, resulting in 60mW max system power consumption when the SoC is in full working mode (i.e. processor, image processing and wireless communication are active simultaneously) in 0.18μm CMOS process.

This paper proposes a reconfigurable multi-processor SoC for media applications called REMUS (REconfigurable Multi-media System), which consists of 512 processing engines and two ARMs. The processing engines are divided into two dynamic configuration groups, which can be easily tailored and extended. The processing engines, DBIs (Data Buffering Interface, DBI) and context interfaces build up a large throughput computing system with thread parallelism, algorithms parallelism and data parallelism. Different algorithms can be mapped in at the same time. REMUS is suitable for many applications such as media decoding and baseband processing, etc. Simulation results show that the processing capability of REMUS is to support 1920*1088 @30fps videos at 200 MHz in real-time decoding of H.264 high-profile streams.

Real time communication (RTC) has critical quality of service (QoS) requirements, which is much more challenging in wireless multi-hop networks. Traditional measurement-based routing schemes often ignore the interference from the coming RTC traffic itself (i.e. self-traffic), so they can not get an accurate quality estimation of the path to serve the coming RTC traffic. In this paper, we propose a novel prediction-based routing metric, PPTT (Path Predicted Transmission Time), to estimate end-to-end delay of RTC traffics. PPTT is traffic-aware by taking explicit consideration of both self-traffic and neighboring traffics interfering with the RTC flow, and thus offers an accurate prediction of transmission delay. By selecting route with minimal PPTT, the quality of service for the coming RTC flow will be improved, in terms of end-to-end delay and goodput. To evaluate the performance, we implement PPTT scheme and study its performance in a wireless multi-hop test bed consisting of 32 nodes equipped with IEEE 802.11 a/b/g combo cards, and we also conduct extensive simulations with different random topologies in network simulator NS2 for a more comprehensive comparison. Experiment results show that this routing metric outperforms other non prediction-based routing metric such as ETX (Expected Transmission Count) and WCETT (Weighted Cumulative Expected Transmission Time) in terms of delay and goodput in wireless multi-hop networks.

Abstract In this paper, we propose a novel traffic‐aware routing metric for real‐time communications (RTC) in wireless multi‐hop networks. Our routing metric, path predicted transmission time (PPTT), is designed to choose a high‐quality path for RTC flow between a source and a destination. PPTT can serve as both single‐radio and multi‐radio routing metric for RTC flow. RTC has critical quality of service (QoS) requirements in terms of delay, bandwidth and so on. Traditional measurement‐based routing schemes ignore the interference from the coming RTC flow itself (i.e. self‐traffic), so they may choose the inefficient path to serve the coming RTC flow due to the inaccurate quality estimation of the transmission path. PPTT takes explicit consideration of both self‐traffic and neighbouring traffic interfering with the RTC flow, and thus offers an accurate estimation of path transmission delay. Through differentiating the links by the wireless channel/radio they are using, PPTT has the capability to choose a high‐quality path for the coming RTC flow in both single‐radio and multi‐radio networks. To evaluate the performance, we implement PPTT scheme and study its performance in a wireless multi‐hop testbed consisting of 32 nodes equipped with two IEEE 802.11a/b/g combo cards, and we also conduct extensive simulations with different random topologies in network simulator NS‐2 for a more comprehensive comparison. The results of simulation and experiment show that this routing metric outperforms other non‐traffic‐aware one such as expected transmission count (ETX) and weighted cumulative expected transmission time (WCETT) in terms of delay and goodput in both single‐radio and multi‐radio wireless networks. Copyright © 2006 John Wiley & Sons, Ltd.

In ad hoc networks, energy conservation is a very important design issue. Several energy-efficient routing protocols try to address this issue. In this paper, we propose a multipath minimum energy routing mechanism to minimize the overall energy consumption of the network. We model an ad hoc network by a set of nodes and links. Each link is associated with an energy cost function which is a function of the total traffic flowing over this link. We focus on the problem of how to split traffic among multiple paths to minimize the sum of the links' energy cost. We present a primal formulation of the problem and introduce a methodology for solving it. The proposed approach can get optimal traffic allocation and can be easily implemented in a distributed manner. We evaluate its performance by comparing it to some existing energy-efficient routing protocols via simulation. Satisfyingly, our proposal achieves more residual energy and balanced energy consumption.

In this paper, we propose a prediction based adaptive load balancing mechanism for mobile ad hoc networks. This mechanism can cooperate with any kind of multipath source routing protocol. It distributes traffic load among multiple disjoint paths based on the measurement and prediction of network traffic. In order to predict network traffic accurately, we first analyze the traffic data collected in our mobile ad hoc network testbed and find out that ad hoc networks traffic is self-similar. Then, we propose a traffic predictor based on wavelet analysis. The simulation results show that our loading balancing approach can distribute traffic properly and lead to a decrease of traffic load imbalance of multiple paths and network congestion. As a result, it reduces the end-to-end packet delay and packet dropping probability and balances the energy consumption of the network.