Causal inference is an important field in data science and cognitive artificial intelligence. It requires the construction of complex probabilistic models to describe the causal relationships between random variables. Probabilistic models rely on probabilistic programming as a flexible framework. However, the computing speed of probabilistic programming is often hindered by the extensive use of Markov chain Monte Carlo (MCMC) algorithms, even though they are powerful in Bayesian inference. To accelerate MCMC, this work presents PROCA, a programmable MCMC-based probabilistic processing unit architecture. PROCA exploits processing-in-memory function units to generate new samples of Markov chains. PROCA is programmable to execute the computation for arbitrary forms of posterior distribution formulas that software probabilistic programming frameworks support. We develop a novel accept/reject prediction methodology to accelerate the sequential MCMC computation, thereby introducing efficient multi-core pipelining methods. We implement and validate the PROCA architecture with commercial process development kits. The implementation is evaluated based on 9 representative benchmarks, covering PyMC official tutorial probabilistic problems, single-variable probabilistic problems, and real-world causal inference problems. Our comprehensive experiments demonstrate that PROCA achieves a speedup of 172~4871 $\times$ compared to Intel Xeon Gold CPU, $42 \sim 1058 \times$ compared to NVIDIA A100 GPU, and $1.765 \times$ over state-of-the-art MCMC accelerators, respectively. PROCA achieves comparable statistical robustness to the software probabilistic programming frameworks. Compared with state-of-the-art MCMC domain-specific accelerators, our design boosts the energy efficiency by $9.47 \times$.

Large Language Models (LLMs) present substantial computational and memory challenges, particularly within their Feed-Forward Network (FFN) layers. While Computing-in-Memory (CIM) using Memristor offers a path to mitigate data movement bottlenecks, naively scaling existing CIM architectures for LLMs leads to new, severe communication overheads due to their typically isotropic (uniform) design. This paper introduces ASMA, an Anisotropy Scaling Memristor-based Accelerator, specifically co-designed for efficient LLM inference. ASMA identifies and exploits the inherent anisotropic scaling characteristics of LLM FFN layers—categorized into additive, concatenative, and sequential dimensions. Key innovations include a novel Subtile hierarchy for optimized pipelining, a hierarchical anisotropic Network-on-Chip (NoC) featuring distinct channels for broadcast and localized communication, and a co-designed compiler that maps LLM computations by leveraging these architectural specializations. Evaluations using a transaction-level simulator demonstrate that ASMA achieves latency improvement up to 81.7% and energy improvement up to 91.5% compared to conventional hierarchical CIM designs. ASMA offers a new paradigm for designing LLM-specific CIM accelerators by embracing workload anisotropy.

The exponential growth of data exerts great pressure on hardware design for database systems. Memory-centric computing (MCC) architecture, which enable compute capabilities near or inside memory storage, demonstrate great potential in enhancing the efficiency of database operations with higher compute parallelism and reduced data movements. However, existing MCC architecture mainly focus on artificial intelligence (AI) computations and those designed for database applications can only run a limited number of standalone queries such as SORT or JOIN, lacking efficient support for increasingly diverse and complex database workloads. For example, realizing a commercial recommendation engine on database requires supporting workloads including but not limited to vector aggregation, convolution or$N$-hop neighborhoods computing, etc. In this work, we develop a memristor-enabled memory-centric instruction-set architecture (MeMCISA) aiming to efficiently accelerate versatile workloads in modern database systems. MeMCISA features scalable multi-bank memristor-based storage organization with near-memory circuitries and caches in banks. An out-of-order (O0O) scheduling scheme is designed for MeMCISA based on a vector instruction set with four types of instructions (bit-level, element-level, vector-level, and control-level), combining memristor-enabled in-memory computing and near-memory computing to efficiently run workloads with varying computational kernels and data sizes. MeMCISA can support parallel instruction executions across different memristor banks as well as different hardware modules within a memristor bank. Furthermore, we develop data dependency handling mechanisms to support vector dependency scenarios in MeMCISA that do not exist in conventional scalar-based instruction sets. A prototype MeMCISA is implemented based on a 40nm CMOS technology with necessary peripheral hardware including instruction buffer and instruction scheduler. To accurately study MeMCISA performance in real-world database systems, a software-hardware co-designed framework integrating reconfigurable MeMCISA prototype is created that can support end-to-end simulations for database workloads starting from raw software codes. Based on this framework, we evaluate MeMCISA performance with standalone database queries as well as complex database workloads from representative benchmarks including UniBench, neural collaborative filtering (NCF), and ResNet-18. Simulation results demonstrate that MeMCISA achieves up to 41.84 × ~ 1767.70 × in speed compared to general-purpose processors (CPUs/GPUs).

Neural ordinary differential equation (Neural-ODE) outperforms conventional deep neural networks (DNNs) in modeling continuous-time or dynamical systems by adopting numerical ODE integration onto a shallow embedded NN. However, Neural-ODE suffers from slow inference due to the costly iterative stepsize search in numerical integration, especially when using higher-order Runge-Kutta (RK) methods and smaller error tolerance for improved integration accuracy. In this work, we first present algorithmic techniques to speedup RK-based Neural-ODE inference: a two-stage coarse-grained/fine-grained structured pruning method based on top-K sparsification that reduces the overall computations by more than 60% in the embedded NN and a history-based stepsize search method based on past integration steps that reduces the latency for reaching accepted stepsize by up to 77% in RK methods. A reconfigurable hardware architecture is co-designed based on proposed speedup techniques, featuring three processing loops to support programmable embedded NN and a variety of higher-order RK methods. Sparse activation processor with multi-dimensional sorters is designed to exploit structured sparsity in activations. Implemented on a Xilinx Virtex-7 XC7VX690T FPGA and experimented on a variety of datasets, the prototype accelerator using a more complex 3rd-order RK method achieves more than 2.6x speedup compared to the latest Neural-ODE FPGA accelerator using the simplest Euler method. Compared to a software execution on Nvidia A100 GPU, the inference speedup can be up to 18x.

Neural ordinary differential equations (NODEs) provide better modeling performance with smaller amount of model parameters in many tasks by embedding neural networks (NNs) in ordinary differential equations (ODEs). They have been shown to outperform in representing continuous-time data and learning dynamic systems, and are promising for on-device inference and training. However, an edge device is limited by area and energy budget, and real-time operations have a tight latency requirement. State-of-the-art NN accelerators are not optimized for the area- and power-hungry memory storage and access for NODE inference and training, and lack the flexibility to incorporate dynamic latency reduction techniques. We present eNODE by architecture-algorithm co-design to achieve efficient and fast inference and training of NODEs. eNODE adopts compact-size depth-first integration and depth-first training for higher energy efficiency. Through function reuse, packetized processing and a unified NN core design, the efficiency of eNODE’s depth-first processing is further enhanced. We propose algorithm innovations, including slope-adaptive stepsize search and priority processing with early stop, to substantially shorten the latency. A hardware prototype is synthesized in a 28 nm CMOS technology for evaluation and benchmarking. eNODE demonstrates up to 6.59× better energy efficiency, 2.38× higher speed, and better area scalability over a SIMD ASIC baseline.

Nonbinary polar codes defined over Galois field GF(q) have shown improved error-correction performance than binary polar codes using successive-cancellation list (SCL) decoding. However, nonbinary operations are complex and a direct-mapped decoder results in a low throughput, representing difficulties for practical adoptions. In this work, we develop, to the best of our knowledge, the first hardware implementation for nonbinary SCL polar decoding. We present a high-throughput decoder architecture using a split-tree algorithm. The sub-trees are decoded in parallel by smaller sub-decoders with a reconciliation stage to maintain constraints between sub-trees. A skimming algorithm is proposed to reduce the reconciliation complexity for further improved throughput. The split-tree nonbinary SCL (S-NBSCL) polar decoder is prototyped using a 28nm CMOS technology for a (128,64) polar code over GF(256). The decoder delivers 26.1 Mb/s throughput, 11.65 Mb/s/mm2area efficiency and 28.8 nJ/b energy efficiency, outperforming the direct-mapped decoder by 10.3×, 4.4× and 2.7×, respectively, while achieving excellent error-correction performance.

Memristive in-memory sorting has been proposed recently to improve hardware sorting efficiency. Using iterative in-memory min computations, data movements between memory and external processing units can be eliminated for improved latency and energy efficiency. However, the bit-traversal algorithm to search the min requires a large number of column reads on memristive memory. In this work, we propose a column-skipping algorithm with help of a near-memory circuit. Redundant column reads can be skipped based on recorded states for improved latency and hardware efficiency. To enhance the scalability, we develop a multi-bank management that enables column-skipping for dataset stored in different memristive memory banks. Prototype column-skipping sorters are implemented with a 1T1R memristive memory in 40nm CMOS technology. Experimented on a variety of sorting datasets, the length-1024 32-bit column-skipping sorter with state recording of 2 demonstrates up to 4.08× speedup, 3.14× area efficiency and 3.39× energy efficiency, respectively, over the latest memristive in-memory sorting.

Successive-cancellation list (SCL) decoding of polar codes has been adopted for 5G wireless communications. How-ever, the performance of moderate code length is not satisfactory. Heuristic or deep-learning-aided (DL-aided) flip algorithms have been developed to improve the performance by locating error bit positions after SCL decoding. In this work, we propose a new flip algorithm with the help of differentiable neural computer (DNC). New state and action encoding are developed to improve DNC training and inference efficiency. The proposed two-phase method is done by a flip DNC (F-DNC) to rank the most likely flip positions for multi-bit flipping, and if decoding still fails, a flip-validate DNC (FV-DNC) is applied to re-select error bit positions in successive flip decoding trials. Supervised training methods are designed for the two DNCs. Simulation results show that the proposed DNC-aided SCL-Flip (DNC-SCLF) decoding demonstrates up to 0.34 dB coding gain or 54.2% reduction in the average number of decoding attempts over prior work.

Memory-augmented neural networks (MANNs) provide better inference performance in many tasks with the help of an external memory. The recently developed differentiable neural computer (DNC) is a MANN that has been shown to outperform in representing complicated data structures and learning long-term dependencies. DNC’s higher performance is derived from new history-based attention mechanisms in addition to the previously used content-based attention mechanisms. History-based mechanisms require a variety of new compute primitives and state memories, which are not supported by existing neural network (NN) or MANN accelerators. We present HiMA, a tiled, history-based memory access engine with distributed memories in tiles. HiMA incorporates a multi-mode network-on-chip (NoC) to reduce the communication latency and improve scalability. An optimal submatrix-wise memory partition strategy is applied to reduce the amount of NoC traffic; and a two-stage usage sort method leverages distributed tiles to improve computation speed. To make HiMA fundamentally scalable, we create a distributed version of DNC called DNC-D to allow almost all memory operations to be applied to local memories with trainable weighted summation to produce the global memory output. Two approximation techniques, usage skimming and softmax approximation, are proposed to further enhance hardware efficiency. HiMA prototypes are created in RTL and synthesized in a 40nm technology. By simulations, HiMA running DNC and DNC-D demonstrates 6.47 × and 39.1 × higher speed, 22.8 × and 164.3 × better area efficiency, and 6.1 × and 61.2 × better energy efficiency over the state-of-the-art MANN accelerator. Compared to an Nvidia 3080Ti GPU, HiMA demonstrates speedup by up to 437 × and 2,646 × when running DNC and DNC-D, respectively.

Technology scaling continues to improve density, but also reduces the critical charge to hold a logic state, causing devices to become more susceptible to accidental disruptions due to noise and soft errors. Increased process variation adds to the reliability challenge, resulting in over designs and extra timing margins at the cost of power consumption, silicon area and performance degradation. We present efficient in situ error detection techniques to exploit datapath characteristics for monitoring circuit errors: pre-edge checking in non-critical paths without hold time constraints; post-edge checking in critical paths without sacrificing performance; and cross-edge checking in moderate paths for the optimal trade-off. The techniques are all realized using the inherent redundancy within a conventional flip-flop design and do not require any logic or sample duplication as done by most existing methods. The detection-enabled flip-flop is implemented using only 31 transistors as a competitive and low-cost solution.

Nonbinary LDPC codes have shown superior performance, but decoding nonbinary codes is complex, incurring a long latency and a much degraded throughput. We propose a low-latency variable processing node by a skimming algorithm, together with a low-latency extended min-sum check processing node by prefetching and relaxing redundancy control. The processing nodes are jointly designed for an optimal pipeline schedule. This low-latency, high-throughput architecture is applied to a class of high-performance (2, dc)-regular nonbinary LDPC codes constructed based on their binary images. A conflict-free memory is proposed to resolve data hazards caused by the non-structured nature of these codes. A complete (2, 4)-regular, (960, 480) GF(64) nonbinary LDPC decoder is demonstrated on a Xilinx Virtex-5 FPGA. The decoder delivers an excellent error-correcting performance at a 9.76 Mb/s coded throughput, representing a significant improvement of state-of-the-art extended min-sum decoder implementations.