Recurrent Neural Networks (RNNs) are widely used in Natural Language Processing (NLP) applications as they inherently capture contextual information across spatial and temporal dimensions. Compared to other classes of neural networks, RNNs have more weight parameters as they primarily consist of fully connected layers. Recently, several techniques such as weight pruning, zero-skipping, and block circulant compression have been introduced to reduce the storage and access requirements of RNN weight parameters. In this work, we present a ReRAM crossbar based processing-in-memory (PIM) architecture with systolic dataflow incorporating block circulant compression for RNNs. The block circulant compression decomposes the operations in a fully connected layer into a series of Fourier transforms and point-wise operations resulting in reduced space and computational complexity. We formulate the Fourier transform and point-wise operations into in-situ multiply-and-accumulate (MAC) operations mapped to ReRAM crossbars for high energy efficiency and throughput. We also incorporate systolic dataflow for communication within the crossbar arrays, in contrast to broadcast and multicast communications, to further improve energy efficiency. The proposed architecture achieves average improvements in compute efficiency of 44× and 17× over a custom FPGA architecture and conventional crossbar based architecture implementations, respectively.

Graph analytics applications are ubiquitous in this era of a connected world. These applications have very low compute to byte-transferred ratios and exhibit poor locality, which limits their computational efficiency on general purpose computing systems. Conventional hardware accelerators employ custom dataflow and memory hierarchy organization to overcome these challenges. Processing-in-memory (PIM) accelerators leverage massively parallel compute capable memory arrays to perform the in-situ operations on graph data or employ custom compute elements near the memory to leverage larger internal bandwidths. In this work, we present GaaS-X, a graph analytics accelerator that inherently supports the sparse graph data representations using an in-situ compute-enabled crossbar memory architectures. We alleviate the overheads of redundant writes, sparse to dense conversions, and redundant computations on the invalid edges that are present in the state of the art crossbar-based PIM accelerators. GaaS-X achieves 7.7× and 2.4× performance and 22× and 5.7×, energy savings, respectively, over two state-of-the-art crossbar accelerators and offers orders of magnitude improvements over GPU and CPU solutions.

In conjugate gradient method to solve partial differential equations, matrix vector operations are required in each iteration; these matrix vector operations can be implemented as 5 point finite difference stencil operations on the grid without explicitely constructing the matrix. We show that a pipelined and superscalar design using high level synthesis (HLS) leads to a significant reduction in latencies for two variants of conjugate gradient methods considered. When comparing these two, we show that the second method has roughly two times lower latency than the former given the same degree of superscalarity. These reductions in latencies for the newer variant of CG is due to parallel implementations of stencil operation on subdomains of the grid, and due to overlap of these stencil operations with dot product operations. In a superscalar design for the stencil operation, the computational domain needs to be partitioned, and boundary data needs to be copied, which requires padding. We propose a novel traversal of the grid for 2D domain decomposition that leads to reduction in latency cost involved with padding. The FPGA implementation of CG is roughly 7 times faster than state-of- the-art sequential implementation, and roughly 4 times faster than state-of-the-art CUDA library parallel implementation for the linear system of dimension 10000 x 10000.