As GPUs and other accelerators become increasingly popular, optimizing I/O operations between on-chip and off-chip memory is increasingly critical. I/O analysis, however, is complex, requiring a deep understanding of application dataflow and memory hierarchy. Developing a practical I/O analysis methodology remains a timely challenge. Self-attention is employed extensively in transformer models, but its quadratic memory complexity poses significant challenges to modern memory systems. In this study, we explore how to use I/O analysis to develop optimal solutions for accelerating exact long-sequence self-attention. We first introduce a novel I/O analysis for tall-and-skinny matrix-matrix multiplication, which captures the dominant data movement behavior of long-sequence self-attention. Guided by systematic I/O analysis, we develop AttenIO, an I/O-driven accelerator for exact long-sequence self-attention with three key optimizations: (1) an analytically derived I/O-optimal tiling and scheduling to minimize I/O operations, (2) fine-grained three-level communication-computation overlapping to hide I/O stalls, and (3) parallel execution patterns for efficient softmax. Our evaluation shows that AttenIO achieves a 1.6×-8.8× speedup over the state-of-the-art solutions. Although AttenIO is designed for self-attention, it also highlights the broader potential of I/O analysis as a principled foundation for guiding high-performance I/O optimizations.

The reconfigurable processing-in-memory (PIM) architecture has garnered significant attention in recent years due to its versatility and ability to overcome storage limitations. However, it faces challenges, such as overly complex mapping and routing caused by the fine granularity of basic logic units, and the inclusion of numerous redundant devices to achieve reconfigurability. To address these issues, we have designed a software-hardware co-design reconfigurable PIM architecture called Re-Meltrix. Its hardware architecture uses an resistive random-access memory array as the foundation, combined with well-designed peripheral circuits. Maintaining a controllable area, it integrates logic, storage, ternary content-address memory, and interconnection modes into a unified tile architecture and implements two-level independent interconnection within and between tiles. This approach achieves a single tile logic capacity multiple times that of the most advanced reconfigurable PIM architectures currently available, thereby resolving mapping and routing difficulties at the hardware level. Our proposed function synthesis, combined with the hardware architecture, specifically optimizes two-level interconnection separation and module segmentation, further reducing interconnection complexity and improving tile usage efficiency. Experiments have demonstrated that our architecture outperforms the state-of-the-art Liquid Silicon by 2.00–$4.31\times $in performance and reduces power consumption by 29%–68%. Compared with the previously published Meltrix, the area has decreased by 15%–35%, with the area and power consumption remaining almost unchanged.

Sparse matrix-matrix multiplication (SpMM) is a critical computational kernel in numerous scientific and machine learning applications. SpMM involves massive irregular memory accesses and poses great challenges to conventional cache-based computer architectures. Recently dedicated SpMM accelerators have been proposed to enhance SpMM performance. However, current SpMM accelerators still face challenges in adapting to varied sparse patterns, fully exploiting inherent parallelism, and optimizing cache performance. To address these issues, we introduce ACES, a novel SpMM accelerator in this study. First, ACES features an adaptive execution flow that dynamically adjusts to diverse sparse patterns. The adaptive execution flow balances parallel computing efficiency and data reuse. Second, ACES incorporates locality-concurrency co-optimizations within the global cache. ACES utilizes a concurrency-aware cache management policy, which considers data locality and concurrency for optimal replacement decisions. Additionally, the integration of a non-blocking buffer with the global cache enhances concurrency and reduces computational stalls. Third, the hardware architecture of ACES is designed to integrate all innovations. The architecture ensures efficient support across the adaptive execution flow, advanced cache optimizations, and fine-grained parallel processing. Our performance evaluation demonstrates that ACES significantly outperforms existing solutions, providing a 2.1× speedup and marking a substantial advancement in SpMM acceleration.

Field-programmable gate arrays (FPGAs) are popular for computational intensive applications and hardware accelerators recently. But they face limitations in memory capacity and its growth, resulting in excessive time spent on data access. The fixed capacity of embedded memory blocks also leads inflexibility and resource waste. Moreover, logic blocks in FPGAs which are insufficient for large-scale applications and fixed memory block positions both lead to high routing overhead. To address these issues, we propose a software-hardware co-designed polymorphic architecture called Meltrix. The hardware architecture, which uses RRAM arrays as the fundamental block, creates a unified fabric that can be reconfigured into logic, storage, and interconnection modes. We achieve multiple times of logic capacity compared with FPGAs' logic blocks and multi-level interconnections inside the tiles, which are used to solve the routing overhead problem in FPGAs. Moreover, the global routing complexity is further reduced by the proposed function synthesis framework, which isolates logic and memory components, synthesizes and maps them to configured tiles of Meltrix. Experiments show that, when comparing with commercial FPGAs and state-out-of-art Liquid-Silicon, Meltrix achieves 1.89-3.14× performance improvement and 2.08-4.17× power reduction in both logic-intensive and memory-intensive applications.

Memristor-based logic-in-memory (LIM) is an emerging technology that enables logic operations within memory, making it a promising solution for data-intensive applications. LIM architectures have different types according to where computations are executed, with each type being suitable for specific design objectives and application domains. However, mapping applications to a single LIM mode restricts the full utilization of different LIM modes. In this paper, we propose LIM-GEN, a data-guided framework for automated generation of heterogeneous LIM architectures. To take advantages of different LIM modes, three LIM modes are combined and used as building blocks to create heterogeneous architectures. Given the data-centric nature and large design space, there is an urgent need of developing new EDA tools for synthesizing such LIM architectures. LIM-GEN includes an automatic hardware synthesis flow, which takes behavior-level descriptions as input to generate application-specific architectures and dataflows. During synthesis, data distribution, task allocation and crossbar mapping are optimized through a design space exploration process. We evaluate LIM-GEN in several data-intensive applications and compare the generated heterogeneous architectures with synthesized architectures with a single LIM mode. The experimental results demonstrate significant improvements in latency, area and power consumption, brought by the heterogeneous architectures generated by LIM-GEN.