Accelerating end-to-end inference of transformer-based large language models (LLMs) is a critical component of AI services in datacenters. However, the diverse compute characteristics of LLMs' end-to-end inference present challenges as previously proposed accelerators only address certain operations or stages (e.g., self-attention, generation stage, etc.). To address the unique challenges of accelerating end-to-end inference, we propose IANUS - Integrated Accelerator based on NPU-PIM Unified Memory System. IANUS is a domain-specific system architecture that combines a Neural Processing Unit (NPU) with a Processing-in-Memory (PIM) to leverage both the NPU's high computation throughput and the PIM's high effective memory bandwidth. In particular, IANUS employs a unified main memory system where the PIM memory is used both for PIM operations and for NPU's main memory. The unified main memory system ensures that memory capacity is efficiently utilized and the movement of shared data between NPU and PIM is minimized. However, it introduces new challenges since normal memory accesses and PIM computations cannot be performed simultaneously. Thus, we propose novel PIM Access Scheduling that manages not only the scheduling of normal memory accesses and PIM computations but also workload mapping across the PIM and the NPU. Our detailed simulation evaluations show that IANUS improves the performance of GPT-2 by 6.2× and 3.2×, on average, compared to the NVIDIA A100 GPU and the state-of-the-art accelerator. As a proof-of-concept, we develop a prototype of IANUS with a commercial PIM, NPU, and an FPGA-based PIM controller to demonstrate the feasibility of IANUS.

Recent advancements in brain-computer interface (BCI) technology for steady-state visual evoked potential (SSVEP)-based target identification have shifted from traditional linear algebra (LA) techniques to more sophisticated neural network (NN) approaches, driven by their increased accuracy and consistent performance across different subjects. However, adopting NN-based algorithms has introduced complexities in wearable BCI systems, mainly due to their extensive parameter sets that demand significant memory capacity. Moreover, the computational intensity of these models requires reevaluating hardware architectures. Additionally, the advent of Transformer-based models has further advanced the state of the art, providing even higher accuracy and reduced variability in cross-subject performance, placing greater demands on hardware resources. This paper provides an overview of recent algorithmic progress in SSVEP-based target identification. Also, it proposes considerations for the hardware architecture needed to efficiently support the computation of cutting-edge Transformer-based models in wearable BCIs from the perspective of algorithm-hardware co-design.

Recently, there is increasing demand for energy-efficient signal processing in wearable visual-stimuli-based brain-computer interface (V-BCI) devices. For the better accuracy and the reduced latency of the V-BCI system, the target identification (TI) algorithm that analyzes brain signals is being advanced, and the importance of an energy-efficient accelerating chip that processes various linear algebra operations constituting the TI algorithms is growing. In this paper, we propose a domain-specific reconfigurable array processor (RAP) with a dynamically reconfigurable and scalable array including 5-heterogeneous processing elements (PEs) for the energy-efficient acceleration of basic linear algebra subprograms (BLAS) and matrix decompositions. The system-on-chip (SoC), including the proposed RAP, was fabricated in 130-nm CMOS technology with an area of 16.87-mm2 and measured at 1.0 V 90 MHz. The RAP achieved an information transfer rate (ITR) of 139.9-bits/min and a TI accuracy of 95.4% on a fabricated chip through an optimized TI algorithm and scalable array processing. In addition, the RAP has$16.8\times $higher TI energy efficiency than prior work and achieved an energy efficiency of 2144.2-bits/min/mW for information transfer processing rate with the proposed TI algorithm. The RAP supports a greater variety of linear algebra operations and data sizes with hardware reconfiguration than the prior accelerators.