While memory-augmented neural networks (MANNs) offer an effective solution for few-shot learning (FSL) by integrating deep neural networks with external memory, the capacity requirements and energy overhead of data movement become enormous due to the large number of support vectors in many-class FSL scenarios. Various in-memory search solutions have emerged to improve the energy efficiency of MANNs. NAND-based multi-bit content addressable memory (MCAM) is a promising option due to its high density and large capacity. Despite its potential, MCAM faces limitations such as a restricted number of word lines, limited quantization levels, and non-ideal effects like varying string currents and bottleneck effects, which lead to significant accuracy drops. To address these issues, we propose several innovative methods. First, the Multi-bit Thermometer Code (MTMC) leverages the extensive capacity of MCAM to enhance vector precision using cumulative encoding rules, thereby mitigating the bottleneck effect. Second, the Asymmetric Vector Similarity Search (AVSS) reduces the precision of the query vector while maintaining that of the support vectors, thereby minimizing the search iterations and improving efficiency in many-class scenarios. Finally, the Hardware-Aware Training (HAT) method optimizes controller training by modeling the hardware characteristics of MCAM, thus enhancing the reliability of the system. Our integrated framework reduces search iterations by up to 32×, and increases overall accuracy by 1.58% to 6.94%.

Vector similarity search (VSS) is crucial in many AI applications, such as few-shot learning (FSL) and approximate nearest-neighbor search (ANNS), but it demands significant memory capacity and incurs substantial energy costs for data transfers during large-scale comparisons. Various in-memory search technologies have been developed to improve energy efficiency, with NAND-based multi-bit content-addressable memory (MCAM) standing out as a promising solution for its high density and large capacity. MCAM can operate in exact-search (ES) mode, supporting only perfect matches with low energy cost, or in approximate-search (AS) mode, enabling flexible VSS. However, AS mode incurs significant energy waste when comparing queries with non-target stored vectors. To address this issue, we propose Hybrid-M, a 3D NAND-based in-memory VSS architecture that integrates both modes into a single hybrid matching process, using ES mode as a filter to reduce redundant searches for AS mode. We apply three techniques to optimize this integration: range encoding for multi-level cells (MLC) to enhance filtering, search voltage shifts to mitigate the impact on AS accuracy and reduce matching currents, and a filtering-aware training method to further improve reliability and energy efficiency. Results show that Hybrid-M achieves comparable accuracy while reducing energy consumption by 67% to 83% compared to MACM-based VSS using only AS mode, across various many-class FSL and ANNS workloads.

The end-cloud synergy with neural networks offers significant promise for enhancing AI-IoT systems. Numerous studies have been conducted to effectively address the dynamic and resource-constrained nature of IoT; however, challenges persist in adaptability to constraints, horizontal integration between technologies, and system-level optimization. In this article, we propose an end-cloud collaborative system (EECS) with an on-demand offloading mechanism for AI-IoT. We enhance three critical areas previously narrowly explored: 1) model pruning to fit into the end device’s constraints; 2) dynamic data transmission between end and cloud; and 3) an effective offloading policy. We pay particular attention to the constraints imposed by multiply-and-add (MAC) operations and bandwidth (BW) at the end, with a focus on image classification tasks. Our contributions are threefold. First, we introduce a two-stage trainable pruning method that can automatically adjust the end model to end constraints and optimize from a system-wide perspective. Second, we propose an adaptive mechanism that accommodates fluctuating BW conditions based on our trainable pruning, integrating static pruning with dynamic inference. Third, we develop a smart offloading policy that enhances decision-making, thereby elevating system-level efficiency. Finally, our EECS shows substantial improvements, achieving$2.6\times $

Recently, in light of the success of quantum computers, research teams have actively developed quantum-inspired computers using classical computing technology. One notable success story is the Ising Machine (IM), which excels in efficiently solving NP-hard combinatorial optimization problems (COPs) in various domains such as finance, drug development, and logistics. However, IMs may encounter the von Neumann bottleneck due to significant data transfers between computing and memory units. To tackle this issue, processing-in-memory (PiM) leveraging resistive random-access memory (ReRAM) has emerged as a potential solution. Nonetheless, existing ReRAM-based IM accelerators face challenges stemming from non-ideal ReRAM devices and the limited flexibility in selecting solver algorithms. To overcome these limitations, we propose ReAIM, a ReRAMbased Adaptive Ising Machine, co-designed with an adaptive parameter search algorithm to dynamically select an appropriate solver algorithm and hardware/software parameters for the target COP. Based on our in-depth analysis, ReAIM accounts for the impact of both software parameters, such as the choice of local search algorithm and the number of spin flips, and hardware characteristics, including ReRAM-induced errors. Its reconfigurable architecture and optimized mapping/scheduling strategies enable efficient execution of the adaptive algorithm across various COPs, even for large-scale problems. Compared to the state-of-the-art SRAM-based IM accelerator, ReAIM achieves a $2.2 \times$ shorter time-to-solution (TTS), showcasing superior hardware performance without compromising solution quality.