Retrieval-augmented generation pipelines store large volumes of embedding vectors in vector databases for semantic search. In Compute Express Link (CXL)-based tiered memory systems, page-level placement often promotes pages containing both hot and cold vectors, leaving many hot vectors in the long-latency, lower-bandwidth CXL tier. This imbalance increases memory access time, reduces throughput, and worsens tail latency. This paper presents Bauhaus, a software-based memory management technique that increases the share of hot vectors in host memory, thereby improving access efficiency, reducing backend stalls, and lowering tail latency. Bauhaus monitors and reorders embedding vectors at vector-level granularity using Processor Event-Based Sampling (PEBS) and Virtual Memory Area (VMA) metadata, clustering hot vectors into contiguous pages to raise hot-page density and enhance the effectiveness of page promotion. Bauhaus, integrated into the Heterogeneous Memory Software Development Kit (HMSDK) developed and publicly released by SK hynix, is evaluated on a production-grade CXL platform. Across various embedding models, datasets, and CXL-to-host memory ratios, Bauhaus achieves 42.0% higher throughput over the baseline and 15.8% over HMSDK. Bauhaus sustains high RAG performance under constrained host memory capacity, offering a practical solution for large-scale vector database deployments in modern datacenter environments.

K-means clustering is widely used in applications such as classification, recommendation, and image processing for its simplicity and efficiency. While often deployed on servers, it is also used on mobile platforms for tasks like sensor data analysis. However, mobile devices face tight hardware and energy constraints, making efficient execution challenging. Prior parallel K-means approaches still suffer from GPU underutilization due to warp divergence and leave CPUs idle. This paper proposes Kubism, a novel software technique that disassembles and reassembles a K-means clustering algorithm to maximize CPU and GPU resource utilization on mobile platforms. Kubism incorporates several key strategies, including reordering operations to minimize unnecessary work, ensuring balanced workloads across processing units to avoid idle time, dynamically adjusting task execution based on real-time performance metrics, and distributing computation efficiently between the CPU and GPU. These methods synergistically improve performance by reducing idle periods and optimizing the use of hardware resources. In our evaluation on the NVIDIA Jetson Orin AGX platform, Kubism achieves up to a 2.65× speedup in individual clustering iterations and an average 1.23× improvement in overall end-to-end execution time compared to prior work.

DRAM accounts for a large fraction of the total cost of ownership of memory systems in deep learning acceleration systems. To achieve sustainable scalability, tiered memory systems with denser technologies become critical. Prior work has proposed various tiered memory systems, but no matter how a system is designed, data movements between tiers consume substantial energy. In particular, as model sizes and memory capacity demands grow, the data movements between memory tiers become more frequent, posing a challenge that tiered memory systems may reduce deployment costs but suffer from low energy efficiency. If a memory system proactively places data into a tier and timely fetches it from the tier, then the excessive data movement between tiers can be mitigated. We find that a system can statically anticipate such behaviors for all pages and localities. With this insight, we propose a new DNN acceleration system called TM-Training using flash memory. TM-Training capitalizes on the repetitive nature of the same computational patterns during execution, enabling the static establishment of optimal data placement for subsequent operations. Moreover, TM-Training employs a new data-splitting scheme to enable precise memory management. Our evaluation demonstrates that TM-Training reduces inter-tier data traffic by 64% and achieves a 55% higher throughput per watt in training than prior work.