The embedding operation, pivotal in modern AI applications such as recommendation systems and natural language processing, transforms high-dimensional sparse data into dense vector representations. However, embedding tables are memory-intensive and pose significant challenges in DRAM-based architectures due to their substantial size. This paper introduces Sage, a scalable architecture for embedding operations in CXL-based pooled memory systems. Sage employs advanced caching and prefetching strategies, leveraging an online clustering algorithm to predict embedding table access patterns, and selectively uses Near-Data Processing (NDP) to mitigate the latency associated with CXL memory access. Our comprehensive evaluation demonstrates that Sage significantly enhances throughput and efficiency, providing a cost-effective solution for large-scale AI models. Our experimental results demonstrate that Sage enhances throughput by 2.84 × as compared to conventional memory management systems.

Message Passing Interface (MPI) is a foundational programming model for high-performance computing. MPI libraries traditionally employ network interconnects (e.g., Ethernet and InfiniBand) and network protocols (e.g., TCP and RoCE) with complex software stacks for cross-node communication.This paper presents cMPI, the first work to optimize MPI point-to-point communication (both one-sided and two-sided) using CXL memory sharing on a real CXL platform, transforming cross-node communication into memory transactions and data copies within CXL memory, bypassing traditional network protocols. We analyze performance across various interconnects and find that CXL memory sharing achieves 7.2 × -8.1 × lower latency than TCP-based interconnects deployed in small- and medium-scale clusters. We address challenges of CXL memory sharing for MPI communication, including data object management over the dax representation [50], cache coherence, and atomic operations. Overall, cMPI outperforms TCP over standard Ethernet NIC and high-end SmartNIC by up to 49 × and 72 × in latency and bandwidth, respectively, for small messages.

Near Data Processing (NDP) is a promising solution for data mining/analysis techniques, which extract useful information from big data. In this paper, we propose a novel NDP-enabled memory disaggregation system called Sidekick, based on a type-2 CXL device and enhanced by an automated allocation technique for clustering algorithms. The key enabler of our migration technique is to understand clustering workflows in a unit of the program context, which is the function call stack for functions, threads, and memory allocations to drive the automated decision. The proposed technique relates the migrated computation tasks with a series of function calls and performs GA-based optimization to identify the optimal allocation scenario for a target clustering algorithm. In Scikit-learn, a popular machine learning library, we use the genetic algorithm to find the optimal memory allocation policy and the operation offloading policy using the program context. The results show that the proposed technique increases the clustering performance as compared to the case, which only uses disaggregated memory without NDP cores, by up to 92% in terms of execution time, while reducing the majority of remote CXL memory accesses.