Recommendation systems represent one of the largest machine learning applications on the planet -- industry-scale recommendation models are trained with petabytes of data and serve billions of users every day. To utilize the rich user signals in the long user history, these models have been scaled up to unprecedented complexity, up to trillions of floating-point operations (TFLOPs) per example. This scale, coupled with the huge amount of training data, necessitates new storage and training algorithms to efficiently improve the quality of these complex recommendation systems.

Serving deep learning based recommendation models (DLRM) at scale is challenging. Existing approaches rely on dedicated ANN indexing and filtering services on CPUs, suffering from non-negligible costs and missing co-design opportunities. Such inefficiency makes them difficult to support complex model architectures, such as learned similarities and multi-task retrieval. In this paper, we present SilverTorch, a model-based serving system that brings all components into one unified model. It unifies model serving by replacing standalone indexing and filtering services with model layers. We propose a model-based GPU Bloom index for feature filtering and a fused Int8 ANN kernel for nearest neighbor search. Through co-design of the ANN search and feature filtering, we reduce GPU memory usage and eliminate computation. Benefiting from this design, we scale up retrieval by introducing an OverArch scoring layer and a multi-task retrieval with a Value Model to aggregate scores. These advancements improve the retrieval accuracy and enable future studies for serving more complex models. Our evaluation on industry-scale datasets shows that SilverTorch achieves up to 23.7× higher throughput compared to the state-of-the-art approaches. We also demonstrate that SilverTorch's solution is 13.35× more cost-efficient than CPU-based solution while improving accuracy via serving more complex models.

The quality of medical services is improved by sharing electronic medical records (EMRs) across multiple medical institutions via cloud edge. However, EMRs contain private information about patients, and cloud servers are untrustworthy, thus they cannot be shared arbitrarily among senders and receivers. Access control encryption (ACE) is a preferred technique that produces encrypted EMRs and then restricts the capabilities of both senders and receivers to enforce the EMR flow via sanitizers. However, existing cross-domain ACE schemes employ a single sender authority to issue encryption keys for senders, which suffers from single point of failure and encryption key escrow that the sender authority can public EMRs arbitrarily. Moreover, they only support coarse-grained access structures such as AND gates, which is not suitable for flexible EMR sharing among medical institutions. To this end, we propose a cross-domain inner-product ACE (CD-IPACE) scheme that features decentralized encryption key generation and fine-grained access structures. Specifically, we construct CD-IPACE from inner-product encryption, threshold structure-preserving signature instantiated with a distributed key generation protocol, and non-interactive zero-knowledge proof, which prevents individual sender authorities from sending ciphertexts, and also protects both data and receiver privacy. Then, we design a secure EMR flow system in cloud edge named ESFlow based on CD-IPACE, which employs edge nodes as sanitizers to check encrypted EMRs and discard illegal ones. Finally, we demonstrate the security and practicality of ESFlow via formal security analysis and extensive experiments.