Behavioral patterns captured in embeddings learned from interaction data are pivotal across various stages of production recommender systems. However, in the initial retrieval stage, practitioners face an inherent tradeoff between embedding expressiveness and the scalability and latency of serving components, resulting in the need for representations that are both compact and expressive. To address this challenge, we propose a training strategy for learning high-dimensional sparse embedding layers in place of conventional dense ones, balancing efficiency, representational expressiveness, and interpretability. To demonstrate our approach, we modified the production-grade collaborative filtering autoencoder ELSA, achieving up to 10× reduction in embedding size with no loss of recommendation accuracy, and up to 100× reduction with only a 2.5% loss. Moreover, the active embedding dimensions reveal an interpretable inverted-index structure that segments items in a way directly aligned with the model's latent space, thereby enabling integration of segment-level recommendation functionality (e.g., 2D homepage layouts) within the candidate retrieval model itself. Source codes, additional results, as well as a live demo are available at https://github.com/zombak79/compressed\_elsa.

Model Embedding CTR Size per 100M (Compression) Dimension Lift Embeddings SBERT [18] 512 (baseline) 204.8 GB Nomic [14] 768 +4.86% 307.2 GB Nomic (Matryoshka) 64 +1.89% 25.6 GB Nomic (CompresSAE) 4096* +3.44% 25.6 GB *Sparse embeddings with 32 nonzero entries.Figure 1: Comparison of embedding models used for candidate retrieval.We report online recommendation performance on a downstream task, relative to SBERT [18], with anytime-valid 99% confidence intervals.

Figure 1: Overview of SAGEA: The training phase (left) consists of: (a) training the ELSA collaborative autoencoder on individual user interactions to learn dense user embeddings, and (b) training a Top-𝑘 sparse autoencoder (SAE) to reconstruct these embeddings via a disentangled, sparse representation.The disentangled representation can be understood as a dictionary of activated concepts (such as preference for "Old music", "Love", "Dark humor", etc., as highlighted in the example).During inference (right), individual group members are encoded into sparse representations via the ELSA encoder E 𝐸𝐿𝑆𝐴 extended with the SAE encoder E 𝑆𝐴𝐸 .These are aggregated using a fairness-preserving strategy to form a group-level sparse embedding.The SAE decoder (D 𝑆𝐴𝐸 ) transforms this back into a dense embedding, which is then decoded by the ELSA decoder (D 𝐸𝐿𝑆𝐴 ) to generate group-wise relevance scores.

We derive a novel graph-based interpretation of linear autoencoder models easer, slim, and their approximate variants. Contrary to popular belief, we reveal that the weights of these models should not be interpreted as dichotomic item similarity but merely as its magnitude. Consequently, we propose a simple modification that considerably improves retrieval ability in sparse domains and yields interpretable inference with negative inputs, as demonstrated by both offline and online experiments. Experiment codes and extended results are available at https://osf.io/bjmuv/.

In the field of recommender systems, shallow autoencoders have recently gained significant attention. One of the most highly acclaimed shallow autoencoders is easer, favored for its competitive recommendation accuracy and simultaneous simplicity. However, the poor scalability of easer (both in time and especially in memory) severely restricts its use in production environments with vast item sets. In this paper, we propose a hyperefficient factorization technique for sparse approximate inversion of the data-Gram matrix used in easer. The resulting autoencoder, sansa, is an end-to-end sparse solution with prescribable density and almost arbitrarily low memory requirements — even for training. As such, sansa allows us to effortlessly scale the concept of easer to millions of items and beyond.