Genomics datasets, such as single-cell transcriptomics, are often very large and highly sparse, posing significant challenges for both storage and computation. As the scale of data generation accelerates, efficiently compressing these datasets becomes crucial. Current compression methods, like the popular Compressed Sparse Column (CSC) format, capitalize only on sparsity but overlook other properties like redundancy, which can offer additional opportunities for compression. Genomics data, especially single-cell assays, often exhibit high redundancy within columns, making traditional sparse formats inefficient for in-core computation. In this paper, we present two extensions to CSC: (1) Value-Compressed Sparse Column (VCSC) and (2) Index-and Value-Compressed Sparse Column (IVCSC). VCSC takes advantage of high redundancy within a column to further compress data up 1.9-fold over CSC on real data, without significant negative impact to performance characteristics. IVCSC extends VCSC by compressing index arrays through delta encoding and byte-packing, achieving up to a 4.4-fold decrease in memory usage over CSC on real data. Our benchmarks show that VCSC and IVCSC can be used in compressed form with little added computational cost. These formats represent a step forward in balancing the growing demands of data storage and processing in the era of large-scale genomics.

Large, sparse matrices are common in a variety of applications in scientific computing and machine learning, and are most commonly stored in CSC, CSR, or COO format. Most extensions or adaptations of these formats focus on enabling faster computation through techniques such as blocking. While common, these formats and extensions primarily take advantage of sparsity or nonzero patterns. In some cases, however, sparse data is highly redundant, with relatively few unique nonzero values. We introduce two novel extensions of CSC that aim to exploit redundancy: 1) Value-Compressed Sparse Column (VCSC) and 2) Index- and Value-Compressed Sparse Column (IVCSC). For each column, VCSC stores three arrays: unique values, value counts, and row indices, capitalizing on per-column value-redundancy by storing each unique value in a column once. IVCSC compresses further by also performing index-compression, storing an array of sections, where each section contains a value, then byte width to store row indices for that value, then positive-delta encoded row indices, then a delimiter. We summarize compression performance for VCSC and IVCSC on five varied, real-world datasets representing a wide range of use cases.