The r -index (Gagie et al., JACM 2020) represented a breakthrough in compressed indexing of repetitive text collections, outperforming its alternatives by orders of magnitude in query time. Its space usage, \(\mathcal{O}(r)\) where \( r \) is the number of runs in the Burrows–Wheeler Transform of the text, is however higher than Lempel–Ziv and grammar-based indexes and makes it uninteresting in various real-life scenarios of milder repetitiveness. In this article, we introduce the sr -index , a variant that limits a large fraction of the space to \({\mathcal{O}}(\min(r,n/s))\) for a text of length \( n \) and a given parameter \( s \) , at the expense of multiplying by \( s \) the time per occurrence reported. The sr -index is obtained by carefully subsampling the text positions indexed by the r -index , in a way that we prove is still able to support pattern matching with guaranteed performance. Our experiments demonstrate that the theoretical analysis falls short in describing the practical advantages of the sr -index , because it performs much better on real texts than on synthetic ones: the sr -index retains the performance of the r -index while using 1.5–4.0 times less space, sharply outperforming virtually every other compressed index on repetitive texts in both time and space. Only a particular Lempel–Ziv-based index uses less space—about half—than the sr -index , but it is an order of magnitude slower. Our second contribution are the r -csa and sr -csa indexes. Just like the r -index adapts the well-known FM-Index to repetitive texts, the r -csa adapts Sadakane’s Compressed Suffix Array (CSA) to this case. We show that the principles used on the r -index turn out to fit naturally and efficiently in the CSA framework. The sr -csa is the corresponding subsampled version of the r -csa . While the CSA performs better than the FM-Index on classic texts with alphabets larger than DNA, our experiments show that the sr -csa outperforms the sr -index on repetitive texts not only over those larger alphabets, but on some DNA texts as well. Overall, our new subsampled indexes sweep the table of the existing indexes for highly repetitive text collection, by combining the exceptional speed of the r -index with drastically reduced storage use.

Compressed full-text indexes are one of the main success stories of bioinformatics data structures but even they struggle to handle some DNA readsets. This may seem surprising since, at least when dealing with short reads from the same individual, the readset will be highly repetitive and, thus, highly compressible. If we are not careful, however, this advantage can be more than offset by two disadvantages: first, since most base pairs are included in at least tens reads each, the uncompressed readset is likely to be at least an order of magnitude larger than the individual’s uncompressed genome; second, these indexes usually pay some space overhead for each string they store, and the total overhead can be substantial when dealing with millions of reads. The most successful compressed full-text indexes for readsets so far are based on the Extended Burrows-Wheeler Transform (EBWT) and use a sorting heuristic to try to reduce the space overhead per read, but they still treat the reads as separate strings and thus may not take full advantage of the readset’s structure. For example, if we have already assembled an individual’s genome from the readset, then we can usually use it to compress the readset well: e.g., we store the gap-coded list of reads' starting positions; we store the list of their lengths, which is often highly compressible; and we store information about the sequencing errors, which are rare with short reads. There is nowhere, however, where we can plug an assembled genome into the EBWT. In this paper we show how to use one or more assembled or partially assembled genome as the basis for a compressed full-text index of its readset. Specifically, we build a labelled tree by taking the assembled genome as a trunk and grafting onto it the reads that align to it, at the starting positions of their alignments. Next, we compute the eXtended Burrows-Wheeler Transform (XBWT) of the resulting labelled tree and build a compressed full-text index on that. Although this index can occasionally return false positives, it is usually much more compact than the alternatives. Following the established practice for datasets with many repetitions, we compare different full-text indices by looking at the number of runs in the transformed strings. For a human Chr19 readset our preliminary experiments show that eliminating separators characters from the EBWT reduces the number of runs by 19%, from 220 million to 178 million, and using the XBWT reduces it by a further 15%, to 150 million.