Shallow cuttings are a fundamental tool in computational geometry and spatial databases for solving offline and online range searching problems. For a set P of N points in 3-D, at SODA'14, Afshani and Tsakalidis designed an optimal O(N log₂N) time algorithm that constructs shallow cuttings for 3-D dominance ranges in internal memory. Even though shallow cuttings are used in the I/O-model to design space and query efficient range searching data structures, an efficient construction of them is not known till now. In this paper, we design an optimal-cost algorithm to construct shallow cuttings for 3-D dominance ranges. The number of I/Os performed by the algorithm is O (N/B log_{M/B}(N/B)), where B is the block size and M is the memory size. As two applications of the optimal-cost construction algorithm, we design fast algorithms for offline 3-D dominance reporting and offline 3-D approximate dominance counting. We believe that our algorithm will find further applications in offline 3-D range searching problems and in improving construction cost of data structures for 3-D range searching problems.

Given a simple polygon P with n vertices, we consider the problem of constructing a data structure for visibility queries: for any query point q ∈ P, compute the visibility polygon of q in P. To obtain O(log n + k) query time, where k is the size of the visibility polygon of q, the previous best result requires O(n³) space. In this paper, we propose a new data structure that uses O(n^{2+ε}) space, for any ε > 0, while achieving the same query time. If only O(n²) space is available, the best known result provides O(log² n + k) query time. We improve this to O(log n log log n + k) time. When restricted to o(n²) space, the only previously known approach, aside from the O(n)-time algorithm that computes the visibility polygon without preprocessing, is an O(n)-space data structure that supports O(k log n)-time queries. We construct a data structure using O(n log n) space that answers visibility queries in O(n^{1/2+ε} + k) time. In addition, for the special case in which q lies on the boundary of P, we build a data structure of O(n log n) space supporting O(log² n + k) query time; alternatively, we achieve O(log n + k) query time using O(n^{1+ε}) space. To achieve our results, we propose a new method for decomposing simple polygons, which may be of independent interest.

Inspired by the classical fractional cascading technique, we introduce new techniques to speed up the following type of iterated search in 3D: The input is a graph $\mathbf{G}$ with bounded degree together with a set $H_v$ of 3D hyperplanes associated with every vertex of $v$ of $\mathbf{G}$. The goal is to store the input such that given a query point $q\in \mathbb{R}^3$ and a connected subgraph $\mathbf{H}\subset \mathbf{G}$, we can decide if $q$ is below or above the lower envelope of $H_v$ for every $v\in \mathbf{H}$. We show that using linear space, it is possible to answer queries in roughly $O(\log n + |\mathbf{H}|\sqrt{\log n})$ time which improves trivial bound of $O(|\mathbf{H}|\log n)$ obtained by using planar point location data structures. Our data structure can in fact answer more general queries (it combines with shallow cuttings) and it even works when $\mathbf{H}$ is given one vertex at a time. We show that this has a number of new applications and in particular, we give improved solutions to a set of natural data structure problems that up to our knowledge had not seen any improvements. We believe this is a very surprising result because obtaining similar results for the planar point location problem was known to be impossible.

Let 𝓓 be a collection of D strings of total length n over an alphabet of size σ. We consider the so-called top-k document retrieval problem: given a short string P and an integer k, list the identifiers of k strings in 𝓓 most relevant to P, in decreasing order of relevance. Relevance may be a fixed value associated with the strings where P occurs, or the number of times P occurs in the strings. While RAM-optimal solutions using O (n log n ) bits and O (|P|/logσ n + k ) time exist, solving the problem optimally within space close to O (n log σ ) bits is open.

In the orthogonal range reporting problem we must pre-process a set P of multi-dimensional points, so that for any axis-parallel query rectangle q all points from q ∩ P can be reported efficiently. In this paper we study the query complexity of multi-dimensional orthogonal range reporting in the pointer machine model. We present a data structure that answers four-dimensional orthogonal range reporting queries in almost-optimal time O(log n log log n + k) and uses O(n log4 n) space, where n is the number of points in P and k is the number of points in q ∩ P. This is the first data structure with nearly-linear space usage that achieves almost-optimal query time in 4d. This result can be immediately generalized to d ≥ 4 dimensions: we show that there is a data structure supporting d-dimensional range reporting queries in time O(logd-3 n log log n + k) for any constant d ≥ 4. * The full version of the paper can be accessed at https://arxiv.org/abs/2211.03161

In this paper we present new data structures for two extensively studied variants of the orthogonal range searching problem. First, we describe a data structure that supports two-dimensional orthogonal range minima queries in O(n) space and O(log∊ n) time, where n is the number of points in the data structure and ∊ is an arbitrarily small positive constant. Previously known linear-space solutions for this problem require O(log1+∊ n) time (Chazelle, 1988) or O(log n log log n) time (Farzan et al., 2012). A modification of our data structure uses space O(n log log n) and supports range minima queries in time O(log log n). Both results can be extended to support three-dimensional five-sided reporting queries. Next, we turn to the four-dimensional orthogonal range reporting problem and present a data structure that answers queries in optimal O(log n/log log n + k) time, where k is the number of points in the answer. This is the first data structure that achieves the optimal query time for this problem. Our results are obtained by exploiting the properties of three-dimensional shallow cuttings.

In this paper we describe a fully-dynamic data structure that supports point location queries in a connected planar subdivision with n edges. Our data structure uses O(n) space, answers queries in O(logn) time, and supports updates in O(logn) time. Our solution is based on a data structure for vertical ray shooting queries that supports queries and updates in O(logn) time.

We present a number of new results about range searching for colored (or "categorical") data: 1. For a set of $n$ colored points in three dimensions, we describe randomized data structures with $O(n\mathop{\rm polylog}n)$ space that can report the distinct colors in any query orthogonal range (axis-aligned box) in $O(k\mathop{\rm polyloglog} n)$ expected time, where $k$ is the number of distinct colors in the range, assuming that coordinates are in $\{1,\ldots,n\}$. Previous data structures require $O(\frac{\log n}{\log\log n} + k)$ query time. Our result also implies improvements in higher constant dimensions. 2. Our data structures can be adapted to halfspace ranges in three dimensions (or circular ranges in two dimensions), achieving $O(k\log n)$ expected query time. Previous data structures require $O(k\log^2n)$ query time. 3. For a set of $n$ colored points in two dimensions, we describe a data structure with $O(n\mathop{\rm polylog}n)$ space that can answer colored "type-2" range counting queries: report the number of occurrences of every distinct color in a query orthogonal range. The query time is $O(\frac{\log n}{\log\log n} + k\log\log n)$, where $k$ is the number of distinct colors in the range. Naively performing $k$ uncolored range counting queries would require $O(k\frac{\log n}{\log\log n})$ time. Our data structures are designed using a variety of techniques, including colored variants of randomized incremental construction (which may be of independent interest), colored variants of shallow cuttings, and bit-packing tricks.

In this paper we study the four-dimensional dominance range reporting problem and present data structures with linear or almost-linear space usage. Our results can be also used to answer four-dimensional queries that are bounded on five sides. The first data structure presented in this paper uses linear space and answers queries in $O(\log^{1+\varepsilon}n + k\log^{\varepsilon} n)$ time, where $k$ is the number of reported points, $n$ is the number of points in the data structure, and $\varepsilon$ is an arbitrarily small positive constant. Our second data structure uses $O(n \log^{\varepsilon} n)$ space and answers queries in $O(\log n+k)$ time. These are the first data structures for this problem that use linear (resp. $O(n\log^{\varepsilon} n)$) space and answer queries in poly-logarithmic time. For comparison the fastest previously known linear-space or $O(n\log^{\varepsilon} n)$-space data structure supports queries in $O(n^{\varepsilon} + k)$ time (Bentley and Mauer, 1980). Our results can be generalized to $d\ge 4$ dimensions. For example, we can answer $d$-dimensional dominance range reporting queries in $O(\log\log n (\log n/\log\log n)^{d-3} + k)$ time using $O(n\log^{d-4+\varepsilon}n)$ space. Compared to the fastest previously known result (Chan, 2013), our data structure reduces the space usage by $O(\log n)$ without increasing the query time.

We introduce the first index that can be built in o(n) time for a text of length n, and can also be queried in o(q) time for a pattern of length q. On an alphabet of size σ, our index uses O(n log σ) bits, is built in O(n log σ / √{log n}) deterministic time, and computes the number of occurrences of the pattern in time O(q/log_σ n + log n log_σ n). Each such occurrence can then be found in O(log n) time. Other trade-offs between the space usage and the cost of reporting occurrences are also possible.

Under the word RAM model, we design three data structures that can be constructed in $O(n\sqrt{\lg n})$ time over $n$ points in an $n \times n$ grid. The first data structure is an $O(n\lg^ε n)$-word structure supporting orthogonal range reporting in $O(\lg\lg n+k)$ time, where $k$ denotes output size and $ε$ is an arbitrarily small constant. The second is an $O(n\lg\lg n)$-word structure supporting orthogonal range successor in $O(\lg\lg n)$ time, while the third is an $O(n\lg^ε n)$-word structure supporting sorted range reporting in $O(\lg\lg n+k)$ time. The query times of these data structures are optimal when the space costs must be within $O(n\ polylog\ n)$ words. Their exact space bounds match those of the best known results achieving the same query times, and the $O(n\sqrt{\lg n})$ construction time beats the previous bounds on preprocessing. Previously, among 2d range search structures, only the orthogonal range counting structure of Chan and Pǎtraşcu (SODA 2010) and the linear space, $O(\lg^ε n)$ query time structure for orthogonal range successor by Belazzougui and Puglisi (SODA 2016) can be built in the same $O(n\sqrt{\lg n})$ time. Hence our work is the first that achieve the same preprocessing time for optimal orthogonal range reporting and range successor. We also apply our results to improve the construction time of text indexes.

We present the first succinct distance oracles for (unweighted) interval graphs and related classes of graphs, using a novel succinct data structure for ordinal trees that supports the mapping between preorder (i.e., depth-first) ranks and level-order (breadth-first) ranks of nodes in constant time. Our distance oracles for interval graphs also support navigation queries – testing adjacency, computing node degrees, neighborhoods, and shortest paths – all in optimal time. Our technique also yields optimal distance oracles for proper interval graphs (unit-interval graphs) and circular-arc graphs. Our tree data structure supports all operations provided by different approaches in previous work, as well as mapping to and from level-order ranks and retrieving the last (first) internal node before (after) a given node in a level-order traversal, all in constant time.

Range searching on categorical, or “colored”, data has been studied extensively for over two decades. In this paper, we obtain the current best results for perhaps the most basic, and most often studied, version of the geometric problem: colored orthogonal range reporting. Given n colored points in two-dimensional space [U]2, we present a data structure with O(n log3/4+ε n) space, for an arbitrarily small constant ε > 0, so that all k distinct colors in any axis-aligned query rectangle can be reported in (optimal) O (log log U + k) time; this is the first method to break the O(n log n) space barrier. In three dimensions, we present a data structure with O(n log9/5+ε n) space and O(log n/ log log n + k) time; this improves the previous space bound of O(n log4 n).

We introduce a compressed suffix array representation that, on a text T of length n over an alphabet of size \(\sigma \) , can be built in O ( n ) deterministic time, within \(O(n\log \sigma )\) bits of working space, and counts the number of occurrences of any pattern P in T in time \(O(|P| + \log \log _w \sigma )\) on a RAM machine of \(w=\Omega (\log n)\) -bit words. This time is almost optimal for large alphabets ( \(\log \sigma =\Theta (\log n)\) ), and it outperforms all the other compressed indexes that can be built in linear deterministic time, as well as some others. The only faster indexes can be built in linear time only in expectation, or require \(\Theta (n\log n)\) bits. For smaller alphabets, where \(\log \sigma = o(\log n)\) , we show how, by using space proportional to a compressed representation of the text, we can build in linear time an index that counts in time \(O(|P|/\log _\sigma n + \log _\sigma ^\epsilon n)\) for any constant \(\epsilon >0\) . This is almost RAM-optimal in the typical case where \(w=\Theta (\log n)\) .

In this paper we describe a fully-dynamic data structure for the planar point location problem in the external memory model. Our data structure supports queries in O(log_B n(log log_B n)^3)) I/Os and updates in O(log_B n(log log_B n)^2)) amortized I/Os, where n is the number of segments in the subdivision and B is the block size. This is the first dynamic data structure with almost-optimal query cost. For comparison all previously known results for this problem require O(log_B^2 n) I/Os to answer queries. Our result almost matches the best known upper bound in the internal-memory model.

The Burrows-Wheeler Transform (BWT) has become an essential tool for compressed text indexing. Computing it efficiently and within little space is essential for the practicality of the indexes that build on it. A recent algorithm (Munro, Navarro & Nekrich, SODA 2017) computes the BWT in O(n) time using O(nlgσ) bits of space for a text of length n over an alphabet of size σ. The result is of theoretical nature and its practicality is far from obvious. In this paper we engineer their solution and show that, while a basic implementation is slow in practice, the algorithm is amenable to parallelization. For a wide range of alphabet sizes, our resulting implementation outperforms all the compact constructions in the space/time tradeoff map. On the smallest alphabets we are outperformed in time, but nevertheless achieve the least space within reasonable time. For example, in DNA sequences, the most widely used application of BWTs, our construction uses 4.84 bits per base and builds the BWT at a rate of 2.13 megabases per second, whereas the closest previous alternative uses around 7.09 bits per base and runs at 4.17 megabases per second.

In this paper, we consider a variant of the color range reporting problem called color reporting with frequencies. Our goal is to pre-process a set of colored points into a data structure, so that given a query range Q, we can report all colors that appear in Q, along with their respective frequencies. In other words, for each reported color, we also output the number of times it occurs in Q. We describe an external-memory data structure that uses O(N(1+log^2D/log N)) words and answers one-dimensional queries in O(1 +K/B) I/Os, where N is the total number of points in the data structure, D is the total number of colors in the data structure, K is the number of reported colors, and B is the block size. Next we turn to an approximate version of this problem: report all colors sigma that appear in the query range; for every reported color, we provide a constant-factor approximation on its frequency. We consider color reporting with approximate frequencies in two dimensions. Our data structure uses O(N) space and answers two-dimensional queries in O(log_B N +log^*B + K/B) I/Os in the special case when the query range is bounded on two sides. As a corollary, we can also answer one-dimensional approximate queries within the same time and space bounds.

For any $ε\in (0,1)$, a $(1+ε)$-approximate range mode query asks for the position of an element whose frequency in the query range is at most a factor $(1+ε)$ smaller than the true mode. For this problem, we design an $O(n/ε)$ bit data structure supporting queries in $O(\lg(1/ε))$ time. This is an encoding data structure which does not require access to the input sequence; we prove the space cost is asymptotically optimal for constant $ε$. Our solution improves the previous best result of Greve et al. (Cell Probe Lower Bounds and Approximations for Range Mode, ICALP'10) by reducing the space cost by a factor of $\lg n$ while achieving the same query time. We also design an $O(n)$-word dynamic data structure that answers queries in $O(\lg n /\lg\lg n)$ time and supports insertions and deletions in $O(\lg n)$ time, for any constant $ε\in (0,1)$. This is the first result on dynamic approximate range mode; it can also be used to obtain the first static data structure for approximate 3-sided range mode queries in two dimensions. We also consider approximate range selection. For any $α\in (0,1/2)$, an $α$-approximate range selection query asks for the position of an element whose rank in the query range is in $[k - αs, k + αs]$, where $k$ is a rank given by the query and $s$ is the size of the query range. When $α$ is a constant, we design an $O(n)$-bit encoding data structure that can answer queries in constant time and prove this space cost is asymptotically optimal. The previous best result by Krizanc et al. (Range Mode and Range Median Queries on Lists and Trees, Nordic Journal of Computing, 2005) uses $O(n\lg n)$ bits, or $O(n)$ words, to achieve constant approximation for range median only. Thus we not only improve the space cost, but also provide support for any arbitrary $k$ given at query time.

An optimal binary search tree for an access sequence on elements is a static tree that minimizes the total search cost. Constructing perfectly optimal binary search trees is expensive so the most efficient algorithms construct almost optimal search trees. There exists a long literature of constructing almost optimal search trees dynamically, i.e., when the access pattern is not known in advance. All of these trees, e.g., splay trees and treaps, provide a multiplicative approximation to the optimal search cost. In this paper we show how to maintain an almost optimal weighted binary search tree under access operations and insertions of new elements where the approximation is an additive constant. More technically, we maintain a tree in which the depth of the leaf holding an element $e_i$ does not exceed $\min(\log(W/w_i),\log n)+O(1)$ where $w_i$ is the number of times $e_i$ was accessed and $W$ is the total length of the access sequence. Our techniques can also be used to encode a sequence of $m$ symbols with a dynamic alphabetic code in $O(m)$ time so that the encoding length is bounded by $m(H+O(1))$, where $H$ is the entropy of the sequence. This is the first efficient algorithm for adaptive alphabetic coding that runs in constant time per symbol.

In this work, we present a collection of new results on two fundamental problems in geometric data structures: orthogonal point location and rectangle stabbing. -We give the first linear-space data structure that supports 3-d point location queries on $n$ disjoint axis-aligned boxes with optimal $O\left( \log n\right)$ query time in the (arithmetic) pointer machine model. This improves the previous $O\left( \log^{3/2} n \right)$ bound of Rahul [SODA 2015]. We similarly obtain the first linear-space data structure in the I/O model with optimal query cost, and also the first linear-space data structure in the word RAM model with sub-logarithmic query time. -We give the first linear-space data structure that supports 3-d $4$-sided and $5$-sided rectangle stabbing queries in optimal $O(\log_wn+k)$ time in the word RAM model. We similarly obtain the first optimal data structure for the closely related problem of 2-d top-$k$ rectangle stabbing in the word RAM model, and also improved results for 3-d 6-sided rectangle stabbing. For point location, our solution is simpler than previous methods, and is based on an interesting variant of the van Emde Boas recursion, applied in a round-robin fashion over the dimensions, combined with bit-packing techniques. For rectangle stabbing, our solution is a variant of Alstrup, Brodal, and Rauhe's grid-based recursive technique (FOCS 2000), combined with a number of new ideas.

We describe a fully dynamic linear-space data structure for point location in connected planar subdivisions, or more generally vertical ray shooting among nonintersecting line segments, that supports queries in $O(\log n(\log\log n)^2)$ time and updates in $O(\log n\log\log n)$ time. This is the first data structure that achieves close to logarithmic query and update time simultaneously, ignoring $\log\log n$ factors. We further show how to reduce the query time to $O(\log n\log\log n)$ in the RAM model with randomization. Alternatively, the query time can be lowered to $O(\log n)$ if the update time is increased to $O(\log^{1+\varepsilon}n)$ for any constant $\varepsilon>0$, or vice versa.

In this paper we study succinct data structures for one-dimensional color reporting and color counting problems. We are given a set of n points with integer coordinates in the range [1,m] and every point is assigned a color from the set {1,...\sigma}. A color reporting query asks for the list of distinct colors that occur in a query interval [a,b] and a color counting query asks for the number of distinct colors in [a,b]. We describe a succinct data structure that answers approximate color counting queries in O(1) time and uses \mathcal{B}(n,m) + O(n) + o(\mathcal{B}(n,m)) bits, where \mathcal{B}(n,m) is the minimum number of bits required to represent an arbitrary set of size n from a universe of m elements. Thus we show, somewhat counterintuitively, that it is not necessary to store colors of points in order to answer approximate color counting queries. In the special case when points are in the rank space (i.e., when n=m), our data structure needs only O(n) bits. Also, we show that \Omega(n) bits are necessary in that case. Then we turn to succinct data structures for color reporting. We describe a data structure that uses \mathcal{B}(n,m) + nH_d(S) + o(\mathcal{B}(n,m)) + o(n\lg\sigma) bits and answers queries in O(k+1) time, where k is the number of colors in the answer, and nH_d(S) (d=\log_\sigma n) is the d-th order empirical entropy of the color sequence. Finally, we consider succinct color reporting under restricted updates. Our dynamic data structure uses nH_d(S)+o(n\lg\sigma) bits and supports queries in O(k+1) time.

We show that the compressed suffix array and the compressed suffix tree of a string T can be built in O(n) deterministic time using O(n log σ) bits of space, where n is the string length and σ is the alphabet size. Previously described deterministic algorithms either run in time that depends on the alphabet size or need ω(n log σ) bits of working space. Our result has immediate applications to other problems, such as yielding the first deterministic linear-time LZ77 and LZ78 parsing algorithms that use O(n log σ) bits.

Let $\mathcal D$ be a collection of $D$ documents, which are strings over an alphabet of size $\sigma$, of total length $n$. We describe a data structure that uses linear space and reports $k$ most relevant documents that contain a query pattern $P$, which is a string of length $p$ packed in $p/\log_\sigma n$ words, in time $O(p/\log_\sigma n+k)$. This is optimal in the RAM model in the general case where $\log D = \Theta(\log n)$, and involves a novel RAM-optimal suffix tree search. Our construction supports an ample set of important relevance measures, such as the number of times $P$ appears in a document (called term frequency), a fixed document importance, and the minimal distance between two occurrences of $P$ in a document. When $\log D = o(\log n)$, we show how to reduce the space of the data structure from $O(n\log n)$ to $O(n(\log\sigma+\log D+\log\log n))$ bits, and to $O(n(\log\sigma+\log D))$ bits in the case of the popular term frequency measure of relevance, at the price of an additive term $O(\log^\varepsilon_\sigma n)$ in the query time, for any constant $\varepsilon>0$. We also consider the dynamic scenario, where documents can be inserted and deleted from the collection. We obtain linear space and query time $O(p(\log\log n)^2/\log_\sigma n+\log n + k\log\log k)$, whereas insertions and deletions require $O(\log^{1+\varepsilon} n)$ time per symbol, for any constant $\varepsilon>0$. Finally, we consider an extended static scenario where an extra parameter $\mathtt{par}(P,d)$ is defined, and the query must retrieve only documents $d$ such that $\mathtt{par}(P,d)\in [\tau_1,\tau_2]$, where this range is specified at query time. We solve these queries using linear space and $O(p/\log_\sigma n + \log^{1+\varepsilon} n + k\log^\varepsilon n)$ time, for any constant $\varepsilon>0$. Our technique is to translate these top-$k$ problems into multidimensional geometric search problems. As a bonus, we describe some improvements to those problems.

In this paper we present some practical modifications of the higher-order entropy-compressed text indexing method of Foschini et al. [6] based upon the Burrows-Wheeler transform and the FM-index. Our method, called FM-Adaptive, applies a wavelet tree to the entire BWT. It partitions each bit vector of nodes in the wavelet tree into blocks and applies the hybrid encoding along with run-length Gamma code rather than the fixed-length code of [14] to each block while explores data-aware compression. FM-Adaptive retains the theoretical performance of previous work and introduces some improvements in practice. At the same time, broad experiments indicate that our index achieves superior performance, especially in terms of compression, in comparison to the state-of-the-art indexing techniques. The source code is available online.

We describe a fully dynamic linear-space data structure for point location in connected planar subdivisions, or more generally vertical ray shooting among non-intersecting line segments, that supports queries in O(log n(log log n)2) time and updates in O(log nlog log n) time. This is the first data structure that achieves close to logarithmic query and update time simultaneously, ignoring log logn factors. We further show how to reduce the query time to O(logn log log n) in the RAM model with randomization. Alternatively, the query time can be lowered to O(log n) if the update time is increased to O(log1+εn) for any constant ε > 0, or vice versa.

In the dynamic indexing problem, we must maintain a changing collection of text documents so that we can efficiently support insertions, deletions, and pattern matching queries. We are especially interested in developing efficient data structures that store and query the documents in compressed form. All previous compressed solutions to this problem rely on answering rank and select queries on a dynamic sequence of symbols. Because of the lower bound in [Fredman and Saks, 1989], answering rank queries presents a bottleneck in compressed dynamic indexing. In this paper we show how this lower bound can be circumvented using our new framework. We demonstrate that the gap between static and dynamic variants of the indexing problem can be almost closed. Our method is based on a novel framework for adding dynamism to static compressed data structures. Our framework also applies more generally to dynamizing other problems. We show, for example, how our framework can be applied to develop compressed representations of dynamic graphs and binary relations.

In recent years, there has been an increasing interest in planted (l, d) motif search (PMS) with applications to discovering significant segments in biological sequences. However, there has been little discussion about PMS over large alphabets. This paper focuses on motif stem search (MSS), which is recently introduced to search motifs on large-alphabet inputs. A motif stem is an l-length string with some wildcards. The goal of the MSS problem is to find a set of stems that represents a superset of all (l , d) motifs present in the input sequences, and the superset is expected to be as small as possible. The three main contributions of this paper are as follows: (1) We build motif stem representation more precisely by using regular expressions. (2) We give a method for generating all possible motif stems without redundant wildcards. (3) We propose an efficient exact algorithm, called StemFinder, for solving the MSS problem. Compared with the previous MSS algorithms, StemFinder runs much faster and reports fewer stems which represent a smaller superset of all (l, d) motifs. StemFinder is freely available at http://sites.google.com/site/feqond/stemfinder.

Most of the attention in statistical compression is given to the space used by the compressed sequence, a problem completely solved with optimal prefix codes. However, in many applications, the storage space used to represent the prefix code itself can be an issue. In this paper, we introduce and compare several techniques to store prefix codes. Let N be the sequence length and n be the alphabet size. Then, a naive storage of an optimal prefix code uses O(n log n) bits. Our first technique shows how to use O(n log log(N/n)) bits to store the optimal prefix code. Then, we introduce an approximate technique that, for any 01, O(n1/clog n) bits to store a prefix code with an average codeword length at most c times the minimum. In all cases, our data structures allow encoding and decoding of any symbol in O(1) time. We experimentally compare our new techniques with the state of the art, showing that we achieve sixfold-to-eightfold space reductions, at the price of a slower encoding (2.5-8 times slower) and decoding (12-24 times slower). The approximations further reduce this space and improve the time significantly, up to recovering the speed of classical implementations, for a moderate penalty in the average code length. As a byproduct, we compare various heuristic, approximate, and optimal algorithms to generate length-restricted codes, showing that the optimal ones are clearly superior and practical enough to be implemented.

In this paper we develop a simple and practical text indexing scheme for compressed suffix arrays (CSA). For a text of n characters, our CSA can be constructed in linear time and needs 2nHk+ n + o(n) bits of space for any k ≤ clogσn - 1 and any constant ckdenotes the kth order entropy. We compare the performance of our method with two established compressed indexing methods, the FM-index and the Sad-CSA. Experiments on the Canterbury Corpus and the Pizza&Chili Corpus show significant advantages of our algorithm over two other indexes in terms of compression and query time. Our storage scheme achieves better performance on all types of data present in these two corpora, except for evenly distributed data, such as DNA. The source code for our CSA is available online.

Given an array A[1...n] of n distinct elements from the set {1, 2, ..., n} a range maximum query RMQ(a, b) returns the highest element in A[a...b] along with its position. In this paper, we study a generalization of this classical problem called Categorical Range Maxima Query (CRMQ) problem, in which each element A[i] in the array has an associated category (color) given by C[i] ∈ [σ]. A query then asks to report each distinct color c appearing in C[a...b] along with the highest element (and its position) in A[a...b] with color c. Let pc denote the position of the highest element in A[a...b] with color c. We investigate two variants of this problem: a threshold version and a top-k version. In threshold version, we only need to output the colors with A[pc] more than the input threshold τ, whereas top-k variant asks for k colors with the highest A[pc] values. In the word RAM model, we achieve linear space structure along with O(k) query time, that can report colors in sorted order of A[•]. In external memory, we present a data structure that answers queries in optimal O(1+k/B) I/O's using almost-linear O(n log* n) space, as well as a linear space data structure with O(log* n + k/B) query I/Os. Here k represents the output size, log* n is the iterated logarithm of n and B is the block size. CRMQ has applications to document retrieval and categorical range reporting -- giving a one-shot framework to obtain improved results in both these problems. Our results for CRMQ not only improve the existing best known results for three-sided categorical range reporting but also overcome the hurdle of maintaining color uniqueness in the output set.

In this paper we describe compressed indexes that support pattern matching queries for strings with wildcards. For a constant size alphabet our data structure uses O(n.log^e(n)) bits for any e>0 and reports all occ occurrences of a wildcard string in O(m+s^g.M(n)+occ) time, where M(n)=o(log(log(log(n)))), s is the alphabet size, m is the number of alphabet symbols and g is the number of wildcard symbols in the query string. We also present an O(n)-bit index with O((m+s^g+occ).log^e(n)) query time and an O(n{log(log(n))}^2)-bit index with O((m+s^g+occ).log(log(n))) query time. These are the first non-trivial data structures for this problem that need o(n.log(n)) bits of space.

We describe a data structure that supports access, rank, and select queries, as well as symbol insertions and deletions, on a string S[1,n] over alphabet $[1..\sigma]$ in time $O(\log n/\log\log n)$, which is optimal even on binary sequences and in the amortized sense. Our time is worst case for the queries and amortized for the updates. This complexity is better than the best previous ones by a $\Theta(1+\log\sigma/\log\log n)$ factor. We also design a variant where times are worst case, yet rank and updates take $O(\log n)$ time. Our structure uses $nH_0(S)+o(n\log\sigma) + O(\sigma\log n)$ bits, where $H_0(S)$ is the zero-order entropy of $S$. Finally, we pursue various extensions and applications of the result.

In the orthogonal range-searching problem, we store a set of input points S in a data structure; the answer to a query Q is a piece of information about points in Q ∩ S , for example, the list of all points in Q ∩ S or the number of points in Q . In the colored (or categorical) range-searching problem, the set of input points is partitioned into categories; the answer to a query is a piece of information about categories of points in a query range. In this article, we describe several new results for one- and two-dimensional range-searching problems. We obtain an optimal adaptive data structure for counting the number of objects in a three-sided range and for counting categories of objects in a one-dimensional range. We also obtain new results on color range reporting in two dimensions, approximate color counting in one dimension, and some other related problems.

Motif stem search (MSS) is a recent motif search problem to search motifs on large-alphabet inputs. A motif stem is an l-length string with some wildcards. The goal of the MSS problem is to find a set of stems that represents a superset of all (l, d) motifs present in the input sequences. The three main contributions of this paper are as follows: (1) We build motif stem representation more precisely by using regular expressions. (2) We give a new method for generating all possible motif stems. (3) We propose an efficient algorithm, called StemFinder, for solving the MSS problem. Compared with the previous algorithms, StemFinder runs much faster and first solves the (17, 8), (19, 9) and (21, 10) challenging instances on protein sequences; moreover, StemFinder reports fewer stems representing a smaller superset of all (l, d) motifs.

We describe a data structure that supports access, rank and select queries, as well as symbol insertions and deletions, on a string S[1, n] over alphabet [1‥σ] in time O(lg n/lg lg n), which is optimal. The time is worst-case for the queries and amortized for the updates. This complexity is better than the best previous ones by a Θ(1 + lg σ/lg lg n) factor. Our structure uses nH0(S) + O(n + σ(lg σ + lg n)) bits, where H0(S) is the zero-order entropy of S and 0 < ε < 1 is any constant. This space redundancy over nH0(S) is also better, almost always, than that of the best previous dynamic structures, o(n lg σ) + O(σ(lg σ + lg n)). We can also handle general alphabets in optimal time, which has been an open problem in dynamic sequence representations.

In the colored (or categorical) range reporting problem the set of input points is partitioned into categories and stored in a data structure; a query asks for categories of points that belong to the query range. In this paper we study two-dimensional colored range reporting in the external memory model and present I/O-efficient data structures for this problem.

We describe a data structure that uses O(n)-word space and reports k most relevant documents that contain a query pattern P in optimal O(|P | + k) time.Our construction supports an ample set of important relevance measures, such as the frequency of P in a document and the minimal distance between two occurrences of P in a document.We show how to reduce the space of the data structure from O(n log n) to O(n(log σ+log D+log log n)) bits, where σ is the alphabet size and D is the total number of documents.

In this paper we describe a new efficient (in fact optimal) data structure for the top-K color problem. Each element of an array A is assigned a color c with priority p(c). For a query range [a, b] and a value K, we have to report K colors with the highest priorities among all colors that occur in A[a‥b], sorted in reverse order by their priorities. We show that such queries can be answered in O(K) time using an O(N log σ) bits data structure, where N is the number of elements in the array and σ is the number of colors. Thus our data structure is asymptotically optimal with respect to the worst-case query time and space. As an immediate application of our results, we obtain optimal time solutions for several document retrieval problems. The method of the paper could be also of independent interest.

In this paper we study the adaptive prefix coding problem in cases where the size of the input alphabet is large. We present an online prefix coding algorithm that uses O(sigma1/lambda+epsiv) bits of space for any constants epsiv > 0, > 1, and encodes the string of symbols in O(loglog sigma) time per symbol in the worst case, where sigma is the size of the alphabet. The upper bound on the encoding length is lambdanH(s) + (lambda/ ln 2 + 2 + epsiv)n + O(sigma1/lambdalog2sigma) bits.

We present a static data structure for orthogonal range reporting on a U x U x U integer grid. Our data structure supports orthogonal range reporting queries in O((log log n)2 + log log U + k) time and uses O(n log4 n) space, where k is the size of the answer. We describe a static data structure for range reporting in R4 with O(log nlog log n +k) query time and a static data structure for range reporting in Rd, d ≥ 5, with query time O((logd-3 n)/(log log n)d-5 +k). We also describe a semi-dynamic data structure that supports range reporting queries in R3 in O(log nlog log n +k) time and insertions in polylogarithmic time.

The goal of the prefix coding is to assign codewords to elements of the input alphabet A, so that no codeword is a prefix of another one, and the total length of the encoded message S is minimized. In the case of static prefix coding, symbol frequencies are known in advance. In the case of adaptive (or dynamic) prefix coding, every symbol Siis encoded before the next symbol Si+1is read.

In this paper we present a new algorithm for adaptive prefix coding. Our algorithm encodes a text S of m symbols in O(m) time, i.e., in O(1) time per symbol. The length of the encoded string is bounded above by (H + 1)m + O(nlog2m) bits where n is the alphabet size and H is the entropy. This is the first algorithm that works in O(m) time and achieves an almost optimal bound on the encoding length in the worst case. Besides that our algorithm does not depend on the explicit tree traversal

In this paper we present new space e cient dynamic data structures for orthogonal range reporting.The described data structures support planar range reporting queries in time O (log n +k log log(4 n/(k +1)))and space O (n log log n ), or in time O (log n+k )and space O (n log e n )for any e > 0. Both data structures can be constructed in O (n log n )time and support insert and delete operations in amortized time O (log 2 n )and O (log n log log n )respectively. These results match the corresponding upper space bounds of Chazelle [6] for the static case. We also present a dynamic data structure for d -dimensional range reporting with search time O (log d .1 n +k ),update time O (log d n ),and space O (n log d .2+e n )for any e > 0.

Summary form only given. We present a parallel algorithm for the construction of minimum redundancy length-restricted codes that is based on the package-merge algorithm of Larmore and Hirschberg (1990). Our algorithm constructs a length-restricted code in O(L) time with n processors on a CREW PRAM. Thus our algorithm has the same time-processor product as the sequential algorithm of (1990). We also consider the problem of constructing the almost-optimal length-restricted codes.

Summary form only given. This paper presents an efficient algorithm for decoding canonical Huffman codes with lookup tables. Canonical codes are a subclass of Huffman codes, that have a numerical sequence property, i.e., codewords with the same length are binary representations of consecutive integers. In the case of decoding with look-up tables we read a number of bits at each step of the decoding process. We look up the value of the read bit sequence in a table and if this bit sequence contains a codeword, we output the corresponding symbol. Otherwise we proceed with the decoding, using the next look-up table or using some other method.