Symmetric distribution properties such as support size, support coverage, entropy, and proximity to uniformity, arise in many applications. Recently, researchers applied different estimators and analysis tools to derive asymptotically sample-optimal approximations for each of these properties. We show that a single, simple, plug-in estimator—profile maximum likelihood (PML)—is sample competitive for all symmetric properties, and in particular is asymptotically sample-optimal for all the above properties.

Motivated by mass-spectrometry protein sequencing, we consider the problem of reconstructing a string from the multisets of its substring composition. We show that all strings of length 7, one less than a prime and one less than twice a prime, can be reconstructed uniquely up to reversal. For all other lengths, we show that unique reconstruction is not always possible and provide sometimes-tight bounds on the largest number of strings with given substring compositions. The lower bounds are derived by combinatorial arguments, while the upper bounds follow from algebraic approaches that lead to precise characterizations of the sets of strings with the same substring compositions in terms of the factorization properties of bivariate polynomials. Using results on the transience of multidimensional random walks, we also provide a reconstruction algorithm that recovers random strings over alphabets of size $\ge4$ from their substring compositions in optimal near-quadratic time. The problem considered is related to the well-known turnpike problem, and its solution may hence shed light on this longstanding open problem as well.

Motivated by the problem of deducing the structure of proteins using mass-spectrometry, we study the reconstruction of a string from the multiset of its substring compositions. We specialize the backtracking algorithm used for the more general turnpike problem for string reconstruction. Employing well known results about transience of random walks in ≥ 3 dimensions, we show that the algorithm reconstructs random strings over alphabet size ≥ 4 with high probability in near-optimal quadratic time.

Late web display advertisements are problematic for both the user experience and the monetary machinery powering the display advertising industry. If a web page is delivered to a user but the ad fails to load in time, the publisher cannot charge the advertiser for that impression. Detecting whether a specific ad will render in time could give the publisher a choice to show that ad or another one. Further, discovering the root causes of latency, possibly over time as new violators emerge, would allow the publisher to address the actionable issues. We propose a system that predicts, at serve time, which ads are likely to have high latency. Once identified we can either ignore those ads, even if they win the auction, or apply a penalty to those ads. In addition, our system collects the daily impression logs, consisting of different types of observations measured at serve time and the associated latency in milliseconds, and analyzes the data to identify the features associated with late ads and likely to be causing the delay.

Over the past decade, several papers, e.g., [1-7] and references therein, have considered universal compression of sources over large alphabets, often using patterns to avoid infinite redundancy. Improving on previous results, we prove tight bounds on expected- and worst-case pattern redundancy, in particular closing a decade-long gap and showing that the worst-case pattern redundancy of i.i.d. distributions is Θ(n1/3)†.

We consider two related problems of estimating properties of a collection of point processes: estimating the multiset of parameters of continuous-time Poisson processes based on their activities over a period of time t, and estimating the multiset of activity probabilities of discrete-time Bernoulli processes based on their activities over n time instants. For both problems, it is sufficient to consider the observations' profile - the multiset of activity counts, regardless of their process identities. We consider the profile maximum likelihood (PML) estimator that finds the parameter multiset maximizing the profile's likelihood, and establish some of its competitive performance guarantees. For Poisson processes, if any estimator approximates the parameter multiset to within distance ε with error probability δ, then PML approximates the multiset to within distance 2ε with error probability at most δ · e4√t·S, where S is the sum of the Poisson parameters, and the same result holds for Bernoulli processes. In particular, for the L1distance metric, we relate the problems to the long-studied distribution-estimation problem and apply recent results to show that the PML estimator has error probability e-(t·S)0.9for Poisson processes whenever the number of processes is k = O(tS log(tS)), and show a similar result for Bernoulli processes. We also show experimental results where the EM algorithm is used to compute the PML.

In the query model of multi-variate function computation, the values of the variables are queried sequentially, in an order that may depend on previously revealed values, until the function's value can be determined. The function's computation query complexity is the lowest expected number of queries required by any query order. Instead of computation, it is often easier to consider verification, where the value of the function is given and the queries aim to verify it. The lowest expected number of queries necessary is the function's verification query complexity. We show that for all symmetric functions of independent binary random variables, the computation and verification complexities coincide. This provides a simple method for finding the query complexity and the optimal query order for computing many functions. We also show that if the symmetry condition is removed, there are functions whose verification complexity is strictly lower than their computation complexity, and mention that the same holds when the independence or binary conditions are removed.

The minimax KL-divergence of any distribution from all distributions in a collection P has several practical implications. In compression, it is called redundancy and represents the least additional number of bits over the entropy needed to encode the output of any distribution in P. In online es- timation and learning, it is the lowest expected log-loss regret when guessing a sequence of random values generated by a distribution in P. In hypothesis testing, it upper bounds the largest number of distinguishable distributions in P. Motivated by problems ranging from population estimation to text classification and speech recognition, several machine-learning and information-theory researchers have recently considered label-invariant observations and properties induced by i.i.d. distributions. A sufficient statistic for all these properties is the data’s profile, the multiset of the number of times each data element appears. Improving on a sequence of previous works, we show that the redun- dancy of the collection of distributions induced over profiles by length-n i.i.d. sequences is between 0.3 · n1/3 and n1/3 log2 n, in particular, establishing its exact growth power.

Pattern maximum likelihood (PML) is a technique for estimating the probability multiset of an unknown distribution. With any random sample, it associates the distribution maximizing the probability of its pattern. The required computation is a maximization of a monomial symmetric polynomial over the monotone simplex. The PML of only very few patterns have been found analytically, and for other patterns, the PML has been approximated by a heuristic algorithm. Taking an algebraic approach, we determine the PML of short patterns by solving a system of multivariate polynomial equations using the method of resultants. Using this approach, we determine the PML of the pattern 1112234, the last length-7 pattern whose PML was unknown. Under two plausible but yet unproved assumptions on the optimal alphabet size and the number of distinct probabilities, we also find the PML distribution of all previously unknown patterns of length up to 14.

Motivated by protein sequencing, we consider the problem of reconstructing a string from the compositions of its substrings. We provide several results, including the following. General classes of strings that cannot be distinguished from their substring compositions. An almost complete characterization of the lengths for which reconstruction is possible. Bounds on the number of strings with the same substring compositions in terms of the number of divisors of the string length plus one. A relation to the turnpike problem and a bivariate polynomial formulation of string reconstruction.

We describe two algorithms for calculating the probability of m-symbol length-n patterns over k-element distributions, a partition-based algorithm with complexity roughly 2O(m log m)and a recursive algorithm with complexity roughly 2O(m+log n)with the precise bounds provided in the text. The problem is related to symmetric-polynomial evaluation, and the analysis reveals a connection to the number of connected graphs.

We consider the problem of classification, where the data of the classes are generated i.i.d. according to unknown probability distributions. The goal is to classify test data with minimum error probability, based on the training data available for the classes. The Likelihood Ratio Test (LRT) is the optimal decision rule when the distributions are known. Hence, a popular approach for classification is to estimate the likelihoods using well known probability estimators, e.g., the Laplace and Good-Turing estimators, and use them in a LRT. We are primarily interested in situations where the alphabet of the underlying distributions is large compared to the training data available, which is indeed the case in most practical applications. We motivate and propose LRT's based on pattern probability estimators that are known to achieve low redundancy for universal compression of large alphabet sources. While a complete proof for optimality of these decision rules is warranted, we demonstrate their performance and compare it with other well-known classifiers by various experiments on synthetic data and real data for text classification.

The probability of error in the Koetter-Vardy algebraic soft-decoding algorithm for Reed-Solomon codes is determined by the multiplicity assignment scheme used. A multiplicity assignment scheme converts the reliability matrix Pi, consisting of the probabilities observed at the channel output, into a multiplicity matrix M that specifies the algebraic interpolation conditions. Using the method of types, Sanov's theorem in particular, we obtain tight exponential bounds on the probability of decoding error for a given multiplicity matrix. These bounds turn out to be essentially the same as the Chernoff bound. We establish several interesting properties of the multiplicity matrix Mdaggerwhich minimizes the exponent of the probability of error. Based on these observations, we develop a low-complexity multiplicity assignment scheme which uses nested bisection to solve for Mdagger. This scheme provides the same probability of error as a known scheme based upon the Chernoff bound, but with much lower complexity. We also derive a simple condition on the reliability matrix Pi which guarantees an exponentially small probability of error. This condition is akin to an error-correction radius, and can be used to study the performance of algebraic soft-decoding.

Standard redundancy measures the excess number of bits needed to compress a sequence as a function of the sequence’s length. Since long sequences can have arbitrarily low minimum description length (MDL), even low standard redundancy can be arbitarily high compared to the sequence’s MDL. By contrast, relative redundancy evaluates the excess number of bits as a function of the sequence’s MDL. Hence unlike standard redundancy, low relative redundancy implies that the number of bits needed to compress any sequence is essentially the lowest possible. Results in [1] show that for iid distributions over binary alphabets, block relative redundancy essentially equals block standard redundancy while sequential relative redundancy is about twice its standard counterpart. We show that unlike binary alphabets, for larger alphabets both block and sequential relative redundancy essentially equal their standard counterparts. We also define and determine expected relative redundancy and show that it is almost same as worst-case relative redundancy.