Social media analysis over blogs (such as tweets) often requires determining top-k mentions of a certain category (e.g., movies) in a collection (e.g., tweets collected over a given day). Such queries require entity linking (EL) function to be executed that is often expensive. We propose TQEL, a framework that minimizes the joint cost of EL calls and top-k query processing. The paper presents two variants - TQEL-exact and TQEL-approximate that retrieve the exact / approximate top-k results. TQEL-approximate, using a weaker stopping condition, achieves significantly improved performance (with the fraction of the cost of TQEL-exact) while providing strong probabilistic guarantees (over 2 orders of magnitude lower EL calls with 95% confidence threshold compared to TQEL-exact). TQEL-exact itself is orders of magnitude better compared to a naive approach that calls EL functions on the entire dataset.

Twitter provides a public streaming API that is strictly limited, making it difficult to simultaneously achieve good coverage and relevance when monitoring tweets for a specific topic of interest. In this paper, we address the tweet acquisition challenge to enhance monitoring of tweets based on the client/application needs in an online adaptive manner such that the quality and quantity of the results improves over time. We propose a Tweet Acquisition System (TAS), that iteratively selects phrases to track based on an explore-exploit strategy. Our experimental studies show that TAS significantly improves recall of relevant tweets and the performance improves when the topics are more specific.

We consider the interference channel with K transmitters and K receivers all having a single antenna, wherein the K × K transfer matrix representing this channel has rank D (D <; K) . The degrees of freedom (DoF) of such channels are not known as the rank deficiency in the transfer matrix creates algebraic dependencies between the channel coefficients. We present a modified version of the [CJ08] alignment scheme, to handle these dependencies while aligning interference, and state the sufficient conditions for achieving half rate per user using this scheme. The difficulties in proving these sufficient conditions are shown for K = 4 and K = 5. We also show that these sufficient conditions are not satisfied for K ≥ 6.

Modern data processing techniques such as entity resolution, data cleaning, information extraction, and automated tagging often produce results consisting of objects whose attributes may contain uncertainty. This uncertainty is frequently captured in the form of a set of multiple mutually exclusive value choices for each uncertain attribute along with a measure of probability for alternative values. However, the lay end-user, as well as some end-applications, might not be able to interpret the results if outputted in such a form. Thus, the question is how to present such results to the user in practice, for example, to support attribute-value selection and object selection queries the user might be interested in. Specifically, in this article we study the problem of maximizing the quality of these selection queries on top of such a probabilistic representation. The quality is measured using the standard and commonly used set-based quality metrics. We formalize the problem and then develop efficient approaches that provide high-quality answers for these queries. The comprehensive empirical evaluation over three different domains demonstrates the advantage of our approach over existing techniques.

One of the difficulties in calculating the capacity of certain Poisson channels is thatH(¿), the entropy of the Poisson distribution with mean¿, is not available in a simple form. In this paper, we derive upper and lower bounds forH(¿) that are asymptotically tight and easy to compute. The derivation of such bounds involves only simple probabilistic and analytic tools. This complements the asymptotic expansions of Knessl (1998), Jacquet and Szpankowski (1999), and Flajolet (1999). The same method yields tight bounds on the relative entropyD(n,p) between a binomial and a Poisson, thus refining the work of Harremoe¿s and Ruzankin (2004). Bounds on the entropy of the binomial also follow easily.

We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for Poisson variables. We show that some natural analogues of the EPI do not in fact hold, but propose an alternative formulation which does always hold. The key to many proofs of Shannon's EPI is the behavior of entropy on scaling of continuous random variables. We believe that Rényi's operation of thinning discrete random variables plays a similar role to scaling, and give a sharp bound on how the entropy of ultra log-concave random variables behaves on thinning. In the spirit of the monotonicity results established by Artstein, Ball, Barthe, and Naor, we prove a stronger version of concavity of entropy, which implies a strengthened form of our discrete EPI.

The Arimoto-Blahut algorithm for computing the capacity of a discrete memoryless channel is revisited. A so-called “squeezing” strategy is used to design algorithms that preserve its simplicity and monotonic convergence properties, but have provably better rates of convergence.

Building on the recent work of Johnson (2007) and Yu (2008), we prove that entropy is a concave function with respect to the thinning operation Talpha. That is, if X and Y are independent random variables on Z+with ultra-log-concave probability mass functions, then H(TalphaX + T1-alphaY) ges alphaH(X) + (1 - alpha)H(Y), 0 les alpha les 1, where H denotes the discrete entropy. This is a discrete analogue of the inequality (h denotes the differential entropy) h(radicalphaX + radic1 - alphaY ) ges alphah(X) + (1 - alpha)h(Y), 0 les alpha les 1, which holds for continuous X and Y with finite variances and is equivalent to Shannon's entropy power inequality. As a consequence we establish a special case of a conjecture of Shepp and Olkin (1981). Possible extensions are also discussed.

Some entropy comparison results are presented concerning compound distributions on nonnegative integers. The main result shows that, under a log-concavity assumption, two compound distributions are ordered in terms of Shannon entropy if both the ldquonumbers of claimsrdquo and the ldquoclaim sizesrdquo are ordered accordingly in the convex order. Several maximum/minimum entropy theorems follow as a consequence. Most importantly, two recent results of Johnson (2008) on maximum entropy characterizations of compound Poisson and compound binomial distributions are proved under fewer assumptions and with simpler arguments.

An "entropy increasing to the maximum" result analogous to the entropic central limit theorem (Barron 1986; Artstein 2004) is obtained in the discrete setting. This involves the thinning operation and a Poisson limit. Monotonic convergence in relative entropy is established for general discrete distributions, while monotonic increase of Shannon entropy is proved for the special class of ultra-log-concave distributions. Overall we extend the parallel between the information-theoretic central limit theorem and law of small numbers explored by Kontoyiannis (2005) and HarremoEumls (2007, 2008, 2009). Ingredients in the proofs include convexity, majorization, and stochastic orders.

Let X1,..., Xnbe n independent, symmetric, random variables on the interval [-1, 1]. Ordentlich (2006) showed that the differential entropy of Sn= Sigmai=1nXiis maximized when Xi, i = 1,...,n-1 are symmetric Bernoulli random variables and Xnis uniform (-1, 1). We give a short derivation of this result via an alternative proof of a key lemma of Ordentlich (2006).

It is shown that the${\hbox{Binomial}}(n, p)$distribution maximizes the entropy in the class of ultra-log-concave distributions of order$n$with fixed mean$np$. This result, which extends a theorem of Shepp and Olkin (1981), is analogous to that of Johnson (2007), who considers the Poisson case. The proof constructs a Markov chain whose limiting distribution is${\hbox{Binomial}}(n, p)$and shows that the entropy never decreases along the iterations of this Markov chain.

The data augmentation (DA) algorithm is a simple and powerful tool in statistical computing. In this note basic information theory is used to prove a nontrivial convergence theorem for the DA algorithm.