Given a probability distribution $p = (p_1, \dots, p_n)$ and an integer $1\leq m \leq n$, a contiguous aggregation of $p$ is a probability distribution $q = (q_1, \dots, q_m)$ such that each $q_i$ is a sum of consecutive elements of $p$. Given $p$ and a positive number $R$, we consider the problem of computing a maximum entropy contiguous aggregation $q$ of $p$, under the constraint that its Shannon entropy $H(q)$ is at most $R$. We devise a dynamic programming algorithm that solves the problem exactly, and two time-efficient greedy algorithms that provide close-to-optimal solutions. We discuss a few scenarios where our problem arises.

Given a random variable X that takes values in a finite setX= {x1, . . . ,xn}, and a positive numberR, we consider the problem of finding a deterministic functionf:X→Ythat maximizes the entropyH(f(X)), while satisfying the constraintH(f(X)) ≤R. This problem occurs in several contexts, including lossy compression with logarithmic loss and the well-known Deterministic Information Bottleneck problem. We prove the NP-hardness of finding the optimal solution to the above maximization problem. On the positive side, we design approximation algorithms that achieve improved approximation factors compared to existing methods in the literature.

In this paper, we consider the problem of constructing optimal average-length binary codes under the constraint that each codeword must contain at most$D$ones, where$D$is a given input parameter. We provide an$O\left(n^{2} D\right)$-time complexity algorithm for the construction of such codes, where$n$is the number of codewords. We also describe several scenarios where the need to design these kinds of codes naturally arises. Our algorithms allow us to construct both optimal average-length prefix binary codes and optimal average-length alphabetic binary codes. In the former case, our$O\left(n^{2} D\right)$-time algorithm substantially improves on the previously known$O\left(n^{2+D}\right)$-time complexity algorithm for the same problem. We also provide a Kraft-like inequality for the existence of (optimal) variable-length binary codes, subject to the above-described constraint on the number of 1's in each codeword.

ABSTRACT A cluster graph is a disjoint union of cliques, obtained by clustering the nodes of a given network and then removing the edges between nodes assigned to different clusters. The Cluster Deletion problem asks for the smallest subset of edges to be removed from a network in order to produce a cluster graph, which is equivalent to determining the largest subset of edges to be preserved. The problem finds application in many fields, including computational biology, bioinformatics, and wireless sensor networks, and it is known to be ‐hard on general graphs. In this work, we formulate the problem as an integer linear program, and we devise a heuristic approach based on edge contraction operations. We test the proposed approaches on both artificial instances and benchmark biological networks.

Alphabetic codes and binary search trees are combinatorial structures that abstract search procedures in ordered sets endowed with probability distributions. In this paper, we design new linear-time algorithms to construct alphabetic codes, and we show that the obtained codes are not too far from being optimal. Moreover, we exploit our results on alphabetic codes to provide new bounds on the average cost of optimal binary search trees. Our results improve on the best-known bounds on the average cost of optimal binary search trees present in the literature.

List union-free families are basic combinatorial structures that appear in different application scenarios, most notably in one-bit compressed sensing. In this paper, we study algorithms for the construction of list union-free families and we provide bounds on the parameters of these families that substantially affect the complexity of the algorithms that utilize them.

In this paper, we study upper bounds on the minimum length of frameproof codes introduced by Boneh and Shaw [3] to protect copyrighted materials. A q-ary (k,n)-frameproof code of length t is a t×n matrix having entries in {0,1,…,q−1} and with the property that for any column c and any other k columns, there exists a row where the symbols of the k columns are all different from the corresponding symbol (in the same row) of the column c. In this paper, we show the existence of q-ary (k,n)-frameproof codes of length $t = O\left( {\frac{{{k^2}}}{q}\log n} \right)$ for q ≤ k, using the Lovász Local Lemma, and of length $t = O\left( {\frac{{{k^2}}}{{\log \left( {q/k} \right)}}\log \left( {n/k} \right)} \right)$ for q > k using the expurgation method. Remarkably, for the practical case of q ≤ k our findings give codes whose length almost matches the lower bound $\Omega \left( {\frac{{{k^2}}}{{q\log k\log n}}} \right)$ on the length of any q-ary (k,n)-frameproof code and, more importantly, allow us to derive an algorithm of complexity O(tn2) for the construction of such codes.

In this paper, we study bounds on the minimum length of ( k, n, d)-superimposed codes introduced by Agarwal et al. [1], in the context of Non-Adaptive Group Testing algorithms with runlength constraints. A ( k, n, d)-superimposed code of length t is a t × n binary matrix such that any two 1’s in each column are separated by a run of at least d 0’s, and such that for any column c and any other k−1 columns, there exists a row where c has 1 and all the remaining k−1 columns have 0. Agarwal et al. proved the existence of such codes with t = Θ( dk log( n/k) + k2log( n/k)). Here we investigate more in detail the coefficients in front of these two main terms as well as the role of lower order terms. We show that improvements can be obtained over the construction in [1] by using different constructions and by an appropriate exploitation of the Lovász Local Lemma in this context. Our findings also suggest O( nk) randomized Las Vegas algorithms for the construction of such codes. We also extend our results to Two-Stage Group Testing algorithms with runlength constraints.

Given a discrete random variable X that takes values in a finite set χ according to a probability mass function (pmf) P, a truncated pmf Q of P is a conditional pmf that results from restricting the domain of X to some subset of χ. Truncated pmf arise in several problems of statistics and probability. In this paper, we propose and analyze a few criteria to truncate pmf's so that the truncated one is as much close as possible to the original pmf, under different information theoretic measures of distance.

Given two discrete random variables X and Y, with probability distributions p = (p1, ..., pn) and q = (q1, ..., qm), respectively, let us denote by C(p, q) the set of all couplings of p and q, that is, the set of all bivariate probability distributions that have p and q as marginals. In this paper, we study the problem of finding a joint probability distribution in C(p, q) of minimum entropy (equivalently, a coupling that maximizes the mutual information between X and Y), and we discuss several situations where the need for this kind of optimization naturally arises. Since the optimization problem is known to be NP-hard, we give an efficient algorithm to find a joint probability distribution in C(p, q) with entropy exceeding the minimum possible at most by 1 bit, thus providing an approximation algorithm with an additive gap of at most 1 bit. Leveraging on this algorithm, we extend our result to the problem of finding a minimum-entropy joint distribution of arbitrary k ≥ 2 discrete random variables X1, ..., Xk, consistent with the known k marginal distributions of the individual random variables X1, ..., Xk. In this case, our algorithm has an additive gap of at most log k from optimum. We also discuss several related applications of our findings and extensions of our results to entropies different from the Shannon entropy.

Given a probability distribution p=(p1, ⋯, pn) and an integer 1 ≤ m1, ⋯, qm) is a contiguous m-aggregation of p if there exist indices such that for each j=1, ⋯, m it holds that qj= Σk=i(j-1)+1ijpk. In this paper, we consider the problem of efficiently finding the contiguous m-aggregation of maximum entropy. We design a dynamic programming algorithm that solves the problem exactly, and two more time-efficient greedy algorithms that provide slightly sub-optimal solutions. We also discuss a few scenarios where our problem matters.

In classical secret sharing schemes a dealer shares a secret among a set of participants in such a way that qualified subsets can reconstruct the secret, while forbidden ones do not get any kind of information about it. The basic parameter to optimize is the size of the shares, that is, the amount of secret information that the dealer has to give to participants. In this paper we formalize a notion of probabilistic secret sharing schemes, in which qualified subsets can reconstruct the secret but only with a certain controlled probability. We show that, by allowing a bounded error in the reconstruction of the secret, it is possible to drastically reduce the size of the shares the participants get (with respect to classical secret sharing schemes). We provide efficient constructions both for threshold access structures on a finite set of participants and for evolving threshold access structures, where the set of participants is potentially infinite. Some of our constructions yield shares of constant size (i.e., not depending on the number of participants) and an error probability of successfully reconstructing the secret which can be made as close to 1 as desired.

We consider a population of interconnected individuals that, with respect to a piece of information, at each time instant can be subdivided into three (time‐dependent) categories: agnostics, influenced, and evangelists. A dynamical process of information diffusion evolves among the individuals of the population according to the following rules. Initially, all individuals are agnostic. Then, a set of people is chosen from the outside and convinced to start evangelizing, that is, to start spreading the information. When a number of evangelists, greater than a given threshold, communicate with a node v, the node v becomes influenced, whereas, as soon as the individual v is contacted by a sufficiently much larger number of evangelists, it is itself converted into an evangelist and consequently it starts spreading the information. The question is: How to choose a bounded cardinality initial set of evangelists so as to maximize the final number of influenced individuals? We prove that the problem is hard to solve, even in an approximate sense. On the positive side, we present exact polynomial time algorithms for trees and complete graphs. For general graphs, we derive exact parameterized algorithms. We also study the problem when the objective is to select a minimum number of evangelists capable of influencing the whole network. Our motivations to study these problems come from the areas of Viral Marketing and spread of influence in social networks. © 2017 Wiley Periodicals, Inc. NETWORKS, Vol. 71(4), 346–357 2018

It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight bounds on $H(f(X))$ when the function $f$ is not one-to-one, and we illustrate a few scenarios where this matters. As an intermediate step towards our main result, we derive a lower bound on the entropy of a probability distribution, when only a bound on the ratio between the maximal and minimal probabilities is known. The lower bound improves on previous results in the literature, and it could find applications outside the present scenario.

It is well known that the entropy H (X) of a finite random variable is always greater or equal to the entropy H (f (X)) of a function f of X, with equality if and only if f is one-to-one. In this paper, we give tights bounds on H(f (X) when the function f is not one-to-one, and we illustrate a few scenarios where this matters. As an intermediate step towards our main result, we prove a lower bound on the entropy of a probability distribution, when only a bound on the ratio between the maximum and the minimum probability is known. Our lower bound improves previous results in the literature, and it could find applications outside the present scenario.

Given two discrete random variables X and Y, with probability distributions p = (p1, ..., pn) and q = (q1, ..., qm), respectively, denote by C(p, q) the set of all joint distributions of X and Y that have p and q as marginals. In this paper, we study the problem of finding the joint probability distribution in C (p, q) of minimum entropy (equivalently, the joint probability distribution that maximizes the mutual information between X and Y), and we discuss several situations where the need for this kind of optimization naturally arises. Since the optimization problem is known to be NP-hard, we give an efficient algorithm to find a joint probability distribution in C(p, q) with entropy exceeding the minimum possible by at most 1, thus providing an approximation algorithm with additive approximation factor of 1. We also discuss some related consequences of our findings.

Consider a group of stations connected through a multiple-access channel, with the constraint that if at a time instant exactly one station transmits a message, then the message is successfully received by any other station, whereas if two or more stations simultaneously transmit their messages then a conflict occurs and all messages are lost. Let us assume that n is the number of stations and that an (arbitrary) subset A of them, |A| ≤ k ≤ n, is active, that is, there are at most k stations that have a message to send over the channel. In the classical conflict resolution problem, the issue is to schedule the transmissions of each station to let every active station use the channel alone (i.e., without conflict) at least once, and this requirement must be satisfied whatever might be the set of active stations A. The parameter to optimize is, usually, the worst case number of transmissions that any station has to attempt before all message transmissions are successful. In this paper, we study the following question: is it possible to obtain a significant improvement on the protocols that solve the classical conflict resolution problem if we allow the protocols to fail over a “small” fraction of all possible subsets of active stations? In other words, is it possible to significantly reduce the number of transmissions that must be attempted if the set of active stations is chosen uniformly at random and the conflict resolution algorithm is only required to work correctly with “high” probability? In this paper, we will show that this is indeed the case. Our main technical tool is a generalization of selectors, a recently introduced combinatorial structure that has found applications in several areas. As it turned out for selectors, we believe that our new combinatorial structures are likely to be useful also outside the present context.

Given a probability distribution p = (p1, ..., pn) and an integer m1, ..., qm) that is “the closest” to p, that is, that best approximates p? It is clear that the answer depends on the function one chooses to evaluate the goodness of the approximation. In this paper we provide a general criterion to approximate p with a shorter vector q by using ideas from majorization theory. We evaluate the goodness of our approximation by means of a variety of information theoretic distance measures.

Identifying the most influential spreaders is an important issue for the study of the dynamics of information diffusion in complex networks. In this paper we analyze the following spreading model. Initially, a few nodes know a piece of information and are active spreaders of it. At subsequent rounds, spreaders communicate the information to their neighbors. Upon receiving the information, a node becomes aware of it but does not necessarily become a spreader; it starts spreading only if it gets the information from a sufficiently large number of its neighbors. We study the problem of choosing a small set of initial spreaders so as to maximize the final number of nodes that become aware of the information.

We introduce two new information theoretic measures of distances among probability distributions and we discuss their possible applications to Econometrics.

We consider the problem of searching for an unknown number in the search space U ={0,...,M-1}. q-ary questions can be asked and some of the answers may be wrong. An arbitrary integer weighted bipartite graph Gamma is given, stipulating the cost Gamma(i,j) of each answer jnei when the correct answer is i, i.e., the cost of a wrong answer. Correct answers are supposed to be cost-less. It is assumed that a maximum cost e for the sum of the cost of all wrong answers can be afforded by the responder during the whole search. We provide tight upper and lower bounds for the largest size M = M(q,e,Gamma,n) for which it is possible to find an unknown number x*isinU with n q-ary questions and maximum lie cost e. Our results improve the bounds of Cicalese et al. (2004) and Ahlswede et al. (2008). The questions in our strategies can be asked in two batches of nonadaptive questions. Finally, we remark that our results can be further generalized to a wider class of error models including also unidirectional errors.

The well known Tunstall algorithm for discrete memoryless sources [17] produces optimal variable-to-fixed length source codes that maximize the expected number of source letters per codeword. Tun stall algorithm achieves this result by constructing parse trees with maximum average height for the source output. In the first part of this paper we introduce a simple variant of Tun stall algorithm in order to optimizes additional natural cost functions of interest. For instance we show how to select, among all parse trees with maximum average height, those having minimum height, minimum variance, minimum external length, and more general natural parameters. In the second part of the paper we consider the problem of selecting, among all parse trees of height bounded by some parameter L, those parse trees having maximum average height. We motivate the problem, and we quantify the loss of performance these parse trees suffer with respect to unrestricted Tuns tall parse trees, when they are used as variable-to-fixed length encoding for a discrete memoryless source.

Two variants of the well-known group testing problem are considered. In the first variant a finite set of items O and an unknown subset PsubeO are given, and one wants to identify the set P by asking the least number of questions of the form "Is |QcapP|=1?", where QsubeO. This problem naturally arises in the design of efficient contention resolution algorithms for certain random multiple-access communication systems [Berger et al. "Random multiple-access communication and group testing," IEEE Trans. Commun., vol. 32, no. 7, pp. 769-779, 1984]. In the second variant of the problem, the answer to the question "Is |QcapP|=1?" is correctly YES if |QcapP|=1 and NO if |QcapP|=0", and it is left to a (possibly malicious) adversary otherwise. This model was introduced in [Damaschke, "Randomized group testing for mutually obscuring defectives", Inf. Process. Lett., vol. 67, pp. 131-135, 1998], in the context of chemical compound testing. In this correspondence several algorithms for these group testing problems are presented, trying to optimize different measures of performance: The overall number of tests performed by the algorithm, the number of stages in which tests can be arranged, and the decoding complexity of identifying the elements of P from tests outcomes. Some of the given algorithms are optimal with respect to more than one of the above criteria. Instrumental to the results presented in the correspondence are new and improved bounds on certain generalization of superimposed codes introduced in [Dyachkov and Rykov, "A generalization of superimposed codes and its application to the multiple-access channel", in Proc. 1984 IEEE Int. Symp. Inf. Theory, pp. 62-64], [De Bonis and Vaccaro, "Constructions of generalized superimposed codes with applications to group testing and conflict resolution in multiple access channels", Theoretic. Comput. Sci., vol. 306, no. 1-3, pp. 223-243, 2003] a result that it is believed to be of independent interest

In this correspondence, we prove that the probability distribution induced on the leaves of a Tunstall parse tree for a given source is a (unique) lower bound in the partially ordered set of the probability distributions induced by all possible parse trees with a same number of leaves, and ordered according to the majorization partial order. We apply this result to the problem of optimally approximating a uniform distribution with flips of a biased coin

Group testing refers to the situation in which one is given a set of objects ${\cal O}$, an unknown subset ${\cal P}\subseteq {\cal O}$, and the task of determining ${\cal P}$ by asking queries of the type "does ${\cal P}$ intersect $\cal Q$?," where $\cal Q$ is a subset of ${\cal O}$. Group testing is a basic search paradigm that occurs in a variety of situations such as quality control testing, searching in storage systems, multiple access communications, and data compression, among others. Group testing procedures have been recently applied in computational molecular biology, where they are used for screening libraries of clones with hybridization probes and sequencing by hybridization. Motivated by particular features of group testing algorithms used in biological screening, we study the efficiency of two-stage group testing procedures. Our main result is the first optimal two-stage algorithm that uses a number of tests of the same order as the information-theoretic lower bound on the problem. We also provide efficient algorithms for the case in which there is a Bernoulli probability distribution on the possible sets ${\cal P}$, and an optimal algorithm for the case in which the outcome of tests may be unreliable because of the presence of "inhibitory" items in ${\cal O}$. Our results depend on a combinatorial structure introduced in this paper. We believe that it will prove useful in other contexts, too.

Let s be the number of unknown positive elements in a population of n members, 2/spl les/s

We consider the problem of bounding the average length of an optimal (Huffman) source code when only limited knowledge of the source symbol probability distribution is available. For instance, we provide tight upper and lower bounds on the average length of optimal source codes when only the largest or the smallest source symbol probability is known. Our results rely on basic results of majorization theory and on the Schur concavity of the minimum average length of variable-length source codes for discrete memoryless sources. In the way to prove our main result we also give closed formula expressions for the average length of Huffman codes for several classes of probability distributions.

We prove that the entropy is a supermodular and subadditive function on the lattice of all n-dimensional probability distributions, ordered according to the partial order relation defined by majorization among vectors.

Abstract We consider the problem of disseminating messages in networks. We are interested in information dissemination algorithms in which machines operate independently without any knowledge of the network topology or size. Three communication tasks of increasing difficulty are studied. In blind broadcasting (BB), the goal is to communicate the source message to all nodes. In acknowledged blind broadcasting (ABB), the goal is to achieve BB and inform the source about it. Finally, in full synchronization (FS), all nodes must simultaneously enter the state terminated after receiving the source message. The algorithms should be efficient both in terms of the time required and the communication overhead they put on the network. We limit the latter by allowing every node to send a message to at most one neighbor in each round. We show that BB is achieved in time at most 2n in any n‐node network and show networks in which time 2n − o(n) is needed. For ABB, we show algorithms working in time (2 + ϵ)n, for any fixed positive constant ϵ and sufficiently large n. Thus, for both BB and ABB, our algorithms are close to optimal. Finally, we show a simple algorithm for FS working in time 3n and a more complicated algorithm which works in time 2.9n. The optimal time of full synchronization remains an open problem. © 2001 John Wiley & Sons, Inc.

We consider the guessing problem proposed by Massey (see Proc. Int. Symp. Information Theory, p.204, 1994) of a cryptanalyst that wants to break a ciphertext with a brute-force attack. The best strategy he can use is to try out all possible keys, one at time in order of decreasing probability, after narrowing the possibilities by some cryptanalysis. In this correspondence we provide both upper and lower bounds on the entropy of the probability distribution on the secret keys in terms of the number of secret keys and of the average number of trials of the cryptanalyst.

Given a random variable X which takes n equiprobable values, we consider several algorithmic questions related to the classical problem of simulating the outcomes of X by using a limited number of biased coins.

Gossiping is the process of information diffusion in which each node of a network holds a packet that must be communicated to all other nodes in the network. We consider the problem of gossiping in communication networks under the restriction that communicating nodes can exchange up to a fixed number p of packets at each round. In the first part of the paper we study the extremal case p=1 and we exactly determine the optimal number of communication rounds to perform gossiping for several classes of graphs, including Hamiltonian graphs and complete k-ary trees. For arbitrary graphs we give asymptotically matching upper and lower bounds. We also study the case of arbitrary p and we exactly determine the optimal number of communication rounds to perform gossiping under this hypothesis for complete graphs, hypercubes, rings, and paths. Finally, we investigate the problem of determining sparse networks in which gossiping can be performed in the minimum possible number of rounds.

/spl acute/A secret sharing scheme permits a secret to be shared among participants in such a way that only qualified subsets of participants can recover the secret, but any nonqualified subset has absolutely no information on the secret. We derive new limitations on the information rate of secret sharing schemes, that measures how much information is being distributed as shares as compared to the size of the secret key, and the average information rate, that is the ratio between the secret size and the arithmetic mean of the size of the shares. By applying the substitution technique, we are able to construct many new examples of access structures where the information rate is bounded away from 1. The substitution technique is a method used to obtain a new access structure by replacing a participant in a previous structure with a new access structure.>

Abstract Broadcasting is the process of disseminating a message originated at one node of a network to all other nodes. In this paper, we consider the problem of broadcasting reliably in the hypercube in presence of either transmission or link failures. We propose broadcasting protocols under various assumptions on the communication model. Our broadcasting protocols make use of Rabin's Information Dispersal Algorithm. © 1993 by John Wiley & Sons, Inc.

Consider a network in which n processors are connected by communication lines and are allowed to communicate with at most one other processor at a time. Broadcast is the task of transmitting a message originated at one node to all other nodes in the network. Presented in this paper is a broadcasting scheme that can tolerate up to $k\leqq \lfloor \log \,n \rfloor $ line failures; that is, it assures that each node in the network will receive the message from the originator when up to $k \leqq \lfloor \log n \rfloor $ lines are faulty. The time required by the broadcast protocol is minimum, except in some cases that might require one unit of time more than the minimum. Moreover, an algorithm for constructing networks supporting the broadcast scheme and having approximately the minimum possible number of lines is given.

The problem of constructing statistically synchronizable codes over arbitrary alphabets and for any finite source is considered. It is shown how to efficiently construct a statistically synchronizable code whose average codeword length is within the least likely codeword probability from that of the Huffman code for the same source. Moreover, a method is given for constructing codes having a synchronizing codeword. The method yields synchronous codes that exhibit high synchronizing capability and low redundancy.>

Abstract Broadcast is the task of transmitting a message originated at a node in a network to all the other nodes. In this paper, we consider the problem of constructing minimal broadcast networks, that is, communication networks such that broadcast can be performed, from any node, in minimum time. The algorithms we propose allow us to improve known bounds on the minimum number of communication lines needed in minimal broadcast networks. We give some numerical evidence that our algorithms also perform well in practice.

Immutable codes, which have recently been introduced as a tool for preventing undesirable changes of data recorded over write-once memories, are considered. The have the property that any change of recorded information over such memories can be detected. A fast algorithm for testing whether a variable-length code is immutable is presented. The complexity of the algorithm is O(L/sup 2/), where L is the sum of the codeword lengths.>

The authors consider statistically synchronizable variable-length codes, i.e. codes that admit decoders able to self-synchronize with high probability if the input sequence of code symbols is long enough. They show that a necessary and sufficient condition for the existence of a statistically self-synchronizing decoder and, therefore, for a code to be statistically synchronizable, is that the code has a synchronizing sequence. They also give a decision procedure to test whether a code has a synchronizing sequence. Finally, they specialize the procedure to obtain a simple and efficient algorithm to test the statistical synchronizability property of prefix codes.>