Black-box machine translation systems have proven incredibly useful for a variety of applications yet by design are hard to adapt, tune to a specific domain, or build on top of. In this work, we introduce a method to improve such systems via automatic pre-processing (APP) using sentence simplification. We first propose a method to automatically generate a large in-domain paraphrase corpus through back-translation with a black-box MT system, which is used to train a paraphrase model that “simplifies” the original sentence to be more conducive for translation. The model is used to preprocess source sentences of multiple low-resource language pairs. We show that this preprocessing leads to better translation performance as compared to non-preprocessed source sentences. We further perform side-by-side human evaluation to verify that translations of the simplified sentences are better than the original ones. Finally, we provide some guidance on recommended language pairs for generating the simplification model corpora by investigating the relationship between ease of translation of a language pair (as measured by BLEU) and quality of the resulting simplification model from back-translations of this language pair (as measured by SARI), and tie this into the downstream task of low-resource translation.

Just as semantic hashing can accelerate information retrieval, binary valued embeddings can significantly reduce latency in the retrieval of graphical data. We introduce a simple but effective model for learning such binary vectors for nodes in a graph. By imagining the embeddings as independent coin flips of varying bias, continuous optimization techniques can be applied to the approximate expected loss. Embeddings optimized in this fashion consistently outperform the quantization of both spectral graph embeddings and various learned real-valued embeddings, on both ranking and pre-ranking tasks for a variety of datasets.

Consider a binary modulo-additive noise channel with noiseless feedback. When the noise is a stationary and ergodic process Z, the capacity is 1- H(Z) (H(·) denoting the entropy rate). It is shown analogously that when the noise is a deterministic sequence z∞, the capacity under finite-state encoding and decoding is 1 - ρ̅(z∞), where ρ̅(·) is Lempel and Ziv's finite-state compressibility. This quantity, termed the porosity σ(·) of the channel, holds as the fundamental limit to communication - even when the encoder is designed with knowledge of the noise sequence. A sequence of schemes are presented that universally achieve porosity for any noise sequence. These results, both converse and achievability, may be interpreted as a channel-coding counterpart to Ziv and Lempel's work in universal source coding, and also as an extension to existing work on communicating across modulo-additive channels with an individual noise sequence. In addition, a potentially more practical architecture is suggested that draws a connection with finite-state predictability, as introduced by Feder, Gutman, and Merhav.

We consider the problem of lossy estimation of an arbitrary smooth function of correlated data in a stream. In this problem, a user sequentially observes correlated random variables and wants to construct an estimate of the specified function so that the mean squared estimation error is small. Techniques from high resolution quantization theory are applied and expanded for this problem, and the optimal distortion-rate exponent for companding quantization is determined. In the process, connections are established to sufficient statistics and to sensitivity matrices, as introduced by Linder et al. in the context of companding quantization under non-difference distortion measures. These results are applied to several example statistical functions, including the sample mean, sample variance, and the p-th order statistic.

Unsupervised learning may be modeled by an extreme version of the universal channel coding problem. Suppose a channel decoder knows only that a randomly generated block code is being employed at the other end of a discrete memoryless channel. The channel statistics, the codebook, the code distribution, the rate, the blocklength, and even the input alphabet are unknown. Using a novel decoding measure, it is shown that the channel outputs may be correctly clustered with vanishing error probability at all rates under capacity. Results provide theoretical motivation for certain heuristic clustering techniques and, more widely, suggest that there are gains to be had via non-pairwise similarity measures.

Consider a binary modulo-additive noise channel with noiseless feedback. When the noise is a stationary and ergodic process Z, the capacity is 1 - H(Z) (H(·) denoting the entropy rate). It is shown analogously that when the noise is a deterministic sequence Z∞, the capacity under finite-state encoding and decoding is 1 - ρ̅(Z∞), where ρ̅(·) is Lempel and Ziv's finite-state compressibility. This quantity is termed the porosity ρ̅(·) of an individual noise sequence. A sequence of schemes are presented that universally achieve porosity for any noise sequence. These results may be interpreted both as a channel-coding counterpart to Ziv and Lempel's work in universal source coding, as well as an extension of the work by Lomnitz and Feder and Shayevitz and Feder on communication across modulo-additive channels.

Communication of quantized information is frequently followed by a computation. We consider situations of distributed functional scalar quantization: distributed scalar quantization of (possibly correlated) sources followed by centralized computation of a function. Under smoothness conditions on the sources and function, companding scalar quantizer designs are developed to minimize mean-squared error (MSE) of the computed function as the quantizer resolution is allowed to grow. Striking improvements over quantizers designed without consideration of the function are possible and are larger in the entropy-constrained setting than in the fixed-rate setting. As extensions to the basic analysis, we characterize a large class of functions for which regular quantization suffices, consider certain functions for which asymptotic optimality is achieved without arbitrarily fine quantization, and allow limited collaboration between source encoders. In the entropy-constrained setting, a single bit per sample communicated between encoders can have an arbitrarily large effect on functional distortion. In contrast, such communication has very little effect in the fixed-rate setting.

Suppose a function of N real source variables X1N= (X1, X2, ..., XN) is desired at a destination constrained to receive a limited number of bits. If the result of evaluating the function, Y = G(X1N), can be itself encoded, this is the optimal strategy-the origin of Y becomes irrelevant to the communication problem. We consider two alternative scenarios: distributed quantization, in which each Ximust be separately encoded; and linear transform coding of X1N. Optimal fixed- and variable-rate scalar quantizers are derived under the conventional assumptions of high-resolution quantization theory, and we find optimal transforms for transform coding. For certain classes of functions, examples demonstrate large improvements over using quantizers designed to minimize distortion of the Xis.