Lexical gaps are words that do not exist in certain languages.They pose challenges for building multilingual lexical resources, for machine translation, and for cross-lingual transfer.Existing lexical gap detection relies on human judgments or fixed conceptual taxonomies.We propose a data-driven framework for identifying cross-lingual lexical gaps.We extracted contextualized embeddings from Korean-English bilingual LLMs for Korean-to-English and English-to-Korean translation pairs.Combinations of LLMs, embedding types, dimensionality, and orthogonal transformations across 100 train-test splits yielded 4000 distinct embedding spaces in each source language.In each space, we computed the semantic similarity between each source word and its nearest neighbor in the target language, and compared their distribution for gap words versus non-gap words.In 94% (Korean-to-English) and 97% (English-to-Korean) of embedding spaces, gap words showed weaker cross-lingual semantic alignment than non-gap words.Logistic classifiers trained on unaligned embedding spaces can reliably separate gap words from non-gap words, achieving AUCs of 0.81 (Korean-to-English) and 0.76 (English-to-Korean) and retrieving 18/19 Korean and 26/27 English gap words.This approach provides a languageagnostic and taxonomy-free method for scalable lexical gap identification.

Costa's celebrated ldquowriting on dirty paperrdquo (WDP) shows that the power-constrained channelY=X+S+Z, with GaussianZ, has the same capacity as the standard AWGN channelY=X+Z, provided that the ldquointerferencerdquoS(no matter how strong it is) is known at the transmitter. While this ability for perfect interference cancelation is very appealing, it relies heavily on the Gaussianity of the (unknown) noiseZ. We construct an example of ldquobadrdquo noise for writing on dirty paper, namely, ldquodifference set noiserdquo. If the interferenceSis strong, then difference-set noise limits the WDP capacity to at most 2 bits. At the same time, like in the AWGN case, the zero-interference capacity grows without bound with the input constraint. Thus almost 100% of the available capacity is lost in WDP in the presence of difference-set noise. This high capacity loss is due to the ldquoentropy amplification propertyrdquo (EAP) of noise with an aperiodic probability distribution. Using the EAP and the duality between WDP and Wyner-Ziv source coding, we also give an example of dramatic rate-loss in quantizing encrypted source.

We consider "bit stealing" scenarios where the rate of a source code must be reduced without prior planning. We first investigate the efficiency of source requantization to reduce rate, which we term successive degradation. We focus on finite-alphabet sources with arbitrary distortion measures as well as the Gaussian-quadratic and high-resolution scenarios. We show an achievable rate-distortion tradeoff and prove that this is the best guaranteeable tradeoff for any good source code. This tradeoff is in general different from the rate-distortion tradeoff with successive refinement, where there is prior planning. But, we show that with quadratic distortion measures, for all sources with finite differential entropy and at least one finite moment, the gap is at most 1/2 bit or 3 dB in the high-resolution limit. In the Gaussian-quadratic case, the gap is at most 1/2 bit for all resolutions. We further consider bit stealing in the form of information embedding, whereby an embedder acts on a quantized source and produces an output at the same rate and in the original source codebook. We develop achievable distortion-rate tradeoffs. Two cases are considered, corresponding to whether or not the source decoder is informed of the embedding rate. In the Gaussian-quadratic case, we show the informed decoder need only augment the regular decoder with simple post-reconstruction distortion compensation in the form of linear scaling for the resulting system to be as efficient as bit stealing via successive degradation. Finally, we show that the penalty for uninformed versus informed decoders is at most 3 dB or 0.21-bit in the Gaussian-quadratic case and that their performance also lies within the 1/2-bit gap to that of successive refinement.

A subset of an Abelian group has unique differences if for all nonzeros. when viewed as additive noise, sets with unique differences amplify the output entropy as much as possible for a large class of input distributions, which is known as entropy amplification property. Aperiodic (noise) distributions arise as extreme cases in the investigation of the rate loss in side information problems such as channel coding with additive interference known at the encoder and lossy source coding with side information at the decoder. The decoder outputs a reconstruction, which is required to satisfy a distortion constraint. Reconstructing the clean source with some distortion is equivalent to reconstructing the encrypted source with the same distortion. Using the EAP, the rate loss can be arbitrarily large and arbitrarily close to 100%.

An adaptive joint power control/rate allocation policies which maximize system throughput for multiaccess and broadcast fading channels with random packet arrivals and queueing is established in this paper.

We consider source requantization in two forms - successive degradation (i.e., source fidelity reduction) and bit stealing (i.e., information embedding) when no forward planning has been done to facilitate the requantization. We focus on finite-alphabet sources with arbitrary distortion measures as well as the Gaussian-quadratic scenario. For the successive degradation problem, we show an achievable distortion-rate trade-off for non-hierarchically structured rate-distortion achieving codes, and compare it to the distortion-rate trade-off of successively refinable codes. We further consider source requantization in the form of bit stealing, whereby an information embedder acts on a quantized source, producing an output at the same rate. Building on the successive degradation results, we develop achievable distortion-rate trade-offs. Two cases are considered, corresponding to whether the source decoder is informed of any bit stealing or not. In the latter case, the embedder must produce outputs in the original source codebook. For the Gaussian-quadratic scenario, all trade-offs are within 0.5 bits/sample of the distortion-rate bound. Furthermore, for bit stealing, the use of simple post-reconstruction processing that is only a function of the embedded rate can eliminate the loss experienced by uninformed decoders.

Watermarking models a copyright protection mechanism where an original source sequence or "covertext" is modified before distribution to the public in order to embed some extra information. The embedding should be transparent (i.e., the modified data sequence or "stegotext" should be similar to the covertext) and robust (i.e., the extra information should be recoverable even if the stegotext is modified further, possibly by a malicious "attacker"). We compute the coding capacity of the watermarking game for a Gaussian covertext and squared-error distortions. Both the public version of the game (covertext known to neither attacker nor decoder) and the private version of the game (covertext unknown to attacker but known to decoder) are treated. While the capacity of the former cannot, of course, exceed the capacity of the latter, we show that the two are, in fact, identical. These capacities depend critically on whether the distortion constraints are required to be met in expectation or with probability one. In the former case, the coding capacity is zero, whereas in the latter it coincides with the value of related zero-sum dynamic mutual information games of complete and perfect information. We also compute the capacity when the attacker is restricted to additive attacks. This capacity turns out to be strictly larger than the watermarking capacity, thus demonstrating that additive attacks are suboptimal. In fact, under the additive attack restriction, capacity turns out to coincide with the capacity of Costa's (1983) model for "writing on dirty paper," thus demonstrating that in Costa's model, the independent and identically distributed (i.i.d.) Gaussian "noise" is the most malevolent power-limited "noise". Additionally, Costa's observation that in the presence of i.i.d. Gaussian "noise," an i.i.d. Gaussian "dirt" process that is noncausally known to the transmitter (but not receiver) does not reduce capacity, is extended to general ergodic "dirt" and to stationary (but not necessarily white) Gaussian "noise".