As machine-learning models grow in size, their implementation requirements cannot be met by a single computer system. This observation motivates distributed settings, in which intermediate computations are performed across a network of processing units, while the central node only aggregates their outputs. However, distributing inference tasks across low-precision or faulty edge devices, operating over a network of noisy communication channels, gives rise to serious reliability challenges. We study the problem of an ensemble of devices, implementing regression algorithms, that communicate through additive noisy channels in order to collaboratively perform a joint regression task. We define the problem formally, and develop methods for optimizing the aggregation coefficients for the parameters of the noise in the channels, which can potentially be correlated. Our results apply to the leading state-of-the-art ensemble regression methods: bagging and gradient boosting. We demonstrate the effectiveness of our algorithms on both synthetic and real-world datasets.

Barrier-error channels have been suggested as a model for non-binary channels that are milder than symmetric-error channels. Such channels are motivated by practical applications in data storage and communications. The barrier-error model allows errors only to (downward) and from (upward) a specific symbol within the alphabet. We study a generalization of prior barrier-error models that considers different numbers of downward and upward errors. The results of this paper include a sufficient condition for error correction, code-size upper and lower bounds, a code construction decomposing to just two constituent symmetric-error codes (prior ones used many), and a decoding algorithm that decodes the two codes jointly. The decoder is shown empirically to achieve better block error rates when compared to previous algorithms.

In data-intensive applications, it is advantageous to perform partial processing close to the data, and communicate intermediate results to a central processor, instead of the data itself. When the communication or computation medium is noisy, the resulting degradation in computation quality at the central processor must be mitigated. We study this problem for the setup of binary classification performed by an ensemble of base functions communicating real-valued confidence levels. We propose a noise-mitigation solution that optimizes the transmission gains and aggregation coefficients of the base functions. Toward that, we formulate a post-training gradient-based optimization algorithm that minimizes the error probability given the training dataset and the noise parameters. We further derive lower and upper bounds on the optimized error probability, and show empirical results that demonstrate the enhanced performance achieved by our approach on real data.

In data-intensive applications, it is advantageous to perform some partial processing close to the data, and communicate to a central processor the partial results instead of the data itself. When the communication medium is noisy, one must mitigate the resulting degradation in computation quality. We study this problem for the setup of binary classification performed by an ensemble of functions communicating real-valued confidence levels. We propose a noise-mitigation solution that works by optimizing the aggregation coefficients at the central processor. Toward that, we formulate a post-training gradient algorithm that minimizes the error probability given the dataset and the noise parameters. We further derive lower and upper bounds on the optimized error probability, and show empirical results that demonstrate the enhanced performance achieved by our scheme on real data.

This paper studies coding on channels with the barrier property: only errors to and from a special barrier state are possible. This model is motivated by storage media that have heterogeneous state structure, not admitting the usual multi-bit scaling of the representation states. Our contributions include derivation of the channel capacity, efficient maximum-likelihood and list decoding algorithms, and finite-block-length analysis using random codes. This work is the first that addresses a barrier channel with separate parameters for the transitions into and out of the barrier state. Earlier work addressed special-case single-parameter models, and focused primarily on the worst-case coding performance.

Resistive memory is a promising technology for achieving unprecedented storage densities and new in-memory computing features. However, to fulfill their promise, resistive memories require array architectures suffering from a severe interference effect called “sneak paths.” In this paper, we address the sneak-path problem through a communication-theory framework. Starting from the fundamental problem of readout with parallel-resistance interference, we develop several tools for detection and coding that significantly improve memory reliability. For the detection problem, we formulate and derive the optimal detector for a realistic array model, and then propose simplifications that enjoy similarly good performance and simpler implementation. Complementing detection for better error rates is done by a new coding scheme that shapes the stored bits to get lower sneak-path incidence. For the same storage rates, the new coding scheme exhibits error rates lower by an order of magnitude compared to known shaping techniques.

This paper studies the problem of reliable resistive-memory readout through the rigorous lens of communication theory. The most dominant reliability issue in resistive memory can be modeled as interference of resistances in parallel to a measured resistance. For this special type of interference we develop detection and coding schemes that are shown to effectively mitigate the effects of sneak-path errors. The uniqueness of this study is that the proposed models combine theoretical rigor with practical richness, hence enabling deep contributions to very real problems.