We propose a novel supervised dictionary learning framework for text classification, integrating the Lempel-Ziv-Welch (LZW) algorithm for data compression and dictionary construction. This two-phase approach refines dictionaries by optimizing dictionary atoms for discriminative power using mutual information and class distribution. Our method facilitates classifier training, such as SVMs and neural networks. We introduce the information plane area rank (IPAR) to evaluate the information-theoretic performance of our algorithm. Tested on six benchmark text datasets, our model performs nearly as well as top models in limited-vocabulary settings, lagging by only about 2% while using just 10% of the parameters. However, its performance drops in diverse-vocabulary contexts due to the LZW algorithm's limitations with low-repetition data. This contrast highlights its efficiency and limitations across different dataset types.

Distributed massive multiple-input multiple-output (mMIMO) is a key technology for improving the performance of future wireless communication systems. As an alternative to channel estimation in mMIMO systems, non-coherent detection offers several advantages. In this paper, we present a comprehensive analysis of non-coherent detection for distributed mMIMO systems. We obtain novel expressions for the signal-to-interference-and-noise ratio (SINR) for differential detection. Building upon a theoretical basis, we show that under non-coherent detection, cooperation is always beneficial in noise-limited conditions. We further investigate scenarios where the gains of cooperation are most evident. We then obtain useful error rate results by deriving expressions for the symbol error probability (SEP) when using differential detection. The results are illustrated by numerical examples and simulations.

Despite their many successful applications, Deep Neural Networks (DNNs) are vulnerable to intentionally designed adversarial examples. Adversarial robustness describes the ability of a machine learning model, e.g., a neural network, to defend against such adversarial attacks. In coding theory, codebooks are designed to minimize the impact of errors occurring with transmission through a noisy channel. Motivated by the similarities between passing a codeword through a noisy channel and defending against adversarial attacks, Error-Correcting Output Codes (ECOCs) are used to achieve state-of-the-art adversarial robustness. Research on codebook designs and the association of codewords to classification labels (assignment) is still at the very early stages, with great room for improvement. In this work, we present novel codebook design and assignment procedures in two stages due to the complexity (NP-hardness) of the underlying problem. A rule-based heuristic codebook design method is proposed in the first stage and an optimization problem to assign the codewords to labels is proposed in the second stage. Since this optimization is NP-hard, a greedy algorithm is proposed to provide a sub-optimal solution. We demonstrate the effectiveness of our framework on three benchmark datasets, under different types of adversarial attacks. The experimental results show that our error-correcting output code framework can effectively improve the adversarial robustness of machine learning models, with up to a 10% increase in accuracy.

We propose a new partial joint processing architecture for distributed massive multiple-input multiple-output (MIMO) networks. As opposed to the traditional full-joint processing architecture, where the channel coefficients of all the users within the cooperating cluster are learnt at the base stations, in the proposed architecture, we allow each base station to learn the channel coefficients only of the users that maintain a strong average received signal-to-noise ratio to that base station based on a predefined threshold. This threshold provides extra flexibility, trading-off channel estimation for performance. We assume a zero-forcing receiver at the central processing unit using estimated channels and unknown terms are set to zero. We then derive an accurate approximation for the instantaneous received signal-to-interference-and-noise ratio of an arbitrary user. We use this approximation to derive closed-form expressions for the achievable rate and symbol error probability of an arbitrary user. Numerical examples are used to illustrate the accuracy of the analysis.

We propose a novel supervised dictionary learning framework, which is based on discriminative power maximization and the classic Lempel-Ziv-Welch (LZW) source coding algorithm that can handle variable-length inputs. In the first stage, the input data is serialized and a dictionary is generated by the LZW algorithm. In the second stage, the dictionary is updated by discarding or selecting a certain amount of atoms in order to increase discriminative power measured by information bottleneck as well as linear similarity metrics. This dictionary learning framework is then analyzed using information bottleneck principles. Experiments on real life datasets from information security and social networking illustrate the effectiveness and accuracy of the proposed algorithms, especially for variable length data. This novel framework also identifies interpretable dictionary atoms, which provide insights to decision processes of the learning system.

For streaming codes an erasure channel is assumed and the decoding delay is one of the main parameters to be considered. In this paper the erasure correcting capability of unit memory convolutional codes based on disjoint RS codes is analyzed. We take a sliding window decoder approach, where only the most current information is decoded before sliding the window one time-step further. We show that when we restrict the decoding delay to a small value, these codes still achieve an excellent erasure correction performance. This makes these codes useful for streaming applications where low latency is required.

We show how Gabidulin codes can be list decoded by using an iterative parametrization approach. For a given received word, our decoding algorithm processes its entries one by one, constructing four polynomials at each step. This then yields a parametrization of interpolating solutions for the data so far. From the final result a list of all codewords that are closest to the received word with respect to the rank metric is obtained.

In this paper, we propose a general framework for Distributed Source Coding (DSC) of multiple sequentially correlated information sources. In particular, we modularize the joint decoding function into a conceptually simple structure by utilizing a notion of "complementarity" on the code space. Assuming a constrained Hamming-distance based correlation, our general framework of DSC achieves arbitrary non-asymmetric-rate compression of multiple sources with a same overall compression rate as asymmetric compression schemes. This DSC framework does not only have flexibility with respect to compression rates, but also accommodates the use of any linear block code and any of its corresponding complements. By investigating the unique connection between syndromes and complementary codewords, we translate the major steps of DSC joint decoding into syndrome decoding followed by channel encoding via a linear block code and also via its complement code. Our aim is to achieve transparency which facilitates practical implementation.

Minimal list decoding for a code C refers to list decoding with radius L(y), where L(y) is the minimum of the distances between the received word y and any codeword in C. In this paper we present a minimal list decoding algorithm for Reed-Solomon (RS) codes. Our approach involves a parametrization of the interpolating polynomials of a minimal Gröbner basis G. We then demonstrate that our parametric approach can be solved by a computationally efficient rational curve fitting solution from a recent paper by Wu. Besides, we present an algorithm to compute the minimum multiplicity as well as the associated optimal values of the parameters. Use of these optimal parameters in the rational interpolation step results in computational as well as memory efficiency.

In this paper, we present a minimal list decoding algorithm for Reed-Solomon (RS) codes. Minimal list decoding for a code C refers to list decoding with radius L, where L is the minimum of the distances between the received word r and any codeword in C. We consider the problem of determining the value of L as well as determining all the codewords at distance L. Our approach involves a parametrization of interpolating polynomials of a minimal Gröbner basis G . We present two efficient ways to compute G. We also show that so-called re-encoding can be used to further reduce the complexity. We then demonstrate how our parametric approach can be solved by a computationally feasible rational curve fitting solution from a recent paper by Wu. Besides, we present an algorithm to compute the minimum multiplicity as well as the optimal values of the parameters associated with this multiplicity, which results in overall savings in both memory and computation.

Existing literature on source coding with side information (SCSI) uses channel codes like LDPC codes and turbo codes and assumes classical unique decoding. In this paper, in contrast to classical decoding, we have taken the list decoding approach and show that the theoretical limit of SCSI can then be achieved. We argue that, as opposed to channel coding, the correct sequence from the list produced by the list decoder can effectively be recovered in case of SCSI with a few CRC bits. The CRC bits, which allow the decoder to identify the correct sequence, incur negligible overhead for large block length. More importantly, these CRC bits are not subject to noise since we are dealing with a virtual noisy channel rather than a real noisy channel. Finally, we present a guideline for designing constructive SCSI schemes using Reed Solomon code, BCH code, and Reed-Muller code, which are the known list-decodable codes.

The “predictable degree property”, a terminology introduced by Forney in 1970, is a property of polynomial matrices over a field F that has proven itself to be fundamentally useful for a range of applications. In this paper we strengthen this property into the “predictable leading monomial” property, and show that this PLM property is shared by minimal Gröbner bases for any positional term order (here: TOP and POT) in F[x]q. The property is useful particularly for minimal interpolation-type problems. Because of the presence of zero divisors, minimal Gröbner bases over a finite ring of the type ℤpr (where p is a prime integer and r is an integer > 1) do not have the PLM property. We show how to construct, from an ordered minimal Gröbner basis, a so-called minimal Gröbner p-basis that does have a PLM property. The parametrization of all shortest linear recurrence relations of a finite sequence over ℤpr is a type of problem for which this is useful and we include an illustrative example.

For (n, k, m) systematic polynomial convolutional encoders, there exists an upperbound on the length of a correctable burst of erasures in terms of code parameters. In this paper, we restrict ourselves to the case k = n - 1 and provide a necessary and sufficient condition to achieve the upperbound in terms of the encoder coefficients. In addition, for selected values of m, we present explicit (n, n - 1, m) systematic polynomial convolutional encoders that achieve the upperbound.

Convolutional codes are considered with code sequences modeled as semi-infinite Laurent series. It is well known that a convolutional codeCover a finite groupGhas a minimal trellis representation that can be derived from code sequences. It is also well known that, for the case thatGis a finite field, any polynomial encoder ofCcan be algebraically manipulated to yield a minimal polynomial encoder whose controller canonical realization is a minimal trellis. In this paper we seek to extend this result to the finite ring caseG= \BBZprby introducing a so-called ldquop-encoderrdquo. We show how to manipulate a polynomial encoding scheme of a noncatastrophic convolutional code over\BBZprto produce a particular type ofp-encoder (ldquominimalp-encoderrdquo) whose controller canonical realization is a minimal trellis with nonlinear features. The minimum number of trellis states is then expressed aspgamma, wheregammais the sum of the row degrees of the minimalp-encoder. In particular, we show that any convolutional code over\BBZpradmits a delay-freep-encoder which implies the novel result that delay-freeness is not a property of the code but of the encoder, just as in the field case. We conjecture that a similar result holds with respect to catastrophicity, i.e., any catastrophic convolutional code over\BBZpradmits a noncatastrophicp-encoder.

The theory of algebraic-geometric codes with respect to the Hamming metric has been well developed. However, in many applications where non-binary signals are transmitted or stored the Lee metric is a more appropriate metric than the Hamming metric. In our previous work, we presented a polynomial-time Lee-metric decoding algorithm for algebraic-geometricable codes. Our algorithm generalizes the interpolation-based Lee-metric decoding algorithm for Reed-Solomon codes in the literature. In this paper, we derive an explicit upper bound on the Lee-error correcting radius of our decoding algorithm. The bound also applies to the Lee-metric Reed-Solomon decoding. As far as we know no such explicit bound is available in the literature.

Let F/sub q/[X/sub 1/,...,X/sub m/] denote the set of polynomials over F/sub q/ in m variables, and F/sub q/[X/sub 1/,...,X/sub m/]/sub /spl les/u/ denote the subset that consists of the polynomials of total degree at most u. Let H(T) be a nontrivial polynomial in T with coefficients in F/sub q/[X/sub 1/,...,X/sub m/]. A crucial step in interpolation-based list decoding of q-ary Reed-Muller (RM) codes is finding the roots of H(T) in F/sub q/[X/sub 1/,...,X/sub m/]/sub /spl les/u/. In this correspondence, we present an efficient root-finding algorithm, which finds all the roots of H(T) in F/sub q/[X/sub 1/,...,X/sub m/]/sub /spl les/u/. The algorithm can be used to speed up the list decoding of RM codes.

This paper describes the algebraic-geometric (AG) codes for the Lee metric and derives a lower bound for the minimum Lee distance of AG codes. A Lee-metric decoding algorithm for AG codes is also discussed. This algorithm gives a performance-complexity, which achieves an error-correcting capability.

In this paper, the Sudan-Guruswami approach to list decoding of Reed-Solomon (RS) codes is cast in a system-theoretic framework. With the data, a set of trajectories or time series is associated which is then modeled as a so-called behavior. In this way, a connection is made with the behavioral approach to system theory. It is shown how a polynomial representation of the modeling behavior gives rise to the bivariate interpolating polynomials of the Sudan-Guruswami approach. The concept of "weighted row reduced" is introduced and used to achieve minimality. Two decoding methods are derived and a parametrization of all bivariate interpolating polynomials is given.