Integrated sensing and communication (ISAC) represents a key paradigm for future wireless networks. However, existing approaches often require waveform modifications, dedicated pilots, or additional overhead that complicates standards integration. We propose “sensing for free”—performing multi-source localization without pilots by reusing random and unknown uplink data symbols, where sensing happens simultaneously with data transmission, making it directly compatible with the 3GPP 5G NR and 6G specifications. With the ever-increasing number of devices in dense 6G networks, this approach becomes particularly compelling when combined with sparse arrays, which can localize a much larger number of sources compared to uniform arrays through the enlarged virtual array. However, existing pilot-free multi-source localization algorithms for sparse arrays have numerous drawbacks. They mostly first reconstruct an extended covariance matrix and then apply subspace methods, which incur prohibitive cubic complexity while being limited to second-order statistics. Performance degrades under non-Gaussian modulated data symbols in cellular wireless networks as the higher-order statistics that could further enhance the localization capability remain unexploited. We address these challenges with an attention-only transformer that directly processes raw signal snapshots for grid-less end-to-end direction-of-arrival (DOA) estimation. The model efficiently captures higher-order statistics while being permutation-invariant and adaptive to varying numbers of snapshots. Our algorithm greatly outperforms state-of-the-art artificial intelligence (AI)-based benchmarks with over 30× reduction in parameters and runtime, and enjoys excellent generalization under practical mismatches. In addition, it can effectively handle multipath propagation and mixed modulation types. Beyond localization, our algorithm can also enhance multi-user MIMO beam training through angular reciprocity. The estimated DOAs in the uplink data transmission stage can significantly prune downlink beam sweeping candidates and enhance system throughput via sensing-assisted beam management. Overall, this work demonstrates how reusing existing random data payloads for sensing can enhance both multi-source localization and beam management, two key AI-for-communication use cases in 3GPP efforts towards 6G.

Recurrent Neural Networks (RNNs) are powerful models for sequential tasks, but their training can be computationally intensive. On the other hand, Echo State Networks (ESNs), a specific RNN architecture, simplify this by configuring a fixed, random reservoir of recurrent weights. Instead of setting random weights as in the ESN case, in this work, we investigate the recurrent weight configuration problem of the fully-fledged RNN for the task of Linear Time-Invariant (LTI) system approximation. Our investigation focuses on a specific RNN architecture with a network of recurrent neurons limited to self-loops and linear activation. We demonstrate that as the recurrent weights of this RNN are trained on large datasets, their distribution converges to a near-identical match of an optimal distribution that can be analytically derived using the available domain knowledge of the LTI system. This insight establishes that domain-informed weight configuration is a highly efficient alternative to data-driven training. Building upon this, we propose a novel deterministic algorithm to set the recurrent weights, which significantly improves approximation accuracy. Numerical results show our domain-informed RNN weight configuration achieves up to a four-order-of-magnitude performance gain over conventional ESNs.

Anomaly-Based Intrusion Detection Systems (IDSs) have been extensively researched for their ability to detect zero-day attacks. These systems establish a baseline of normal behavior using benign traffic data and flag deviations from this norm as potential threats. They generally experience higher false alarm rates than signature-based IDSs. Unlike image data, where the observed features provide immediate utility, raw network traffic necessitates additional processing for effective detection. It is challenging to learn useful patterns directly from raw traffic data or simple traffic statistics (e.g., connection duration, package inter-arrival time) as the complex relationships are difficult to distinguish. Therefore, some feature engineering becomes imperative to extract and transform raw data into new feature representations that can directly improve the detection capability and reduce the false positive rate. We propose a geometric feature learning method to optimize the feature extraction process. We employ contrastive feature learning to learn a feature space where normal traffic instances reside in a compact cluster. We further utilize H-Score feature learning to maximize the compactness of the cluster representing the normal behavior, enhancing the subsequent anomaly detection performance. Our evaluations using the NSL-KDD and N-BaloT datasets demonstrate that the proposed IDS powered by feature learning can consistently outperform state-of-the-art anomaly-based IDS methods by significantly lowering the false positive rate. Furthermore, we deploy the proposed IDS on a Raspberry Pi 4 and demonstrate its applicability on resource-constrained Internet of Things (IoT) devices, highlighting its versatility for diverse application scenarios.

Linguistic steganography aims to conceal information within natural language text without being detected. An effective steganography approach should encode the secret message into a minimal number of language tokens while preserving the natural appearance and fluidity of the stego-texts. We present a new framework to enhance the embedding efficiency of stego-texts generated by modifying the output of a large language model (LLM). The novelty of our approach is in abstracting the sequential steganographic embedding process as a Constrained Markov Decision Process (CMDP), which takes into consideration the long-term dependencies instead of merely the immediate effects. We constrain the solution space such that the discounted accumulative total variation divergence between the selected probability distribution and the original distribution given by the LLM is below a threshold. To find the optimal policy, we first show that the functional optimization problem can be simplified to a convex optimization problem with a finite number of variables. A closed-form solution for the optimal policy is then presented to this equivalent problem. It is remarkable that the optimal policy is deterministic and resembles water-filling in some cases. The solution suggests that usually adjusting the probability distribution for the state that has the least random transition probability should be prioritized, but the choice should be made by taking into account the transition probabilities at all states instead of only the current state.

Open Radio Access Network (O-RAN) architectures enhance flexibility for 6G and NextG networks. However, it also brings significant challenges in O-RAN testing with evaluating abundant, high-dimensional key performance indicators (KPIs). In this paper, we introduce a novel two-stage framework to learn temporally-aware low-dimensional representations of O-RAN testing KPIs. To be specific, stage one employs an information-theoretic H-score to train a hybrid self-attentive transformer and echo state network (ESN) reservoir, called Transformer-ESN, capturing temporal dynamics and producing task-aligned 8-dimensional embeddings. Stage two evaluates these embeddings by training a lightweight multilayer perceptron (MLP) predictor exclusively on them for key target KPIs such as reference signal received quality (RSRQ) and spectral efficiency. Using real-world O-RAN testbed data (video streaming with interference), our approach demonstrates a significant advantage specifically when training samples are very limited. In this scenario, the low-dimensional representations learned from the Transformer-ESN yield mean square error (MSE) reductions of up to 41.9% for RSRQ and 29.9% for spectral efficiency compared to predictions from the original high-dimensional data. The framework exhibits high efficiency for O-RAN testing, significantly reducing testing complexities for O-RAN systems.

Deep learning is making a profound impact in the physical layer of wireless communications. Despite exhibiting outstanding empirical performance in tasks such as MIMO receive processing, the reasons behind the demonstrated superior performance improvement remain largely unclear. In this work, we advance the field of Explainable AI (xAI) in the physical layer of wireless communications utilizing signal processing principles. Specifically, we focus on the task of MIMO-OFDM receive processing (e.g., symbol detection) using reservoir computing (RC), a framework within recurrent neural networks (RNNs), which outperforms both conventional and other learning-based MIMO detectors. Our analysis provides a signal processing-based, first-principles understanding of the corresponding operation of the RC. Building on this fundamental understanding, we are able to systematically incorporate the domain knowledge of wireless systems (e.g., channel statistics) into the design of the underlying RNN by directly configuring the untrained RNN weights for MIMO-OFDM symbol detection. The introduced RNN weight configuration has been validated through extensive simulations demonstrating significant performance improvements. This establishes a foundation for explainable RC-based architectures in MIMO-OFDM receive processing and provides a roadmap for incorporating domain knowledge into the design of neural networks for NextG systems.

In this paper, we introduce StructNet-CE, a novel real-time online learning framework for MIMO-OFDM channel estimation, which only utilizes over-the-air (OTA) reference signals (RS) for online channel estimation on a slot basis without assuming the availability of any channel knowledge. To achieve real-time and efficient channel learning, the design of StructNet-CE leverages the structural information inherent in the MIMO-OFDM system: the repetitive structure of modulation constellation and the invariant property of symbol classification to inter-stream interference. The embedded structural information enables StructNet-CE to conduct channel estimation through the underlying symbol detection task and accurately learn MIMO channels through the limited RS with the scattered RS configuration adopted in 5G/5G-Advanced slots. Numerical experiments demonstrate that the channel estimation performance is significantly improved by incorporating the structural knowledge, achieving a mean square error (MSE) reduction ranging from around 44.41% to 95.54% compared to existing methods. Furthermore, StructNet-CE is compatible and readily applicable to current and future wireless networks, demonstrating the effectiveness, importance, and relevance of combining machine learning techniques with domain knowledge for wireless systems.

Orthogonal time frequency space (OTFS) is a promising modulation scheme for wireless communication in high-mobility scenarios. Recently, a reservoir computing (RC) based approach has been introduced for online subframe-based symbol detection in the OTFS system, where only the limited over-the-air (OTA) pilot symbols are utilized for training. However, the previous RC-based approach does not design the RC architecture based on the properties of the OTFS system to fully unlock the potential of RC. This paper introduces a novel two-dimensional RC (2D-RC) approach for online symbol detection on a subframe basis in the OTFS system. The 2D-RC is designed to have a two-dimensional (2D) filtering structure to equalize the 2D circular channel effect in the delay-Doppler (DD) domain of the OTFS system. With the introduced architecture, the 2D-RC can operate in the DD domain with only a single neural network, unlike our previous work which requires multiple RCs to track channel variations in the time domain. Experimental results demonstrate the advantages of the 2D-RC approach over the previous RC-based approach and the compared model-based methods across different modulation orders.

Computing eigenvalue decomposition (EVD) of a given linear operator, or finding its leading eigenvalues and eigenfunctions, is a fundamental task in many machine learning and scientific simulation problems. For high-dimensional eigenvalue problems, training neural networks to parameterize the eigenfunctions is considered as a promising alternative to the classical numerical linear algebra techniques. This paper proposes a new optimization framework based on the low-rank approximation characterization of a truncated singular value decomposition, accompanied by new techniques called nesting for learning the top-$L$ singular values and singular functions in the correct order. The proposed method promotes the desired orthogonality in the learned functions implicitly and efficiently via an unconstrained optimization formulation, which is easy to solve with off-the-shelf gradient-based optimization algorithms. We demonstrate the effectiveness of the proposed optimization framework for use cases in computational physics and machine learning.

We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of$m$samples containing both variables and$n$samples missing one fixed variable. We adopt the minimax framework with$l_{p}^{p}$loss functions. Recent work established that univariate minimax estimator combinations achieve minimax risk with the optimal first-order constant for$p\geq 2$in the regime$m=o(n)$, questions remained for$p\leq 2$and various$f$-divergences. In our study, we affirm that these composite estimators are indeed minimax optimal for$l_{p}^{p}$loss functions, specifically for the range$1\leq p\leq 2$, including the critical$l_{1}$loss. Additionally, we ascertain their optimality for a suite of$f$-divergences, such as KL,$\chi^{2}$, Squared Hellinger, and Le Cam divergences.

Automated spectrum monitoring necessitates the accurate detection of low probability of intercept (LPI) radio frequency (RF) anomaly signals to identify unwanted interference in wireless networks. However, detecting these unforeseen low-power RF signals is fundamentally challenging due to the scarcity of labeled RF anomaly data. In this paper, we introduce WANDA (Wireless ANomaly Detection Algorithm), an automated framework designed to detect LPI RF anomaly signals in low signal-to-interference ratio (SIR) environments without relying on labeled data. WANDA operates through a two-step process: (i) Information extraction, where a convolutional neural network (CNN) utilizing soft Hirschfeld-Gebelein-Rényi correlation (HGR) as the loss function extracts informative features from RF spectrograms; and (ii) Anomaly detection, where the extracted features are applied to a one-class support vector machine (SVM) classifier to infer RF anomalies. To validate the effectiveness of WANDA, we present a case study focused on detecting unknown Bluetooth signals within the WiFi spectrum using a practical dataset. Experimental results demonstrate that WANDA outperforms other methods in detecting anomaly signals across a range of SIR values (-10 dB to 20 dB).

We present a novel framework for learning system design with neural feature extractors. First, we introduce the feature geometry, which unifies statistical dependence and feature representations in a function space equipped with inner products. This connection defines function-space concepts on statistical dependence, such as norms, orthogonal projection, and spectral decomposition, exhibiting clear operational meanings. In particular, we associate each learning setting with a dependence component and formulate learning tasks as finding corresponding feature approximations. We propose a nesting technique, which provides systematic algorithm designs for learning the optimal features from data samples with off-the-shelf network architectures and optimizers. We further demonstrate multivariate learning applications, including conditional inference and multimodal learning, where we present the optimal features and reveal their connections to classical approaches.

The limited over-the-air (OTA) pilot symbols in multiple-input-multiple-output orthogonal-frequency-division-multiplexing (MIMO-OFDM) systems presents a major challenge for detecting transmitted data symbols at the receiver, especially for machine learning-based approaches. While it is crucial to explore effective ways to exploit pilots, one can also take advantage of the data symbols to improve detection performance. Thus, this paper introduces an online attention-based approach, namely RC-AttStructNet-DF, that can efficiently utilize pilot symbols and be dynamically updated with the detected payload data using the decision feedback (DF) mechanism. Reservoir computing (RC) is employed in the time domain network to facilitate efficient online training. The frequency domain network adopts the novel 2D multi-head attention (MHA) module to capture the time and frequency correlations, and the structural-based StructNet to facilitate the DF mechanism. The attention loss is designed to learn the frequency domain network. The DF mechanism further enhances detection performance by dynamically tracking the channel changes through detected data symbols. The effectiveness of the RC-AttStructNet-DF approach is demonstrated through extensive experiments in MIMO-OFDM and massive MIMO-OFDM systems with different modulation orders and under various scenarios.

Orthogonal time frequency space (OTFS) is a promising modulation scheme for wireless communication in high-mobility scenarios. Recently, a reservoir computing (RC) based approach has been introduced for online subframe-based symbol detection in the OTFS system, where only a limited number of over-the-air (OTA) pilot symbols are utilized for training. However, this approach does not leverage the domain knowledge specific to the OTFS system to fully unlock the potential of RC. This paper introduces a novel two-dimensional RC (2D-RC) method that incorporates the domain knowledge of the OTFS system into the design for symbol detection in an online subframe-based manner. Specifically, as the channel interaction in the delay-Doppler (DD) domain is a two-dimensional (2D) circular operation, the 2D-RC is designed to have the 2D circular padding procedure and the 2D filtering structure to embed this knowledge. With the introduced architecture, 2D-RC can operate in the DD domain with only a single neural network, instead of necessitating multiple RCs to track channel variations in the time domain as in previous work. Numerical experiments demonstrate the advantages of the 2D-RC approach over the previous RC-based approach and compared model-based methods across different OTFS system variants and modulation orders.

A deep autoencoder (DAE)-based end-to-end communication over the two-user Z-interference channel (ZIC) with finite-alphabet inputs is designed in this paper. The design is for imperfect channel state information (CSI) where both estimation and quantization errors exist. The proposed structure jointly optimizes the encoders and decoders to generate interference-aware constellations that adapt their shape to the interference intensity in order to minimize the bit error rate. A normalization layer is designed to guarantee an average power constraint in the DAE while allowing the architecture to generate constellations with nonuniform shapes. This brings further shaping gain compared to standard uniform constellations such as quadrature amplitude modulation. The performance of the DAE-ZIC is compared with two conventional methods, i.e., standard and rotated constellations. The proposed structure significantly enhances the performance of the ZIC. Simulation results confirm bit error rate reduction in all interference regimes (weak, moderate, and strong). At a signal-to-noise ratio of 20dB, the improvements reach about two orders of magnitude when only quantization error exists, indicating that the DAE-ZIC is highly robust to the interference compared to the conventional methods.

We study the problem of estimating the joint probability mass function (pmf) over two random variables. In particular, the estimation is based on the observation of m samples containing both variables and n samples missing one fixed variable. We adopt the minimax framework with $l_p^p$ loss functions, and we show that the composition of uni-variate minimax estimators achieves minimax risk with the optimal first-order constant for p ≥ 2, in the regime m = o(n).

We study kernel methods in machine learning from the perspective of feature subspace. We establish a one-to-one correspondence between feature subspaces and kernels and propose an information-theoretic measure for kernels. In particular, we construct a kernel from Hirschfeld–Gebelein–Rényi maximal correlation functions, coined the maximal correlation kernel, and demonstrate its information-theoretic optimality. We use the support vector machine (SVM) as an example to illustrate a connection between kernel methods and feature extraction approaches. We show that the kernel SVM on maximal correlation kernel achieves minimum prediction error. Finally, we interpret the Fisher kernel as a special maximal correlation kernel and establish its optimality.

In this paper, we develop a learning-based symbol detection algorithm for massive MIMO-OFDM systems. To exploit the structure information inherited in the received signals from massive antenna array, multi-mode reservoir computing is adopted as the building block to facilitate over-the-air training in time domain. In addition, alternating recursive least square optimization method, and decision feedback mechanism are utilized in our algorithm to achieve the real-time learning capability. That is, the neural network is trained purely online with its weights updated on an OFDM symbol basis to promptly and adaptively track the dynamic environment. Furthermore, an online learning-based module is devised to compensate the nonlinear distortion caused by RF circuit components. On top of that, a learning-efficient classifier named StructNet is introduced in frequency domain to further improve the symbol detection performance by utilizing the QAM constellation structural pattern. Evaluation results demonstrate that our algorithm achieves substantial gain over traditional model-based approach and state-of-the-art learning-based techniques under dynamic channel environment and RF circuit nonlinear distortion. Moreover, empirical result reveals our NN model is robust to training label error, which benefits the decision feedback mechanism.

In this paper, we introduce a structure-based neural network architecture, namely RC-Struct, for MIMO-OFDM symbol detection. The RC-Struct exploits the temporal structure of the MIMO-OFDM signals through reservoir computing (RC). A binary classifier leverages the repetitive constellation structure in the system to perform multi-class detection. The incorporation of RC allows the RC-Struct to be learned in a purely online fashion with extremely limited pilot symbols in each OFDM subframe. The binary classifier enables the efficient utilization of the precious online training symbols and allows an easy extension to high-order modulations without a substantial increase in complexity. Experiments show that the introduced RC-Struct outperforms both the conventional model-based symbol detection approaches and the state-of-the-art learning-based strategies in terms of bit error rate (BER). The advantages of RC-Struct over existing methods become more significant when rank and link adaptation are adopted. The introduced RC-Struct sheds light on combining communication domain knowledge and learning-based receive processing for 5G/5G-Advanced and Beyond.

Current transfer learning algorithm designs mainly focus on the similarities between source and target tasks, while the impacts of the sample sizes of these tasks are often not sufficiently addressed. This paper proposes a mathematical framework for quantifying the transferability in multi-source transfer learning problems, with both the task similarities and the sample complexity of learning models taken into account. In particular, we consider the setup where the models learned from different tasks are linearly combined for learning the target task, and use the optimal combining coefficients to measure the transferability. Then, we demonstrate the analytical expression of this transferability measure, characterized by the sample sizes, model complexity, and the similarities between source and target tasks, which provides fundamental insights of the knowledge transferring mechanism and the guidance for algorithm designs. Furthermore, we apply our analyses for practical learning tasks, and establish a quantifiable transferability measure by exploiting a parameterized model. In addition, we develop an alternating iterative algorithm to implement our theoretical results for training deep neural networks in multi-source transfer learning tasks. Finally, experiments on image classification tasks show that our approach outperforms existing transfer learning algorithms in multi-source and few-shot scenarios.

While the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation and the Wyner common information share similar information processing purposes of extracting common knowledge structures between random variables, the relationships between these approaches are generally unclear. In this paper, we demonstrate such relationships by considering the Wyner common information in the weakly dependent regime, called ε-common information. We show that the HGR maximal correlation functions coincide with the relative likelihood functions of estimating the auxiliary random variables in ε-common information, which establishes the fundamental connections these approaches. Moreover, we extend the ε-common information to multiple random variables, and derive a novel algorithm for extracting feature functions of data variables regarding their common information. Our approach is validated by the MNIST problem, and can potentially be useful in multi-modal data analyses.

A modal decomposition is a useful tool that deconstructs the statistical dependence between two random variables by decomposing their joint distribution into orthogonal modes. Historically, modal decompositions have played important roles in statistics and information theory, e.g., in the study of maximal correlation. They are defined using the singular value decompositions of divergence transition matrices (DTMs) and conditional expectation operators corresponding to joint distributions. In this paper, we first characterize the set of all DTMs, and illustrate how the associated conditional expectation operators are the only weak contractions among a class of natural candidates. While modal decompositions have several modern machine learning applications, such as feature extraction from categorical data, the sample complexity of estimating them in such scenarios has not been analyzed. Hence, we also establish some non-asymptotic sample complexity results for the problem of estimating dominant modes of an unknown joint distribution from training data.

One primary focus in multimodal feature extraction is to find the representations of individual modalities that are maximally correlated. As a well-known measure of dependence, the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation be-´ comes an appealing objective because of its operational meaning and desirable properties. However, the strict whitening constraints formalized in the HGR maximal correlation limit its application. To address this problem, this paper proposes Soft-HGR, a novel framework to extract informative features from multiple data modalities. Specifically, our framework prevents the “hard” whitening constraints, while simultaneously preserving the same feature geometry as in the HGR maximal correlation. The objective of Soft-HGR is straightforward, only involving two inner products, which guarantees the efficiency and stability in optimization. We further generalize the framework to handle more than two modalities and missing modalities. When labels are partially available, we enhance the discriminative power of the feature representations by making a semi-supervised adaptation. Empirical evaluation implies that our approach learns more informative feature mappings and is more efficient to optimize.

Task transfer learning is a popular technique in image processing applications that uses pre-trained models to reduce the supervision cost of related tasks. An important question is to determine task transferability, i.e. given a common input domain, estimating to what extent representations learned from a source task can help in learning a target task. Typically, transferability is either measured experimentally or inferred through task relatedness, which is often defined without a clear operational meaning. In this paper, we present a novel metric, H-score, an easily-computable evaluation function that estimates the performance of transferred representations from one task to another in classification problems using statistical and information theoretic principles. Experiments on real image data show that our metric is not only consistent with the empirical transferability measurement, but also useful to practitioners in applications such as source model selection and task transfer curriculum learning.

It is commonly believed that the hidden layers of deep neural networks (DNNs) attempt to extract informative features for learning tasks. In this paper, we formalize this intuition by showing that the features extracted by DNN coincide with the result of an optimization problem, which we call the "universal feature selection" problem, in a local analysis regime. We interpret the weights training in DNN as the projection of feature functions between feature spaces, specified by the network structure. Our formulation has direct operational meaning in terms of the performance for inference tasks, and gives interpretations to the internal computation results of DNNs. Results of numerical experiments are provided to support the analysis.

We propose a non-adaptive unequal error protection (UEP) querying policy based on superposition coding for the noisy 20 questions problem. In this problem, a player wishes to successively refine an estimate of the value of a continuous random variable by posing binary queries and receiving noisy responses. When the queries are designed non-adaptively as a single block and the noisy responses are modeled as the outputs of a binary symmetric channel the 20 questions problem can be mapped to an equivalent problem of channel coding with UEP. A new non-adaptive querying strategy based on UEP superposition coding is introduced whose estimation error decreases with an exponential rate of convergence that is significantly better than that of the UEP repetition coding introduced by Variani et al. (2015). In fact, we show that the proposed non-adaptive UEP querying policy achieves the same order convergence rate as the adaptive policy.

In this paper, we formulate the generalized softmax learning (GSL) problem, as a symmetric extension of the softmax regression problem. We further study the geometric structure of GSL and demonstrate the equivalence of GSL and the original softmax regression problem. Besides, this geometric structure indicates the symmetry between a neural network and its reverse network, and the symmetric roles of the weights and feature in a neural network. Finally, we present a numerical simulation to verify these symmetry properties in neural networks.

We address the problem of optimal feature selection for a Gaussian vector pair in the weak dependence regime, when the inference task is not known in advance. In particular, we show that multiple formulations all yield the same solution, and correspond to the singular value decomposition (SVD) of the canonical correlation matrix. Our results reveal key connections between canonical correlation analysis (CCA), principal component analysis (PCA), the Gaussian information bottleneck, Wyner's common information, and the Ky Fan (nuclear) norms.

In this paper, we propose an open-loop unequal-error-protection querying policy based on superposition coding for the noisy 20 questions problem. In this problem, a player wishes to successively refine an estimate of the value of a continuous random variable by posing binary queries and receiving noisy responses. When the queries are designed non-adaptively as a single block and the noisy responses are modeled as the output of a binary symmetric channel, the 20 questions problem can be mapped to an equivalent problem of channel coding with unequal error protection (UEP). A new non-adaptive querying strategy based on UEP superposition coding is introduced, whose estimation error decreases with an exponential rate of convergence that is significantly better than that of the UEP repetition coding introduced by Variani et al. (2015). With the proposed querying strategy, the rate of exponential decrease in the number of queries matches the rate of a closed-loop adaptive scheme, where queries are sequentially designed with the benefit of feedback. Furthermore, the achievable error exponent is significantly better than that of random block codes employing equal error protection.

We develop an information theoretic framework for addressing feature selection in applications where the inference task is not specified in advance and the data is from a large alphabet. We introduce a natural notion of universality for such problems, and show that locally optimal solutions are straight forward to obtain, admit natural interpretations via information geometry, have computationally efficient implementations, and represent a practically useful learning methodology. Our development also reveals the key role of Hirschfeld-Gebelein-Renyi maximal correlation and the alternating conditional expectations (ACE) algorithm in such problems.

In this paper, we model the unsupervised learning of a sequence of observed data vector as a problem of extracting joint patterns among random variables. In particular, we formulate an information-theoretic problem to extract common features of random variables by measuring the loss of total correlation given the feature. This problem can be solved by a local geometric approach, where the solutions can be represented as singular vectors of some matrices related to the pairwise distributions of the data. In addition, we illustrate how these solutions can be transferred to feature functions in machine learning, which can be computed by efficient algorithms from data vectors. Moreover, we present a generalization of the HGR maximal correlation based on these feature functions, which can be viewed as a nonlinear generalization to linear PCA. Finally, the simulation result shows that our extracted feature functions have great performance in real-world problems.

In this paper, we show that the conditional expectation operators corresponding to a family of source-channel models, defined by natural exponential families with quadratic variance functions and their conjugate priors, have orthonormal polynomials as singular vectors. These models include the Gaussian channel with Gaussian source, the Poisson channel with gamma source, and the binomial channel with beta source. To derive the singular vectors of these models, we prove and employ the equivalent condition that their conditional moments are strictly degree preserving polynomials.

Statistical learning has attracted considerable recent research interest due to the wide-ranging demands of big data analytics. The recent introduction of communication theory and information coupling theory into this area suggests a new perspective on statistical learning and inference for data analytics. This paper investigates inference of one data variable from heterogeneous data variables, a problem that plays an increasingly important role in the emerging applications of big data analytics. To generalize the existing conceptual approach, information coupling filtering under hidden data structure or unknown knowledge of interactions among data variables is developed. A least-mean-squares (LMS) filtering approach for non-stationary data similar to an equalizer is suggested, while the training data gives the depth of the filter analogously to model selection in learning theory. The information combining in diversity communication is extended to fuse more data variables for even greater precision of inference. Extending from multiuser detection, an algorithm based on Multiple Signal Classification (MUSIC) is demonstrated to identify useful data variables for inference, as a novel solution to knowledge discovery. A series of examples illustrate the effectiveness of this framework, suggesting that statistical communication theory and statistical signal processing can substantially contribute to statistical learning theory.

In this paper, we propose an unequal error protection coding strategy based on superposition coding for the noisy 20 questions problem. In this problem, a player wishes to successively refine an estimate of the value of a continuous random variable by posing binary queries and receiving noisy responses. When the queries are designed non-adaptively as a single block and the noisy responses are modeled as the output of a binary symmetric channel the 20 questions problem can be mapped to an equivalent problem of channel coding with unequal error protection (UEP). A superposition coding strategy with UEP is introduced that has error exponent that is significantly better than that of the UEP repetition code introduced by Variani et al. [1].

The maximum rate at which classical information can be reliably transmitted per use of a quantum channel strictly increases in general with N , the number of channel outputs that are detected jointly by the quantum joint-detection receiver (JDR). This phenomenon is known as superadditivity of the maximum achievable information rate over a quantum channel. We study this phenomenon for a pure-state classical-quantum channel and provide a lower bound on CN/N, the maximum information rate when the JDR is restricted to making joint measurements over no more than N quantum channel outputs, while allowing arbitrary classical error correction. We also show the appearance of a superadditivity phenomenon-of mathematical resemblance to the aforesaid problem-in the channel capacity of a classical discrete memoryless channel when a concatenated coding scheme is employed, and the inner decoder is forced to make hard decisions on N -length inner codewords. Using this correspondence, we develop a unifying framework for the above two notions of superadditivity, and show that for our lower bound to CN/N to be equal to a given fraction of the asymptotic capacity C of the respective channel, N must be proportional to V/C2, where V is the respective channel dispersion quantity.

This paper considers the problem of communication over a discrete memoryless channel (DMC) or an additive white Gaussian noise (AWGN) channel subject to the constraint that the probability that an adversary who observes the channel outputs can detect the communication is low. In particular, the relative entropy between the output distributions when a codeword is transmitted and when no input is provided to the channel must be sufficiently small. For a DMC whose output distribution induced by the “off” input symbol is not a mixture of the output distributions induced by other input symbols, it is shown that the maximum amount of information that can be transmitted under this criterion scales like the square root of the blocklength. The same is true for the AWGN channel. Exact expressions for the scaling constant are also derived.

In this paper, we investigate how to efficiently extract informative features of high-dimensional data through noisy channels. Specifically, we decompose the feature space of the data into a sequence of score functions with decreasing information volumes, such that different scores are uncorrelated. From this decomposition, the features of the data become a sequence of score functions such that the most informative lowdimensional feature can be selected as the first few scores. This greatly simplifies the feature selection problem. In addition, we apply this spectrum decomposition to data with high-dimensional structures, i.e., the hidden Markov model (HMM). We show that in HMM, it is desirable to consider a particular class of score functions called as the node scores, which allows us to efficiently extract informative features of the hidden variables by applying the spectrum decomposition approach. Finally, we develop efficient algorithms to extract such features from node scores, and present an example to illustrate the performance of the node scores.

This paper considers the problem of communication over a discrete memoryless channel subject to the constraint that the probability that an adversary who observes the channel outputs can detect the communication is low. Specifically, the relative entropy between the output distributions when a codeword is transmitted and when no input is provided to the channel must be sufficiently small. For a channel whose output distribution induced by the zero input symbol is not a mixture of the output distributions induced by other input symbols, it is shown that the maximum number of bits that can be transmitted under this criterion scales like the square root of the blocklength. Exact expressions for the scaling constant are also derived.

Widespread use of the Internet and social networks invokes the generation of big data, which is proving to be useful in a number of applications. To deal with explosively growing amounts of data, data analytics has emerged as a critical technology related to computing, signal processing, and information networking. In this paper, a formalism is considered in which data are modeled as a generalized social network and communication theory and information theory are thereby extended to data analytics. First, the creation of an equalizer to optimize information transfer between two data variables is considered, and financial data are used to demonstrate the advantages of this approach. Then, an information coupling approach based on information geometry is applied for dimensionality reduction, with a pattern recognition example to illustrate the effectiveness of this formalism. These initial trials suggest the potential of communication theoretic data analytics for a wide range of applications.

In this paper, we propose a generalized analog network coding (GANC) scheme for a non-orthogonal multiple-access relay channel (MARC), where two sources transmit their information simultaneously to the destination with the help of a relay. In the GANC scheme, the relay receives interfered signals from the two sources and generates signals to be transmitted with a relay function. We focus on the design of the optimal relay function to achieve the minimum pair-wise error probability (PEP) of the system. Specifically, we first covert the relay function optimization problem to a transformation matrix (TM) design problem by presenting the received complex signals as signal matrices composed of real and imaginary parts. Then, we propose an optimization criteria, i.e.,maximizing the minimal squared Euclidean distance(MMSED), to improve the PEP performance, since the PEP is determined by the Euclidean distance of the received constellation at destination. Next, we prove that the MMSED can be equivalently converted to a convex problem by introducing an intermediate matrix. We solve this convex problem by using the Lagrangian method and obtain the closed-form expression of the optimal TM. We further improve the PEP performance by optimizing transmission power of the two sources. Simulation results show that the proposed GANC scheme has a better PEP performance compared to other alternative schemes.

In this paper, we extend the information theoretic framework that was developed in earlier works to multi-hop network settings. For a given network, we construct a novel deterministic model that quantifies the ability of the network in transmitting private and common messages across users. Based on this model, we formulate a linear optimization problem that explores the throughput of a multi-layer network, thereby offering the optimal strategy as to what kind of common messages should be generated in the network to maximize the throughput. With this deterministic model, we also investigate the role of feedback for multi-layer networks, from which we identify a variety of scenarios in which feedback can improve transmission efficiency. Our results provide fundamental guidelines as to how to coordinate cooperation between users to enable efficient information exchanges across them.

Random access has been widely studied in literature, but its time dynamics remains a pretty open research problem at this time, particularly for cognitive radio networks that are operating most in transient status but being investigated usually in steady-state. Modifying prey-predator model, we therefore consider radio resources as preys and users as predators to dynamically understand the network system behavior. We start from exploring ALOHA, then include the sensing mechanism into the scenario. Furthermore, we incorporate partially or randomly connected graph to practically represent realistic interactions among users and resources. By modeling sensing errors and delay, for the first time, the time dynamics of a cognitive radio network can be fully characterized, and consequently random access operating conditions can be practically understood and specified for network engineering design.

We investigate superadditivity in the maximum achievable rate of reliable classical communication over a quantum channel. The maximum number of classical information bits extracted per use of the quantum channel strictly increases as the number of channel outputs jointly measured at the receiver increases. This phenomenon is called superadditivity. We provide an explanation of this phenomenon by comparing a quantum channel with a classical discrete memoryless channel (DMC) under concatenated codes. We also give a lower bound on the maximum accessible information per channel use at a finite length of quantum measurements in terms of V, which is the quantum version of channel dispersion, and C, the classical capacity of the quantum channel.

The problem of secret key agreement by public discussion is studied under a general multiterminal network, where each user can both send and receive over a private channel. Single-letter upper and lower bounds are for the maximum achievable key rate. The bounds are shown to match for a large class of private channels. A counter-example shows that the bounds do not match in general, and a better cooperative scheme can narrow the gap.

In this paper, we extend the information theoretical framework that was developed in [1] to multi-hop communication networks. For a given network, we construct a deterministic model that models the ability of the channels in transmitting private and common messages between users in this network. Based on this model, we formulate a linear optimization problem to study the network throughput, where the solution indicates what kind of common messages should be generated in a network to optimize the throughput. Our results provide fundamental guidelines of how users in a network should cooperate with each other to communicate efficiently.

The bit-wise unequal error protection problem, for the case when the number of groups of bits${\ell}$is fixed, is considered for variable-length block codes with feedback. An encoding scheme based on fixed-length block codes with erasures is used to establish inner bounds to the achievable performance for finite expected decoding time. A new technique for bounding the performance of variable-length block codes is used to establish outer bounds to the performance for a given expected decoding time. The inner and the outer bounds match one another asymptotically and characterize the achievable region of rate-exponent vectors, completely. The single-message message-wise unequal error protection problem for variable-length block codes with feedback is also solved as a necessary step on the way.

Many network information theory problems face the similar difficulty of single letterization. We argue that this is due to the lack of a geometric structure on the space of probability distribution. In this paper, we develop such a structure by assuming that the distributions of interest are close to each other. Under this assumption, the K-L divergence is reduced to the squared Euclidean metric in an Euclidean space. Moreover, we construct the notion of coordinate and inner product, which will facilitate solving communication problems. We will also present the application of this approach to the point-to-point channel and the general broadcast channel, which demonstrates how our technique simplifies information theory problems.

This paper investigates network information theory problems where the external noise is Gaussian distributed. In particular, the Gaussian broadcast channel with coherent fading and the Gaussian interference channel are considered. It is shown that in these problems, non-Gaussian code ensembles can achieve higher rates than the Gaussian ones. It is also shown that the strong Shamai-Laroia conjecture on the Gaussian ISI channel does not hold. In order to analyze non-Gaussian code ensembles over Gaussian networks, a geometrical tool using the Hermite polynomials is proposed. This tool provides a coordinate system to analyze a class of non-Gaussian input distributions that are invariant over Gaussian networks.

A wideband Rayleigh fading channel is considered with causal channel state information (CSI) at the transmitter and no receiver CSI. A simple orthogonal code with energy detection rule at the receiver (similar to pulse position modulation in IEEE Trans. Inf. Theory, vol. 46, no. 4, Apr. 2000 and IEEE Trans. Inf. Theory, vol. 52 no. 5, May 2006) is shown to achieve the capacity of this channel in the wideband limit. This strategy transmits energy only when the channel gain exceeds a threshold, hence only needs causal transmitter CSI. In the wideband limit, this capacity without any receiver CSI is the same as the capacity with full receiver CSI, which is proportional to the logarithm of the bandwidth. Similar threshold-based pulse position modulation is shown to achieve the capacity per unit cost of the dirty-tape channel (dirty paper channel with causal transmitter CSI and no receiver CSI), which equals its capacity per unit cost with full receiver CSI. Then, a general discrete channel with i.i.d. states is considered. Each input has an associated cost and a zero cost input “0” exists. The channel state is assumed to be known at the transmitter in a causal manner. Capacity per unit cost is found for this channel and a simple orthogonal code is shown to achieve this capacity. Later, a novel orthogonal coding scheme is proposed for the case of causal transmitter CSI and a condition for equivalence of capacity per unit cost for causal and noncausal transmitter CSI is derived.

Inner and outer bounds are derived on the optimal performance of fixed-length block codes on discrete memoryless channels with feedback and errors-and-erasures decoding. First, an inner bound is derived using a two-phase encoding scheme with communication and control phases together with the optimal decoding rule for the given encoding scheme, among decoding rules that can be represented in terms of pairwise comparisons between the messages. Then, an outer bound is derived using a generalization of the straight-line bound to errors-and-erasures decoders and the optimal error-exponent tradeoff of a feedback encoder with two messages. In addition, upper and lower bounds are derived, for the optimal erasure exponent of error-free block codes in terms of the rate. Finally, a proof is provided for the fact that the optimal tradeoff between error exponents of a two-message code does not improve with feedback on discrete memoryless channels (DMCs).

With the emergence of the adaptive bit rate (ABR) streaming technology, the video/content streaming technology is shifting toward a file-based content distribution. That is, video content is encoded into a set of smaller media files containing video of 2–10 seconds before transmission. This file-based content distribution, coupled with increasingly rapid adoption of smartphones, requires an efficient file-based distribution algorithm to satisfy the QoS demand in wireless networks. In this paper, we study the transmission of a finite-sized file over wireless networks using multipath routing, with the objective to minimize file transmission delay instead of average packet delay. The file transmission delay is defined as the time interval from the instant that a file is first transmitted to the time at which the file can be reconstructed in the destination node. We observe that file transmission delay depends not only on the mean of the packet delay but also on its distribution, especially the tail. This observation leads to a better understanding of the file transfer delay in wireless networks and a minimum delay file transmission strategy. In a wireless multipath communication scenario, we propose to use packet level erasure code (e.g., digital fountain code) to transmit data file with redundancy. Given that a file with k packets is encoded into n packets for transmission, the use of digital fountain code allows the file to be received when only k out of n packets are received. By adding redundant packets, the destination node does not have to wait for the packet to arrive late, hence reducing the delay of the file transmission. We characterize the tradeoff between the code rate (i.e., the ratio of the number of transmitted packets to the number of the original packets) and the file delay reduction. As a rule of thumb, we provide practical guidelines in determining an appropriate code rate for a fixed file to achieve a reasonable transmission delay. We show that only a few redundant packets are needed to achieve a significant reduction in file transmission delay.

We study the general coherent-state hypothesis testing problem and the capacity of the pure-loss optical channel with a coherent processing receiver (a receiver that uses coherent feedback control and direct detection). We describe the binary hypothesis minimum probability of error receiver as optimizing the communication efficiency at each instant, based on recursively updated knowledge of the receiver. Using this viewpoint, we give a natural generalization of the designs to general M-ary hypothesis testing problems. We analyze the information capacity with coherent receivers, and compare the result with that with direct detection receivers and with arbitrary quantum receivers (the Holevo limit), using the appropriate scalings in the low photon number regime.

We introduce the concept of a fiber aided wireless network architecture (FAWNA), which allows high-speed mobile connectivity by leveraging the speed of optical networks. Specifically, we consider a single-input, multiple-output (SIMO) FAWNA, which consists of a SIMO wireless channel interfaced with an optical fiber channel through wireless-optical interfaces. We propose a design where the received wireless signal at each interface is sampled and quantized before being sent over the fiber. The capacity of our scheme approaches the capacity of the architecture, exponentially with fiber capacity. We also show that for a given fiber capacity, there is an optimal operating wireless bandwidth and number of interfaces. We show that the optimal way to divide the fiber capacity among the interfaces is to ensure that each interface gets enough rate so that its noise is dominated by front end noise rather than by quantizer distortion. We also show that rather than dynamically change rate allocation based on channel state, a less complex, fixed rate allocation scheme can be adopted with very small loss in performance.

We study the role played by queue length information in the operation of flow control and server allocation policies. We first consider a simple model of a single server queue with congestion-based flow control. The input rate at any instant is decided by a flow control policy, based on the queue occupancy. We identify a simple “two-threshold” control policy, which achieves the best possible exponential scaling for the queue congestion probability, for any rate of control. We show that when the control channel is reliable, the control rate needed to ensure the optimal decay exponent for the congestion probability can be made arbitrarily small. However, if control channel erasures occur probabilistically, we show the existence of a critical erasure probability threshold beyond which the congestion probability undergoes a drastic increase due to the frequent loss of control packets. We also determine the optimal amount of error protection to apply to the control signals by using a simple bandwidth sharing model. Finally, we show that the queue length based server allocation problem can also be treated using this framework and that the results obtained for the flow control setting can also be applied to the server allocation case.

A dense antenna array architecture is developed to ease the circuit requirements of the radio frequency (RF) front-end in beamforming applications. In the architecture, antennas are spaced more closely than would otherwise be required to exploit the available degrees of freedom. Such an array structure is analogous to temporally oversampled data conversion systems, which have reduced quantizer resolution requirements. For a linear, uniformly-spaced array, we develop a spatial-domain version of ΔΣ quantization, and show that with binary quantization for the in-phase and quadrature components of antenna weights, even relatively modest amounts of oversampling can reproduce beamforming patterns of interest to practically useful levels of accuracy.

Bit-wise unequal error protection problem with two layers is considered for variable length block-codes with feedback. Inner and outer bounds are derived for achievable performance for finite expected decoding time. These bounds completely characterize the error exponent of the special bits as a function of overall rate R, overall error exponent E and the rate of the special bits Rs. Single message Message-wise unequal protection problem is also solved as a step on the way.

Over discrete memoryless channels (DMC), linear decoders (maximizing additive metrics) afford several nice properties. In particular, if suitable encoders are employed, the use of decoding algorithms with manageable complexities is permitted. For a compound DMC, decoders that perform well without the channel's knowledge are required in order to achieve capacity. Several such decoders have been studied in the literature, however, there is no such known decoder which is linear. Hence, the problem of finding linear decoders achieving capacity for compound DMC is addressed, and it is shown that under minor concessions, such decoders exist and can be constructed. A geometric method based on the very noisy transformation is developed and used to solve this problem.

The Rayleigh flat fading channel at low SNR is considered. With full channel state information (CSI) at the transmitter and receiver, its capacity is shown to be essentially SNR log(1SNR) nats/symbol, as SNR goes to zero. In fact, this rate can be achieved with a just one bit of CSI at the transmitter (per fading realization) and with no receiver CSI. The capacity for the case of noisy transmitter CSI is also found. Then a Rayleigh block fading channel of coherence interval T ≤ 1/ SNR is considered which has causal feedback and no a priori CSI. A training based scheme is proposed for such channels, which achieves a rate of SNR logT nats/symbol in the limit of small SNR and large T. Thus, when coherence interval T is of the order 1/SNR, without any a priori CSI at either end, the capacity with full CSI at both ends is achievable. For smaller values of T, a rate of SNR logT nats/symbol is shown to be achievable.

The goal of this paper is to characterize the tradeoff between the rate of control and network congestion for flow control policies. We consider a simple model of a single server queue with congestion-based flow control. The input rate at any instant is decided by a flow control policy, based on the queue occupancy. We identify a simple 'two threshold' control policy, which achieves the best possible congestion probability, for any rate of control. We show that in the absence of control channel errors, the control rate needed to ensure the optimal decay exponent for the congestion probability can be made arbitrarily small. However, if control channel errors occur probabilistically, we show the existence of a critical error probability threshold beyond which the congestion probability undergoes a drastic increase due to the frequent loss of control packets. Finally, we determine the optimal amount of error protection to apply to the control signals by using a simple bandwidth sharing model.

This paper shows that the capacity achieving input distribution for a fading Gaussian broadcast channel is not Gaussian in general. The construction of non-Gaussian distributions that strictly outperform Gaussian ones, for certain characterized fading distributions, is provided. The ability of analyzing non-Gaussian input distributions with closed form expressions is made possible in a local setting. It is shown that there exists a specific coordinate system, based on Hermite polynomials, which parametrizes Gaussian neighborhoods and which is particularly suitable to study the entropic operators encountered with Gaussian noise.

A new technique is proposed for upper bounding the error probability of fixed length block codes with feedback. Error analysis is inspired by Gallager's error analysis for block codes without feedback. Zigangirov-D'yachkov (Z-D ) encoding scheme is analyzed with the technique on binary input channels and k-ary symmetric channels. A strict improvement is obtained for k-ary symmetric channels.

An information-theoretic framework for unequal error protection is developed in terms of the exponential error bounds. The fundamental difference between thebit-wiseandmessage-wiseunequal error protection (UEP) is demonstrated, for fixed-length block codes on discrete memoryless channels (DMCs) without feedback. Effect of feedback is investigated via variable-length block codes. It is shown that, feedback results in a significant improvement in bothbit-wiseandmessage-wise UEPs(except the single message case for missed detection). The distinction between false-alarm and missed-detection formalizations formessage-wise UEPis also considered. All results presented are at rates close to capacity.

This paper introduces low-complexity block codes, termed pattern-eliminating codes (PEC), which achieve a potentially large performance improvement over channels with residual inter-symbol interference (ISI). The codes are systematic, require no decoding and allow for simple encoding. They operate by prohibiting the occurrence of harmful symbol patterns. On some discrete-time communication channels, the (n, n - 1) PEC can prohibit all occurrences of symbol patterns causing worst- case ISI. The effectiveness of a PEC is shown to be uniquely determined by the sign-signature of the channel response, and a simple criterion is given for identifying channels for which the (n, n - 1) code is effective. It is also shown that for most channel signatures, the (n, n - 1) PEC can be augmented by a (O, n - 1) runlength-limiting (RLL) code at no additional coding overhead. This paper also explores properties of the (n, n-b) PEC for b > 1. The simulation results show that the (n, n - 1) PEC can provide error-rate reductions of several orders of magnitude, even with rate penalty taken into account. It is also shown that channel conditioning, such as equalization, can have a large effect on the code performance and potentially large gains can be derived from optimizing the equalizer jointly with a pattern-eliminating code.

On a discrete memoryless channel (DMC), the maximum likelihood (ML) decoding rule is not only optimal (for equiprobable messages and average error probability), but it is also linear (the metric that it maximizes is additive over the block length), which in particular, makes ML practically conceivable. On a compound DMC, the use of ML is ruled out by the channelpsilas law ignorance. In order to account for this, the universal decoders proposed in (Feder et al., 1998) can be employed. However, none of these decoders is linear. Hence, we consider the problem of finding good linear decoders for compound DMCpsilas. We show that on most compound sets, when universality requires to be capacity achieving, there exists a universal decoding rule which is generalized linear, in the sense that it maximizes only a finite number of additive metrics. A local to global geometric method is developed to solve this problem. By considering very noisy channels, the global problem is reduced, in the limit, to an inner product space problem, for which insightful solutions can be found. We describe a heuristic method used, in this problem, to ldquoliftrdquo local results to global results.

Various formulations are considered where some information is more important than other and needs better protection. Our information theoretic framework in terms of exponential error bounds provides some fundamental limits and optimal strategies for such problems of unequal error protection. Even for data-rates approaching the channel capacity, it shows how a crucial part of information can be protected with exponential reliability. Channels without feedback are analyzed first, which is useful later in analyzing channels with feedback. A new channel parameter, called the Red-Alert Exponent, is fundamentally important in such problems.

Fixed length block codes on discrete memoryless channels with feedback are considered for errors and erasures decoding. Upper and lower bounds are derived for the error exponent in terms of the rate and the erasure exponents. In addition the converse result of Burnashev for variable length block codes is extended to include list decoding.

Transmitter knowledge of channel state has a great impact on wideband fading channel capacity. However, in the low SNR regime, power per dimension does not suffice to provide an accurate measurement of the channel over the entire spectrum. In the presence of feedback, we may collect information at the transmitter about some aspects of the channel quality over a certain portion of the spectrum. In this work, we investigate the effect of such information. We consider channel testing with a finite amount of energy over a block-fading channel in both time and frequency. We consider a transmission scheme in which the wideband channel is decomposed into many parallel narrowband subchannels, each used with a binary modulation scheme. The quality of each subchannel corresponds to the crossover probability of a binary symmetric channel. We use a multi-armed bandit approach to consider the relative costs and benefits of allotting energy for testing versus transmission, and for repeated testing a single subchannel versus testing different subchannels. We give both upper and lower bounds on the number of subchannels that should be probed for throughput maximization under the scheme we have chosen. Our bounds are in terms of available transmission energy, available bandwidth and fading characteristics of the channel. Moreover, in our numerical results, the two bounds are close.

We study the problem of communication reliability of wireless networks in a fading environment based on the outage probability formulation. The exact expression for the disconnect probability, the probability that a transmission by a node is not received correctly by any other node in the network, is obtained for one and two dimensional random networks. We obtain the end-to-end reliability of multi-hop transmission using the outage probability metric and develop algorithms for finding the most reliable route subject to power constraints as well as the minimum energy route subject to a reliability constraint. Finally, we study the tradeoff between outage probability and transmission power, with and without route diversity.

Power efficiency is a capital issue in the study of mobile wireless nodes owing to constraints on their battery size and weight. In practice, especially for low-power nodes, it is often the case that the power consumed for non-transmission processes is not always negligible. In this paper, we consider the channels with a special form of overhead: a processing energy cost whenever a non-zero signal is transmitted. We show that under certain conditions, achieving the capacity of such channels requires intermittent, or `bursty', transmissions. Thus, an optimal sleeping schedule can be specified for wireless nodes to achieve the optimal power efficiency. We show that in the low SNR regime, there is a simple relation between the optimal burstiness and the overhead cost: one should use a fraction of the available degrees of freedom at an SNR level of radic2epsiv, where epsiv is the normalized overhead energy cost. We extend this result to use bursty Gaussian transmissions in multiple parallel channels with different noise levels. Our result can be intuitively interpreted as a 'glue pouring' process, generalizing the wellknown water pouring solution. We then use this approach to compute the achievable rate region of the multiple access channel with overhead cost.

This paper is eligible for the student paper award. We formulate a broadcast problem, where based on their quality of observations, outputs at various receivers are represented on a graph (called "degradation graph"). If receiver Z is a physically degraded version of receiver Y, then node Z is a child of node Y in this graph. This generalization of the classical degraded broadcast channel provides a framework for various situations where at least some information should be available to receivers with partial (or noisier) observations. Upper and lower bounds are obtained on the capacity region. The upper bound is based on auxiliary variables, whose structure is described by the mirror image of the channel's degradation graph. As a special case of our problem, a packet broadcast network is considered.

We study the problem of transmission-side diversity and routing in a static wireless network. It is assumed that each node in the network is equipped with a single omnidirectional antenna and that multiple nodes are allowed to coordinate their transmissions in order to obtain energy savings. We derive analytical results for achievable energy savings for both line and grid network topologies. It is shown that the energy savings of and are achievable in line and grid networks with a large number of nodes, respectively. We then develop a dynamic-programming-based algorithm for finding the optimal route in an arbitrary network, as well as suboptimal algorithms with polynomial complexity. We show through simulations that these algorithms can achieve average energy savings of about in random networks, as compared to the noncooperative schemes.

A wireless network with fading and a single source-destination pair is considered. The information reaches the destination via multiple hops through a sequence of layers of single-antenna relays. At high signal-to-noise ratio (SNR), the simple amplify-and-forward strategy is shown to be optimal in terms of degrees of freedom, because it achieves the degrees of freedom equal to a point-to-point multiple-input multiple-output (MIMO) system. Hence, the lack of coordination in relay nodes does not reduce the achievable degrees of freedom. The performance of this amplify-and-forward strategy degrades with increasing network size. This phenomenon is analyzed by finding the tradeoffs between network size, rate, and diversity. A lower bound on the diversity-multiplexing tradeoff for concatenation of multiple random Gaussian matrices is obtained. Also, it is shown that achievable network size in the outage formulation (short codes) is a lot smaller than the ergodic formulation (long codes).

We consider a multiple-input multiple-output (MIMO) wideband Rayleigh block-fading channel where the channel state is unknown to both the transmitter and the receiver and there is only an average power constraint on the input. We compute the capacity and analyze its dependence on coherence length, number of antennas and receive signal-to-noise ratio (SNR) per degree of freedom. We establish conditions on the coherence length and number of antennas for the noncoherent channel to have a “near-coherent” performance in the wideband regime. We also propose a signaling scheme that is near-capacity achieving in this regime. We compute the error probability for this wideband noncoherent MIMO channel and study its dependence on SNR, number of transmit and receive antennas and coherence length. We show that error probability decays inversely with coherence length and exponentially with the product of the number of transmit and receive antennas. Moreover, channel outage dominates error probability in the wideband regime. We also show that the critical as well as cutoff rates are much smaller than channel capacity in this regime.

Channel capacity in the limit of vanishing signal-to-noise ratio (SNR) per degree of freedom is known to be linear in SNR for fading and nonfading channels, regardless of channel state information at the receiver (CSIR). It has recently been shown that the significant engineering difference between the coherent and the noncoherent fading channels, including the requirement of peaky signaling and the resulting spectral efficiency, is determined by how the capacity limit is approached as SNR tends to zero, or in other words, the sublinear term in the capacity expression. In this paper, we show that this sublinear term is determined by the channel coherence level, which we define to quantify the relation between the SNR and the channel coherence time. This allows us to trace a continuum between the case with perfect CSIR and the case with no CSIR at all. Using this approach, we also evaluate the performance of suboptimal training schemes.

We investigate the fault diagnosis problem for all- optical networks with probabilistic link and node failures in this paper. Our major contribution is the development of diagnosis algorithms that minimize the operating effort to identify failures. We achieve this by employing the fault diagnosis approach based on proactive probing with perfect feedback: knowledge of the network state is progressively refined through a sequence of optical probe signals, each of which is determined upon the results of previous probe signals (i.e. probe syndromes). To detect and localize failures in all-optical networks with probabilistic node and link failures, we introduce a network transformation that maps both link and node failures in an undirected graph into arc failures in a directed graph and apply our previously developed run-length probing scheme [1] to the directed graph. Our analytical and numerical investigation verifies our previously established guideline for efficient fault diagnosis algorithms: each probe should provide approximately 1-bit of state information, and thus the total number of probes required is approximately equal to the entropy of the network state. Hence the complexity of optical network fault management functionality is fundamentally related to the information entropy of the network state.

A wideband fading channel is considered with causal channel state information (CSI) at the transmitter and no receiver CSI. A simple orthogonal code with energy detection rule at the receiver is shown to achieve the capacity of this channel in the limit of large bandwidth. This code transmits energy only when the channel gain is large enough. In this limit, this capacity without any receiver CSI is the same as the capacity with full receiver CSI - a phenomenon also true for dirty paper coding. For Rayleigh fading, this capacity (per unit time) is proportional to the logarithm of the bandwidth. Our coding scheme is motivated from the Gel'fand-Pinsker and dirty paper coding. Nonetheless, our scheme requires only causal transmitter CSI (CSIT) in contrast with Gel'fand-Pinsker and dirty paper coding, which require non-causal CSIT. A general discrete channel with i.i.d. states is considered later. Each input has an associated cost and a zero cost input "0" exists. The channel state is known at the transmitter in a causal manner. Capacity per unit cost is found for this channel and a simple orthogonal code is shown to achieve it. Later, a novel orthogonal coding scheme is proposed for the case of causal CSIT and a condition for equal capacity per unit cost with causal and non-causal CSIT is derived

We consider a multiple-input, multiple-output (MIMO) wideband Rayleigh block fading channel where the channel state is unknown to both the transmitter and the receiver and there is only an average power constraint on the input. We compute the error probability and study its dependence on receive signal-to-noise ratio (SNR), number of transmit and receive antennas and coherence length. We show that error probability decays inversely with coherence length and exponentially with the product of the number of transmit and receive antennas. Moreover, channel outage dominates error probability in the wideband regime. We also show that the critical as well as cut-off rates are much smaller than channel capacity in this regime

The concept of a fiber aided wireless network architecture (FAWNA) is introduced in [Ray et al., Allerton Conference 2005], which allows high-speed mobile connectivity by leveraging the speed of optical networks. In this paper, we consider a single-input, multiple-output (SIMO) FAWNA, which consists of a SIMO wireless channel and an optical fiber channel, connected through wireless-optical interfaces. We propose a scheme where the received wireless signal at each interface is quantized and sent over the fiber. Though our architecture is similar to that of the classical CEO problem, our problem is different from it. We show that the capacity of our scheme approaches the capacity of the architecture, exponentially with fiber capacity. We also show that for a given fiber capacity, there is an optimal operating wireless bandwidth and an optimal number of wireless-optical interfaces. The wireless-optical interfaces of our scheme have low complexity and do not require knowledge of the transmitter code book. They are also extendable to FAWNAs with large number of transmitters and interfaces and, offer adaptability to variable rates, changing channel conditions and node positions

Network management and control contribute to at least half of the operating cost of current optical networks. All-optical networks with end-to-end transparent lightpaths promise significant cost savings using optical switching at network nodes. However, this cost saving cannot be realized unless the cost of network management is also reduced. In this paper we explore a promising technique towards that goal. The fault diagnosis problem for all-optical networks is investigated via an information theoretic approach, with the objective to minimize the operating 'cost' of failure detection and localization in the optical layer. Under a probabilistic link failure model, we first interpret the run-length probing scheme previously developed for Eulerian graphs as a constrained source-coding algorithm, and characterize its performance via the code rate of its corresponding run-length code. We then extend the run-length probing scheme to non-Eulerian graphs via two alternative approaches: the disjoint-trail decomposition approach and the path-augmentation approach, and obtain their performance analytically. The analytical and numerical results indicate that the run-length probing scheme is asymptotically optimum for both Eulerian and non-Eulerian graphs of large size. The property of the run-length probing scheme also suggests that each probe in an efficient probing scheme should provide approximately one bit of network state information and thus the total number of probes (or equivalently, the operating cost of failure identification) is lower-bounded and approximated by the entropy of the network states. We believe that our approach using information theory in an inter-disciplinary effort can provide new insights on network management, and substantial cost-reduction for all-optical networks can be realized

We develop a novel auction-based algorithm to allow users to fairly compete for a wireless fading channel. We use the second-price auction mechanism whereby user bids for the channel, during each time slot, based on the fade state of the channel, and the user that makes the highest bid wins use of the channel by paying the second highest bid. Under the assumption that each user has a limited budget for bidding, we show the existence of a Nash equilibrium strategy, and the Nash equilibrium leads to a unique allocation for certain channel state distribution, such as the exponential distribution and the uniform distribution over [0, 1]. For uniformly distributed channel state, we establish that the aggregate throughput received by the users using the Nash equilibrium strategy is at least 3/4 of what can be obtained using an optimal centralized allocation that does not take fairness into account. We also show that the Nash equilibrium strategy leads to an allocation that is Pareto optimal (i.e., it is impossible to make some users better off without making some other users worse off). Based on the Nash equilibrium strategies of the second-price auction with money constraint, we further propose a centralized opportunistic scheduler that does not suffer the shortcomings associated with the proportional fair scheduler.

A wireless sensory relay network consists of one source node, one destination node and multiple intermediate relay nodes. In this paper, we study the achievable rates and the scaling laws of power-constrained wireless relay networks in the wideband regime, assuming that relay nodes have no a priori knowledge of channel-state information (CSI) for both the backward channels and the forward channels. We examine the achievable rates in the joint asymptotic regime of the number of relay nodes n, the channel coherence interval L, and the bandwidth W (or the SNR per link rho). We first study narrowband relay networks in the low SNR regime. We investigate a relaying scheme, namely amplify-and-forward (AF) with network training, in which the source node and the destination node broadcast training symbols and each relay node carries out channel estimation and then applies AF relaying to relay information. We provide an equivalent source-to-destination channel model, and characterize the corresponding achievable rate. Our findings show that when rhoL, proportional to the transmission energy in each fading block, is bounded below, the achievable rate has the same scaling order as in coherent relaying, thus enabling us to characterize the scaling law of the relay networks in the low SNR regime. We then generalize the study to power-constrained wideband relay networks, where frequency-selective fading is taken into account. Again, the focus is on the achievable rates by using AF with network training for information relaying. In particular, we examine the scaling behavior of the achievable rates corresponding to two power allocation policies across the frequency subbands at relay nodes, namely, a simple equal power allocation policy and the optimal power allocation policy. We identify the conditions under which the scaling law of the wideband relay networks can be achieved by both power allocation policies. Somewhat surprising, our findings indicate that these two power allocation policies result in achievable rates of the same scaling order, and the scaling law can be characterized under the condition that L/W, proportional to the energy per fading block per subband, is bounded below, and that W is sublinear in n

It is known that, when using a multi-tone FSK scheme in a wideband fading channel to achieve the wideband capacity limit, the codeword probability of error decays very slowly with bandwidth. In this paper, we consider a modified multi-tone FSK scheme which employs a feedback link. We show that, a small amount of feedback improves the error performance significantly

We consider a multiple-input, multiple-output (MIMO) wideband Rayleigh block fading channel, where the channel state is unknown to both the transmitter and the receiver. With only an average power constraint, we compute the capacity of this channel and consider its interaction with the coherence length, number of transmit and receive antennas and receive signal-to-noise ratio (SNR) per degree of freedom. We establish how large the coherence length has to be in order for the non-coherent channel to have a "near coherent" performance in the wideband regime. More specifically, we show that if the coherence length of the channel is above a certain SNR (bandwidth) dependent threshold, the non-coherent and coherent capacities are the same in the large bandwidth regime. We also propose a signaling scheme that is near-optimal in the wideband regime

In the wideband limit, certain types of "flash" signaling, such as flash frequency-shift keying (FSK), achieve the capacity of multipath fading channels. It is not clear, however, whether these asymptotic results translate into insights for practical fading channels in the finite-bandwidth, power-limited regime. It is known that, for flash FSK, the size of the input alphabet grows slowly with increasing bandwidth, leading to very high-peak power per tone. Thus, for flash FSK, the codeword probability of error decays very slowly with bandwidth and feasible rates approach the wideband capacity limit extremely slowly. Without contradicting the above results, our results in this paper point to a more optimistic outlook, from the point of view of error exponents and achievable rates, for the applicability of flash techniques in practical scenarios. We consider multitone FSK (MFSK), which has the same asymptotic capacity-achieving property as flash FSK in the wideband limit, but allows a larger input alphabet size with the same bandwidth. First, we show, using an error exponent approach, that multitone FSK allows lower peak power per tone than flash FSK. Next, we present the capacity of single-tone and two-tone FSK schemes with hard-decision detection at finite bandwidths. For typical channel parameters, the capacities are close to the wideband capacity limit.

The effect of channel coherence on the capacity and energy efficiency of noncoherent fading channels at low SNR is studied. A simple characterization is given, and a new approach is developed, which can be used to study a wide variety of problems for communications over a wideband channel. The flat block fading channel is studied, which transmits one scalar symbol per symbol time distorted by a multiplicative fading coefficient and the additive Gaussian noise.

We study the capacity of fading channels with no CSI at both the transmitter and the receiver. We focus on the low SNR regime, and study the impact of channel memory on the capacity. While the current results on these issues are based on various limiting assumptions, we use a new approach of asymptotic analysis to capture the relation among the key system parameters, and depict the continuum between the extreme cases.

In a point-to-point wireless fading channel, multiple transmit and receive antennas can be used to improve the reliability of reception (diversity gain) or increase the rate of communication for a fixed reliability level (multiplexing gain). In a multiple-access situation, multiple receive antennas can also be used to spatially separate signals from different users (multiple-access gain). Recent work has characterized the fundamental tradeoff between diversity and multiplexing gains in the point-to-point scenario. In this paper, we extend the results to a multiple-access fading channel. Our results characterize the fundamental tradeoff between the three types of gain and provide insights on the capabilities of multiple antennas in a network context.

Flash signalling (with vanishing duty cycle) frequency shift keying (FSK) is known to be a capacity-achieving modulation for multipath fading channels in the limit of infinite bandwidth. However, since capacity-achieving schemes using flash FSK build the richness of their codebooks in frequency, the data rates of such schemes increase slowly with bandwidth. We seek to establish schemes that exhibit better performance than flash FSK for large but finite bandwidth. We consider multitone FSK, which we have shown in our previous work that can also achieve the infinite-bandwidth capacity limit for multipath fading channels, but is more general than FSK. Given its increased generality vis-a-vis FSK, and its optimality from a capacity point of view, multitone FSK is a good candidate for transmission over very wide bandwidth. In this paper, we discuss upper and lower bounds of error probabilities for the family of multitone FSK in Rayleigh fading channels. We find that these two bounds coincide in the infinite bandwidth limit and are therefore asymptotically tight. We compare the error probabilities of FSK and multitone FSK in different situations and conclude that FSK is the preferable scheme when average power is the biting constraint, and multitone FSK may be preferable when peak power is a limiting factor. We also explore the relationship among capacity and parameters related to time efficiency and spectrum efficiency.

Multiple antennas can be used for increasing the amount of diversity or the number of degrees of freedom in wireless communication systems. We propose the point of view that both types of gains can be simultaneously obtained for a given multiple-antenna channel, but there is a fundamental tradeoff between how much of each any coding scheme can get. For the richly scattered Rayleigh-fading channel, we give a simple characterization of the optimal tradeoff curve and use it to evaluate the performance of existing multiple antenna schemes.

We study the capacity of multiple-antenna fading channels. We focus on the scenario where the fading coefficients vary quickly; thus an accurate estimation of the coefficients is generally not available to either the transmitter or the receiver. We use a noncoherent block fading model proposed by Marzetta and Hochwald (see ibid. vol.45, p.139-57, 1999). The model does not assume any channel side information at the receiver or at the transmitter, but assumes that the coefficients remain constant for a coherence interval of length T symbol periods. We compute the asymptotic capacity of this channel at high signal-to-noise ratio (SNR) in terms of the coherence time T, the number of transmit antennas M, and the number of receive antennas N. While the capacity gain of the coherent multiple antenna channel is min{M, N} bits per second per Hertz for every 3-dB increase in SNR, the corresponding gain for the noncoherent channel turns out to be M* (1 - M*/T) bits per second per Hertz, where M*=min{M, N, [T/2]}. The capacity expression has a geometric interpretation as sphere packing in the Grassmann manifold.

In this paper, we study the capacity of multiple antenna fading channels. We focus on the scenario where the fading coefficients vary quickly; thus an accurate estimation of the coefficients is generally not available to either the transmitter or the receiver. We use a block fading model proposed by Marzetta and Hochwald (see IEEE Trans. on Info. Theory, vol.45, no.1, p.139-57, 1999). The model does not assume any prior knowledge of the fading coefficients, but only assumes that the coefficients remain constant for a coherent interval T as an approximation of the continuously varying channel. We compute the asymptotic capacity of this channel at high SNR in terms of T, the number of transmit antennas M and the number of receive antennas N. While the capacity gain of the coherent multi-antenna channel is min{M,N} bps/Hz for every 3 dB increase in SNR, the corresponding gain for the non-coherent channel turns out to be M/sup */(1-M/sup *//T) bps/Hz, where M/sup */=min{M,N,[T/2]}. The capacity expression has a geometric interpretation of sphere packing in the Grassmann manifold.