Reinforcement learning (RL), while being the benchmark for policy formulation, often struggles to deliver robust solutions across varying scenarios, leading to marked performance drops under environmental perturbations. Traditional adversarial training, based on a two-player max-min game, is known to bolster the robustness of RL agents, but it faces challenges: first, the complexity of the worst-case optimization problem may induce \emph{over-optimism}, and second, the choice of a specific set of potential adversaries might lead to \emph{over-pessimism} by considering implausible scenarios. In this work, we first observe that these two challenges do not balance out each other. Thus, we propose to apply variational optimization to optimize over the worst-case distribution of the adversary instead of a single worst-case adversary. Moreover, to counteract over-optimism, we train the RL agent to maximize the lower quantile of the cumulative rewards under worst-case adversary distribution. Our novel algorithm demonstrates a significant advancement over existing robust RL methods, corroborating the importance of the identified challenges and the effectiveness of our approach. To alleviate computational overhead associated with the proposed approach, we also propose a simplified version with lower computational burden and only minimal performance degradation. Extensive experiments validate that our approaches consistently yield policies with superior robustness.

We introduce a new approach to processing complex-valued data using DNNs consisting of parallel real-valued subnetworks with coupled outputs. Our proposed class of architectures, referred to as Steinmetz Neural Networks, incorporates multi-view learning to construct more interpretable representations in the latent space. Moreover, we present the Analytic Neural Network, which incorporates a consistency penalty that encourages analytic signal representations in the latent space of the Steinmetz neural network. This penalty enforces a deterministic and orthogonal relationship between the real and imaginary components. Using an information-theoretic construction, we demonstrate that the generalization gap upper bound posited by the analytic neural network is lower than that of the general class of Steinmetz neural networks. Our numerical experiments depict the improved performance and robustness to additive noise, afforded by our proposed networks on benchmark datasets and synthetic examples.

Regularizing continual learning techniques is important for anticipating algorithmic behavior under new realizations of data. We introduce a new approach to continual learning by imposing the properties of a parabolic partial differential equation (PDE) to regularize the expected behavior of the loss over time. This class of parabolic PDEs has a number of favorable properties that allow us to analyze the error incurred through forgetting and the error induced through generalization. Specifically, we do this through imposing boundary conditions where the boundary is given by a memory buffer. By using the memory buffer as a boundary, we can enforce long term dependencies by bounding the expected error by the boundary loss. Finally, we illustrate the empirical performance of the method on a series of continual learning tasks.

Regularizing neural networks is important for anticipating model behavior in regions of the data space that are not well represented. In this work, we propose a regularization technique for enforcing a level of smoothness in the mapping between the input space and the loss. We specify the level of regularity by requiring that the loss of the network satisfies an elliptic operator over the data domain. To do this, we modify the usual empirical risk minimization objective such that we instead minimize a new objective that satisfies an elliptic operator over points within the domain. This allows us to use existing theory on elliptic operators to anticipate the behavior of the error for points outside the training set. We propose a tractable computational method that approximates the behavior of the elliptic operator while being computationally efficient. Finally, we analyze the properties of the proposed regularization to understand the performance on common problems of distribution shift and group imbalance. Numerical experiments empirically confirm the promise of the proposed regularization technique.

Transformer models have achieved remarkable empirical successes, largely due to their in-context learning capabilities. Inspired by this, we explore training an autoregressive transformer for in-context reinforcement learning (ICRL). In this setting, we initially train a transformer on an offline dataset consisting of trajectories collected from various RL tasks, and then fix and use this transformer to create an action policy for new RL tasks. Notably, we consider the setting where the offline dataset contains trajectories sampled from suboptimal behavioral policies. In this case, standard autoregressive training corresponds to imitation learning and results in suboptimal performance. To address this, we propose the Decision Importance Transformer (DIT) framework, which emulates the actor-critic algorithm in an in-context manner. In particular, we first train a transformer-based value function that estimates the advantage functions of the behavior policies that collected the suboptimal trajectories. Then we train a transformer-based policy via a weighted maximum likelihood estimation loss, where the weights are constructed based on the trained value function to steer the suboptimal policies to the optimal ones. We conduct extensive experiments to test the performance of DIT on both bandit and Markov Decision Process problems. Our results show that DIT achieves superior performance, particularly when the offline dataset contains suboptimal historical data.

Estimating the unknown reward functions driving agents' behavior is a central challenge in inverse games and reinforcement learning. This paper introduces a unified framework for reward function recovery in two-player zero-sum matrix games and Markov games with entropy regularization. Given observed player strategies and actions, we aim to reconstruct the underlying reward functions. This task is challenging due to the inherent ambiguity of inverse problems, the non-uniqueness of feasible rewards, and limited observational data coverage. To address these challenges, we establish reward function identifiability using the quantal response equilibrium (QRE) under linear assumptions. Building on this theoretical foundation, we propose an algorithm to learn reward from observed actions, designed to capture all plausible reward parameters by constructing confidence sets. Our algorithm works in both static and dynamic settings and is adaptable to incorporate other methods, such as Maximum Likelihood Estimation (MLE). We provide strong theoretical guarantees for the reliability and sample-efficiency of our algorithm. Empirical results demonstrate the framework’s effectiveness in accurately recovering reward functions across various scenarios, offering new insights into decision-making in competitive environments.

Natural-language interaction between passengers and autonomous vehicles is essential for trust, safety, and user experience, but deploying Large Language Models (LLMs) on automotive edge platforms is constrained by compute, memory, energy, and privacy. We present Pi-talk, an edge-only system that enables real-time passenger–vehicle dialogue using a Small Language Model (SLM) running entirely on embedded hardware. Pi-talk performs multimodal fusion of onboard camera, ultrasonic distance, and navigation context via a lightweight encoder–adapter module that aligns modalities into compact semantic tokens for a pre-trained SLM. The SLM produces context-aware explanations of driving decisions, route options, and situational updates without cloud connectivity. Safety is enforced through a real-time safety envelope that gates responses and actions using distance thresholds and timing constraints. We further adapter-tune the SLM (on-device or offline) and deploy it with INT8 quantization and an Open Neural Network Exchange (ONNX) runtime to achieve efficient batch = 1 inference on Raspberry-Pi–class hardware. We evaluate task quality (evaluation loss), end-to-end latency, CPU utilization, and memory footprint, and include ablations contrasting unimodal vs. fused inputs. Results show that Pi-talk sustains few-second, edge-only inference while meeting stringent resource and latency limits and maintaining the safety envelope required for autonomous operation. To our knowledge, Pi-talk is among the first edge-only, multimodal passenger–vehicle dialogue systems that both fine-tune and run a small language model entirely on Raspberry Pi–class, CPU-only hardware with an explicit while enforcing a runtime safety envelope.

Accelerating model convergence in resource-constrained environments is essential for fast and efficient neural network training. This work presents learn2mix, a new training strategy that adaptively adjusts class proportions within batches, focusing on classes with higher error rates. Unlike classical training methods that use static class proportions, learn2mix continually adapts class proportions during training, leading to faster convergence. Empirical evaluations on benchmark datasets show that neural networks trained with learn2mix converge faster than those trained with existing approaches, achieving improved results for classification, regression, and reconstruction tasks under limited training resources and with imbalanced classes. Our empirical findings are supported by theoretical analysis.

One of the major challenges in estimating conditional potential outcomes and conditional average treatment effects (CATE) is the presence of hidden confounders. Since testing for hidden confounders cannot be accomplished only with observational data, conditional unconfoundedness is commonly assumed in the literature of CATE estimation. Nevertheless, under this assumption, CATE estimation can be significantly biased due to the effects of unobserved confounders. In this work, we consider the case where in addition to a potentially large observational dataset, a small dataset from a randomized controlled trial (RCT) is available. Notably, we make no assumptions on the existence of any covariate information for the RCT dataset, we only require the outcomes to be observed. We propose a CATE estimation method based on a pseudo-confounder generator and a CATE model that aligns the learned potential outcomes from the observational data with those observed from the RCT. Our method is applicable to many practical scenarios of interest, particularly those where privacy is a concern (e.g., medical applications). Extensive numerical experiments are provided demonstrating the effectiveness of our approach for both synthetic and real-world datasets.

One of the most significant challenges in Conditional Average Treatment Effect (CATE) estimation is the statistical discrepancy between distinct treatment groups. To address this issue, we propose a model-agnostic data augmentation method for CATE estimation. First, we derive regret bounds for general data augmentation methods suggesting that a small imputation error may be necessary for accurate CATE estimation. Inspired by this idea, we propose a contrastive learning approach that reliably imputes missing potential outcomes for a selected subset of individuals formed using a similarity measure. We augment the original dataset with these reliable imputations to reduce the discrepancy between different treatment groups while inducing minimal imputation error. The augmented dataset can subsequently be employed to train standard CATE estimation models. We provide both theoretical guarantees and extensive numerical studies demonstrating the effectiveness of our approach in improving the accuracy and robustness of numerous CATE estimation models.

Methods in the field of quickest change detection rapidly detect in real-time a change in the data-generating distribution of an online data stream. Existing methods have been able to detect this change point when the densities of the pre-and post-change distributions are known. Recent work has extended these results to the case where the pre-and post-change distributions are known only by their score functions. This work considers the case where the pre-and post-change score functions are known only to correspond to distributions in two disjoint sets. This work selects a pair of least-favorable distributions from these sets to robustify the existing score-based quickest change detection algorithm, the properties of which are studied. This paper calculates the least-favorable distributions for specific model classes and provides methods of estimating the leastfavorable distributions for common constructions. Simulation results are provided demonstrating the performance of our robust change detection algorithm.

McKean-Vlasov stochastic differential equations (MV-SDEs) provide a mathematical description of the behavior of an infinite number of interacting particles by imposing a dependence on the particle density. We study the influence of explicitly including distributional information in the parameterization of the SDE. We propose a series of semi-parametric methods for representing MV-SDEs, and corresponding estimators for inferring parameters from data based on the properties of the MV-SDE. We analyze the characteristics of the different architectures and estimators, and consider their applicability in relevant machine learning problems. We empirically compare the performance of the different architectures and estimators on real and synthetic datasets for time series and probabilistic modeling. The results suggest that explicitly including distributional dependence in the parameterization of the SDE is effective in modeling temporal data with interaction under an exchangeability assumption while maintaining strong performance for standard Itô-SDEs due to the richer class of probability flows associated with MV-SDEs.

Federated Learning (FL) is a collaborative machine learning framework that allows multiple users to train models utilizing their local data in a distributed manner. However, considerable statistical heterogeneity in local data across devices often leads to suboptimal model performance compared with independently and identically distributed (IID) data scenarios. In this paper, we introduce DynamicFL, a new FL framework that investigates the trade-offs between global model performance and communication costs for two widely adopted FL methods: Federated Stochastic Gradient Descent (FedSGD) and Federated Averaging (FedAvg). Our approach allocates diverse communication resources to clients based on their data statistical heterogeneity, considering communication resource constraints, and attains substantial performance enhancements compared to uniform communication resource allocation. Notably, our method bridges the gap between FedSGD and FedAvg, providing a flexible framework leveraging communication heterogeneity to address statistical heterogeneity in FL. Through extensive experiments, we demonstrate that DynamicFL surpasses current state-of-the-art methods with up to a 10% increase in model accuracy, demonstrating its adaptability and effectiveness in tackling data statistical heterogeneity challenges.

In this work, we introduce REFORMA, a novel robust reinforcement learning (RL) approach to design controllers for unmanned aerial vehicles (UAVs) robust to unknown disturbances during flights. These disturbances, typically due to wind turbulence, electromagnetic interference, temperature extremes and many other external physical interference, are highly dynamic and difficult to model. REFORMA can perform a real-time online adaptation to these disturbances and generate appropriate velocity actions as countermeasures to stabilize the drone. REFORMA consists of two components: a base policy trained completely in simulation using model-free RL and an adaptation module trained via supervised learning with on-policy datasets. By varying the disturbance strength in an adaptation module, i.e., adopting adaptive adversary, the policy is then able to handle extreme cases when the velocity of the drone is immediately affected by disturbances. Finally, we demonstrate the effectiveness of our method through extensive simulated experiments. To the best of our knowledge, REFORMA is the first robust RL approach that uses adaptive adversaries to tackle uncertain disturbances in drone tasks.

Decision Transformers (DTs) have been highly effective for offline reinforcement learning (RL) tasks, successfully modeling the sequences of actions in a given set of demonstrations. However, DTs may perform poorly in stochastic environments, which are prevalent in robotics scenarios. In this paper, we identify that the root cause of this performance degradation is the growing variance of returns-to-go, the signal used by DTs to predict actions, accumulated over the horizon. Building upon this insight, we propose an extension to DTs that allows them to be steered toward high-reward regions, where the expected returns are estimated using temporal difference learning. This way, we not only mitigate the growing variance problem but also eliminate the need for DTs to have access to returns-to-go during evaluation and deployment phases. We show that our method outperforms state-of-the-art offline RL methods in both simulated and real-world robotic arm environments.

The goal of this paper is to develop distributionally robust optimization (DRO) estimators, specifically for multidimensional Extreme Value Theory (EVT) statistics. EVT supports using semi-parametric models called max-stable distributions built from spatial Poisson point processes. While powerful, these models are only asymptotically valid for large samples. However, since extreme data is by definition scarce, the potential for model misspecification error is inherent to these applications, thus DRO estimators are natural. In order to mitigate over-conservative estimates while enhancing out-of-sample performance, we study DRO estimators informed by semi-parametric max-stable constraints in the space of point processes. We study both tractable convex formulations for some problems of interest (e.g. CVaR) and more general neural network based estimators. Both approaches are validated using synthetically generated data, recovering prescribed characteristics, and verifying the efficacy of the proposed techniques. Additionally, the proposed method is applied to a real data set of financial returns for comparison to a previous analysis. We established the proposed model as a novel formulation in the multivariate EVT domain, and innovative with respect to performance when compared to relevant alternate proposals.

Advancing loss function design is pivotal for optimizing neural network training and performance. This work introduces Random Linear Projections (RLP) loss, a novel approach that enhances training efficiency by leveraging geometric relationships within the data. Distinct from traditional loss functions that target minimizing pointwise errors, RLP loss operates by minimizing the distance between sets of hyperplanes connecting fixed-size subsets of feature-prediction pairs and feature-label pairs. Our empirical evaluations, conducted across benchmark datasets and synthetic examples, demonstrate that neural networks trained with RLP loss outperform those trained with traditional loss functions, achieving improved performance with fewer data samples, and exhibiting greater robustness to additive noise. We provide theoretical analysis supporting our empirical findings.

Parabolic partial differential equations (PDEs) appear in many disciplines to model the evolution of various mathematical objects, such as probability flows, value functions in control theory, and derivative prices in finance. It is often necessary to compute the solutions or a function of the solutions to a parametric PDE in multiple scenarios corresponding to different parameters of this PDE. This process often requires resolving the PDEs from scratch, which is time-consuming. To better employ existing simulations for the PDEs, we propose a framework for finding solutions to parabolic PDEs across different scenarios by meta-learning an underlying base distribution.%tasks. We build upon this base distribution to propose a method for computing solutions to parametric PDEs under different parameter settings. Finally, we illustrate the application of the proposed methods through extensive experiments in generative modeling, stochastic control, and finance. The empirical results suggest that the proposed approach improves generalization to solving new PDEs.

Classical quickest change detection algorithms require modeling pre-change and post-change distributions. Such an approach may not be feasible for various machine learning models because of the complexity of computing the explicit distributions. Additionally, these methods may suffer from a lack of robustness to model mismatch and noise. This paper develops a new variant of the classical Cumulative Sum (CUSUM) algorithm for the quickest change detection. This variant is based on Fisher divergence and the Hyvärinen score and is called the Hyvärinen score-based CUSUM (SCUSUM) algorithm. The SCUSUM algorithm allows the applications of change detection for unnormalized statistical models, i.e., models for which the probability density function contains an unknown normalization constant. The asymptotic optimality of the proposed algorithm is investigated by deriving expressions for average detection delay and the mean running time to a false alarm. Numerical results are provided to demonstrate the performance of the proposed algorithm.

Classical change detection algorithms typically require modeling pre-change and post-change distributions. The calculations may not be feasible for various machine learning models because of the complexity of computing the partition functions and normalized distributions. Additionally, these methods may suffer from a lack of robustness to model mismatch and noise. In this paper, we develop a new variant of the classical Cumulative Sum (CUSUM) change detection, namely Score-based CUSUM (SCUSUM), based on Fisher divergence and the Hyvärinen score. Our method allows the applications of the quickest change detection for unnormalized distributions. We provide a theoretical analysis of the detection delay given the constraints on false alarms. We prove the asymptotic optimality of the proposed method in some particular cases. We also provide numerical experiments to demonstrate our method’s computation, performance, and robustness advantages.

We consider a fundamental class of assortment optimization problems in an offline data-driven setting. The firm does not know the underlying customer choice model but has access to an offline dataset consisting of the historically offered assortment set, customer choice, and revenue. The objective is to use the offline dataset to find an optimal assortment. Due to the combinatorial nature of assortment optimization, the problem of insufficient data coverage is likely to occur in the offline dataset. Therefore, designing a provably efficient offline learning algorithm becomes a significant challenge. To this end, based on the principle of pessimism, we propose a novel algorithm called Pessimistic ASsortment opTimizAtion (PASTA for short), which can correctly identify the optimal assortment by only requiring the offline data to cover the optimal assortment under general settings. In particular, we establish the first regret bound for the offline assortment optimization problem under the celebrated multinomial logit model (MNL). We also propose an efficient computational procedure to solve our pessimistic assortment optimization problem. Our numerical studies demonstrate the superiority of the proposed method over the existing baseline method.

Off-policy evaluation (OPE) is important for closing the gap between offline training and evaluation of reinforcement learning (RL), by estimating performance and/or rank of target (evaluation) policies using offline trajectories only. It can improve the safety and efficiency of data collection and policy testing procedures in situations where online deployments are expensive, such as healthcare. However, existing OPE methods fall short in estimating human feedback (HF) signals, as HF may be conditioned over multiple underlying factors and are only sparsely available; as opposed to the agent-defined environmental rewards (used in policy optimization), which are usually determined over parametric functions or distributions. Consequently, the nature of HF signals makes extrapolating accurate OPE estimations to be challenging. To resolve this, we introduce an OPE for HF (OPEHF) framework that revives existing OPE methods in order to accurately evaluate the HF signals. Specifically, we develop an immediate human reward (IHR) reconstruction approach, regularized by environmental knowledge distilled in a latent space that captures the underlying dynamics of state transitions as well as issuing HF signals. Our approach has been tested over *two real-world experiments*, adaptive *in-vivo* neurostimulation and intelligent tutoring, and a simulation environment (visual Q&A). Results show that our approach significantly improves the performance toward estimating HF signals accurately, compared to directly applying (variants of) existing OPE methods.

This work considers the problem of transferring causal knowledge between tasks for Individual Treatment Effect (ITE) estimation. To this end, we theoretically assess the feasibility of transferring ITE knowledge and present a practical framework for efficient transfer. A lower bound is introduced on the ITE error of the target task to demonstrate that ITE knowledge transfer is challenging due to the absence of counterfactual information. Nevertheless, we establish generalization upper bounds on the counterfactual loss and ITE error of the target task, demonstrating the feasibility of ITE knowledge transfer. Subsequently, we introduce a framework with a new Causal Inference Task Affinity (CITA) measure for ITE knowledge transfer. Specifically, we use CITA to find the closest source task to the target task and utilize it for ITE knowledge transfer. Empirical studies are provided, demonstrating the efficacy of the proposed method. We observe that ITE knowledge transfer can significantly (up to 95%) reduce the amount of data required for ITE estimation.

Point processes often have a natural interpretation with respect to a continuous process. We propose a point process construction that describes arrival time observations in terms of the state of a latent diffusion process. In this framework, we relate the return times of a diffusion in a continuous path space to new arrivals of the point process. This leads to a continuous sample path that is used to describe the underlying mechanism generating the arrival distribution. These models arise in many disciplines, such as financial settings where actions in a market are determined by a hidden continuous price or in neuroscience where a latent stimulus generates spike trains. Based on the developments in Itô’s excursion theory, we propose methods for inferring and sampling from the point process derived from the latent diffusion process. We illustrate the approach with numerical examples using both simulated and real data. The proposed methods and framework provide a basis for interpreting point processes through the lens of diffusions.

Detecting an abrupt and persistent change in the underlying distribution of online data streams is an important problem in many applications. This paper proposes a new robust score-based algorithm called RSCUSUM, which can be applied to unnormalized models and addresses the issue of unknown post-change distributions. RSCUSUM replaces the Kullback-Leibler divergence with the Fisher divergence between pre- and post-change distributions for computational efficiency in unnormalized statistical models and introduces a notion of the “least favorable” distribution for robust change detection. The algorithm and its theoretical analysis are demonstrated through simulation studies.

Analyzing large-scale data from simulations of turbulent flows is memory intensive, requiring significant resources. This major challenge highlights the need for data compression techniques. In this study, we apply a physics-informed Deep Learning technique based on vector quantization to generate a discrete, low-dimensional representation of data from simulations of three-dimensional turbulent flows. The deep learning framework is composed of convolutional layers and incorporates physical constraints on the flow, such as preserving incompressibility and global statistical characteristics of the velocity gradients. The accuracy of the model is assessed using statistical, comparison-based similarity and physics-based metrics. The training data set is produced from Direct Numerical Simulation of an incompressible, statistically stationary, isotropic turbulent flow. The performance of this lossy data compression scheme is evaluated not only with unseen data from the stationary, isotropic turbulent flow, but also with data from decaying isotropic turbulence, and a Taylor-Green vortex flow. Defining the compression ratio (CR) as the ratio of original data size to the compressed one, the results show that our model based on vector quantization can offer CR = 85 with a mean square error (MSE) of$O$(10−3), and predictions that faithfully reproduce the statistics of the flow, except at the very smallest scales where there is some loss. Compared to the recent study based on a conventional autoencoder where compression is performed in a continuous space, our model improves the CR by more than 30 percent, and reduces the MSE by an order of magnitude. Our compression model is an attractive solution for situations where fast, high quality and low-overhead encoding and decoding of large data are required.

We propose a new structured pruning framework for compressing Deep Neural Networks (DNNs) with skip-connections, based on measuring the statistical dependency of hidden layers and predicted outputs. The dependence measure defined by the energy statistics of hidden layers serves as a model-free measure of information between the feature maps and the output of the network. The estimated dependence measure is subsequently used to prune a collection of redundant and uninformative layers. Extensive numerical experiments on various architectures show the efficacy of the proposed pruning approach with competitive performance to state-of-the-art methods.

Current objective functions used for training neural MCMC proposal distributions implicitly rely on architectural restrictions to yield sensible optimization results, which hampers the development of highly expressive neural MCMC proposal architectures. In this work, we introduce and demonstrate a semi-empirical procedure for determining approximate objective functions suitable for optimizing arbitrarily parameterized proposal distributions in MCMC methods. Our proposed Ab Initio objective functions consist of the weighted combination of functions following constraints on their global optima and transformation invariances that we argue should be upheld by general measures of MCMC efficiency for use in proposal optimization. Our experimental results demonstrate that Ab Initio objective functions maintain favorable performance and preferable optimization behavior compared to existing objective functions for neural MCMC optimization. We find that Ab Initio objective functions are sufficiently robust to enable the confident optimization of neural proposal distributions parameterized by deep generative networks extending beyond the regimes of traditional MCMC schemes.

Collaborations among multiple organizations, such as financial institutions, medical centers, and retail markets in decentralized settings are crucial to providing improved service and performance. However, the underlying organizations may have little interest in sharing their local data, models, and objective functions. These requirements have created new challenges for multi-organization collaboration. In this work, we propose Gradient Assisted Learning (GAL), a new method for multiple organizations to assist each other in supervised learning tasks without sharing local data, models, and objective functions. In this framework, all participants collaboratively optimize the aggregate of local loss functions, and each participant autonomously builds its own model by iteratively fitting the gradients of the overarching objective function. We also provide asymptotic convergence analysis and practical case studies of GAL. Experimental studies demonstrate that GAL can achieve performance close to centralized learning when all data, models, and objective functions are fully disclosed.

Federated Learning allows the training of machine learning models by using the computation and private data resources of many distributed clients. Most existing results on Federated Learning (FL) assume the clients have ground-truth labels. However, in many practical scenarios, clients may be unable to label task-specific data due to a lack of expertise or resource. We propose SemiFL to address the problem of combining communication-efficient FL such as FedAvg with Semi-Supervised Learning (SSL). In SemiFL, clients have completely unlabeled data and can train multiple local epochs to reduce communication costs, while the server has a small amount of labeled data. We provide a theoretical understanding of the success of data augmentation-based SSL methods to illustrate the bottleneck of a vanilla combination of communication-efficient FL with SSL. To address this issue, we propose alternate training to 'fine-tune global model with labeled data' and 'generate pseudo-labels with the global model.' We conduct extensive experiments and demonstrate that our approach significantly improves the performance of a labeled server with unlabeled clients training with multiple local epochs. Moreover, our method outperforms many existing SSFL baselines and performs competitively with the state-of-the-art FL and SSL results.

Understanding multivariate dependencies in both the bulk and the tails of a distribution is an important problem for many applications, such as ensuring algorithms are robust to observations that are infrequent but have devastating effects. Archimax copulas are a family of distributions endowed with a precise representation that allows simultaneous modeling of the bulk and the tails of a distribution. Rather than separating the two as is typically done in practice, incorporating additional information from the bulk may improve inference of the tails, where observations are limited. Building on the stochastic representation of Archimax copulas, we develop a non-parametric inference method and sampling algorithm. Our proposed methods, to the best of our knowledge, are the first that allow for highly flexible and scalable inference and sampling algorithms, enabling the increased use of Archimax copulas in practical settings. We experimentally compare to state-of-the-art density modeling techniques, and the results suggest that the proposed method effectively extrapolates to the tails while scaling to higher dimensional data. Our findings suggest that the proposed algorithms can be used in a variety of applications where understanding the interplay between the bulk and the tails of a distribution is necessary, such as healthcare and safety.

We propose a neural network architecture that enables non-parametric calibration and generation of multivariate extreme value distributions (MEVs). MEVs arise from Extreme Value Theory (EVT) as the necessary class of models when extrapolating a distributional fit over large spatial and temporal scales based on data observed in intermediate scales. In turn, EVT dictates that $d$-max-decreasing, a stronger form of convexity, is an essential shape constraint in the characterization of MEVs. As far as we know, our proposed architecture provides the first class of non-parametric estimators for MEVs that preserve these essential shape constraints. We show that the architecture approximates the dependence structure encoded by MEVs at parametric rate. Moreover, we present a new method for sampling high-dimensional MEVs using a generative model. We demonstrate our methodology on a wide range of experimental settings, ranging from environmental sciences to financial mathematics and verify that the structural properties of MEVs are retained compared to existing methods.

In order to meet the ever-growing demand for data traffic, highly efficient multiple access schemes, such as OFDMA, are widely used in modern communication standards. In such multiple access schemes, adaptive modulation and coding (AMC) are used to optimize the transmission rate of each user. However, feedback information, such as SNR and Doppler shift, characterizing the communication environment of each user is indispensable of key importance for AMC. In the past, these information and parameters were often estimated using reference signals. However, the reference signal becomes overhead, resulting in throughput degradation and processing delay. Furthermore, the computation burden can be large as it is necessary to perform channel parameter estimation individually for each user. Previously, over the single-user channel, we have proposed a joint SNR and Doppler shift detection method via a spectrogram-based data-driven method, without the reference signal. This paper extends this framework to multiuser OFDM multiple access channels. In the newly proposed method, SNR and Doppler shift for all users can be detected simultaneously via deep learning-guided object detection algorithms from each spectrogram image. Simulation results are provided to validate the effectiveness of the proposed method.

It has been conjectured that the Fisher divergence is more robust to model uncertainty than the conventional Kullback-Leibler (KL) divergence. This motivates the design of a new class of robust generative auto-encoders (AE) referred to as Fisher auto-encoders. Our approach is to design Fisher AEs by minimizing the Fisher divergence between the intractable joint distribution of observed data and latent variables, with that of the postulated/modeled joint distribution. In contrast to KL-based variational AEs (VAEs), the Fisher AE can exactly quantify the distance between the true and the model-based posterior distributions. Qualitative and quantitative results are provided on both MNIST and celebA datasets demonstrating the competitive performance of Fisher AEs in terms of robustness compared to other AEs such as VAEs and Wasserstein AEs.

In this paper, we propose a new structural technique for pruning deep neural networks with skip-connections. Our approach is based on measuring the importance of feature maps in predicting the output of the model using their Fisher scores. These scores subsequently used for removing the less informative layers from the graph of the network. Extensive experiments on the classification of CIFAR-10, CIFAR-100, and SVHN data sets demonstrate the efficacy of our compressing method both in the number of parameters and operations.

The design of handcrafted neural networks requires a lot of time and resources. Recent techniques in Neural Architecture Search (NAS) have proven to be competitive or better than traditional handcrafted design, although they require domain knowledge and have generally used limited search spaces. In this paper, we propose a novel framework for neural architecture search, utilizing a dictionary of models of base tasks and the similarity between the target task and the atoms of the dictionary; hence, generating an adaptive search space based on the base models of the dictionary. By introducing a gradient-based search algorithm, we can evaluate and discover the best architecture in the search space without fully training the networks. The experimental results show the efficacy of our proposed task-aware approach.

We propose a new generative modeling technique for learning multidimensional cumulative distribution functions (CDFs) in the form of copulas. Specifically, we consider certain classes of copulas known as Archimedean and hierarchical Archimedean copulas, popular for their parsimonious representation and ability to model different tail dependencies. We consider their representation as mixture models with Laplace transforms of latent random variables from generative neural networks. This alternative representation allows for computational efficiencies and easy sampling, especially in high dimensions. We describe multiple methods for optimizing the network parameters. Finally, we present empirical results that demonstrate the efficacy of our proposed method in learning multidimensional CDFs and its computational efficiency compared to existing methods.

With the increase in demand for wireless data traffic, high-speed communication systems are required in many environment. Adaptive modulation and coding is an important technique to realize this, but it requires feedback of communication environment information represented by SNR. Conventional SNR estimation methods have a problem of degrading estimation accuracy in fast-moving environments and the presence of carrier frequency offset (CFO). Convolutional neural network (CNN) is capable of estimating the SNR from the trained received signal dataset accurately. This paper investigates an input signal representation to further improve the SNR classification accuracy by CNN. We then propose to combine respective spectrogram data of IQ domains. It can extract the features related to SNR from the received signal to the maximum extent possible and thus achieve highly accurate SNR estimation. Simulation results show that the proposed approach outperforms the other existing candidates.

We consider the distributed learning setting where each agent or learner holds a specific parametric model and a data source. The goal is to integrate information across a set of learners and data sources to enhance the prediction accuracy of a given learner. A natural way to integrate information is to build a joint model across a group of learners that shares common parameters of interest. However, the underlying parameter sharing patterns across a set of learners may not be known a priori. Misspecifying the parameter sharing patterns or the parametric model for each learner often yields a biased estimator that degrades the prediction accuracy. We propose a general method to integrate information across a set of learners that is robust against misspecification of both models and parameter sharing patterns. The main crux of our proposed method is to sequentially incorporate additional learners that can enhance the prediction accuracy of an existing joint model based on user- specified parameter sharing patterns across a set of learners. Theoretically, we show that the proposed method can data-adaptively select a parameter sharing pattern that enhances the predictive performance of a given learner. Extensive numerical studies are conducted to assess the performance of the proposed method.

This paper proposes a novel convolutional neural network (CNN) based joint classification method to characterize the signal-to-noise power ratio (SNR) and Doppler shift using spectrogram images, in order to enable efficient adaptive modulation and coding (AMC) designs. It is necessary to maintain high communication performances even in stringent environments where transceivers move at high speed due to the diversification of wireless applications. To optimize the transmission rate in such dynamic environments, AMC scheme is known to be effective. AMC is generally designed based on feedback information (FBI) such as the SNR and Doppler shift acquired on the receiving side. Here, the challenge is an increase in calculation burden, processing delay and estimation accuracy of the FBI. We focused on the spectrogram which is composed of power values in the time and frequency domains. Its two-dimensional fluctuation represents the Doppler shift as well as noise values. In the proposed method, such key information for AMC can be simultaneously extracted from a single spectrogram via a trained CNN. Therefore it is expected to contribute to reducing the computational burden and speeding up the signal processing. Simulation results are presented to demonstrate that the proposed method achieves better performance than traditional methods.

It is known that the empirical spectral distribution of random matrices obtained from linear codes of increasing length converges to the well-known Marchenko-Pastur law, if the Hamming distance of the dual codes is at least 5. In this paper, we prove that the convergence rate in probability is at least of the order$n^{-1/4}$where$n$is the length of the code.

A central issue of many statistical learning problems is to select an appropriate model from a set of candidate models. Large models tend to inflate the variance (or overfitting), while small models tend to cause biases (or underfitting) for a given fixed dataset. In this work, we address the critical challenge of model selection to strike a balance between model fitting and model complexity, thus gaining reliable predictive power. We consider the task of approaching the theoretical limit of statistical learning, meaning that the selected model has the predictive performance that is as good as the best possible model given a class of potentially misspecified candidate models. We propose a generalized notion of Takeuchi's information criterion and prove that the proposed method can asymptotically achieve the optimal out-sample prediction loss under reasonable assumptions. It is the first proof of the asymptotic property of Takeuchi's information criterion to our best knowledge. Our proof applies to a wide variety of nonlinear models, loss functions, and high dimensionality (in the sense that the models' complexity can grow with sample size). The proposed method can be used as a computationally efficient surrogate for leave-one-out cross-validation. Moreover, for modeling streaming data, we propose an online algorithm that sequentially expands the model complexity to enhance selection stability and reduce computation cost. Experimental studies show that the proposed method has desirable predictive power and significantly less computational cost than some popular methods.

We propose a new architecture for distributed image compression from a group of distributed data sources. The work is motivated by practical needs of data-driven codec design, low power consumption, robustness, and data privacy. The proposed architecture, which we refer to as Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC), is able to train distributed encoders and one joint decoder on correlated data sources. Its compression capability is much better than the method of training codecs separately. Meanwhile, the performance of our distributed system with 10 distributed sources is only within 2 dB peak signal-to-noise ratio (PSNR) of the performance of a single codec trained with all data sources. We experiment distributed sources with different correlations and show how our data-driven methodology well matches the Slepian-Wolf Theorem in Distributed Source Coding (DSC). To the best of our knowledge, this is the first data-driven DSC framework for general distributed code design with deep learning.

Motivated by the ever-increasing demands for limited communication bandwidth and low-power consumption, we propose a new methodology, named joint Variational Autoencoders with Bernoulli mixture models (VAB), for performing clustering in the compressed data domain. The idea is to reduce the data dimension by Variational Autoencoders (VAEs) and group data representations by Bernoulli mixture models (BMMs). Once jointly trained for compression and clustering, the model can be decomposed into two parts: a data vendor that encodes the raw data into compressed data, and a data consumer that classifies the received (compressed) data. In this way, the data vendor benefits from data security and communication bandwidth, while the data consumer benefits from low computational complexity. To enable training using the gradient descent algorithm, we propose to use the Gumbel-Softmax distribution to resolve the infeasibility of the back-propagation algorithm when assessing categorical samples.

Nonparametric regression has proven to be successful in extracting features from limited data in neurological applications. However, due to data scarcity, most brain-computer interfaces still rely on linear classifiers. This work leverages the robustness of the James-Stein theorem in nonparametric regression to harness the potentials of deep learning and foster its successful application in neural engineering with small data sets. We propose a novel method that combines James-Stein regression for feature extraction, and deep neural network for decoding; we refer to the architecture as deep James-Stein neural network (DJSNN). We apply the DJSNN to decode eye movement goals in a memory-guided visual saccades to one of eight target locations. The results demonstrate that the DJSNN outperforms existing methods by a substantial margin, especially at deep cortical sites.

In this work, we introduce a new procedure for applying Restricted Boltzmann Machines (RBMs) to missing data inference tasks, based on linearization of the effective energy function governing the distribution of observations. We compare the performance of our proposed procedure with those obtained using existing reconstruction procedures trained on incomplete data. We place these performance comparisons within the context of the perception-distortion trade-off observed in other data reconstruction tasks, which has, until now, remained unexplored in tasks relying on incomplete training data.

We develop a framework for estimating unknown partial differential equations (PDEs) from noisy data, using a deep learning approach. Given noisy samples of a solution to an unknown PDE, our method interpolates the samples using a neural network, and extracts the PDE by equating derivatives of the neural network approximation. Our method applies to PDEs which are linear combinations of user-defined dictionary functions, and generalizes previous methods that only consider parabolic PDEs. We introduce a regularization scheme that prevents the function approximation from overfitting the data and forces it to be a solution of the underlying PDE. We validate the model on simulated data generated by the known PDEs and added Gaussian noise, and we study our method under different levels of noise. We also compare the error of our method with a Cramer-Rao lower bound for an ordinary differential equation (ODE). Our results indicate that our method outperforms other methods in estimating PDEs, especially in the low signal-to-noise (SNR) regime.

Classical supervised classification tasks search for a nonlinear mapping that maps each encoded feature directly to a probability mass over the labels. Such a learning framework typically lacks the intuition that encoded features from the same class tend to be similar and thus has little interpretability for the learned features. In this paper, we propose a novel supervised learning model named Supervised-Encoding Quantizer (SEQ). The SEQ applies a quantizer to cluster and classify the encoded features. We found that the quantizer provides an interpretable graph where each cluster in the graph represents a class of data samples that have a particular style. We also trained a decoder that can decode convex combinations of the encoded features from similar and different clusters and provide guidance on style transfer between sub-classes.

Marine buoys aid in the battle against Illegal, Unreported and Unregulated (IUU) fishing by detecting fishing vessels in their vicinity. Marine buoys, however, may be disrupted by natural causes and buoy vandalism. In this paper, we formulate marine buoy placement as a clustering problem, and propose dropout k-means and dropout k-median to improve placement robustness to buoy disruption.We simulated the passage of ships in the Gabonese waters near West Africa using historical Automatic Identification System (AIS) data, then compared the ship detection probability of dropout k-means to classic k-means and dropout k-median to classic k-median. With 5 buoys, the buoy arrangement computed by classic k-means, dropout k-means, classic k-median and dropout k-median have ship detection probabilities of 38%, 45%, 48% and 52%.

Speech Emotion Recognition (SER) has emerged as a critical component of the next generation of human-machine interfacing technologies. In this work, we propose a new dual-level model that predicts emotions based on both MFCC features and mel-spectrograms produced from raw audio signals. Each utterance is preprocessed into MFCC features and two mel-spectrograms at different time-frequency resolutions. A standard LSTM processes the MFCC features, while a novel LSTM architecture, denoted as Dual-Sequence LSTM (DS-LSTM), processes the two mel-spectrograms simultaneously. The outputs are later averaged to produce a final classification of the utterance. Our proposed model achieves, on average, a weighted accuracy of 72.7% and an unweighted accuracy of 73.3% - a 6% improvement over current state-of-the-art unimodal models - and is comparable with multimodal models that leverage textual information as well as audio signals.

A technique for compressing deep neural models achieving competitive performance to state-of-the-art methods is proposed. The approach utilizes the mutual information between the feature maps and the output of the model in order to prune the redundant layers of the network. Extensive numerical experiments on both CIFAR-10, CIFAR-100, and Tiny ImageNet data sets demonstrate that the proposed method can be effective in compressing deep models, both in terms of the numbers of parameters and operations. For instance, by applying the proposed approach to DenseNet model with 0.77 million parameters and 293 million operations for classification of CIFAR-10 data set, a reduction of 62.66% and 41.00% in the number of parameters and the number of operations are respectively achieved, while increasing the test error only by less than 1%.

Various types of parameter restart schemes have been proposed for proximal gradient algorithm with momentum to facilitate their convergence in convex optimization. However, under parameter restart, the convergence of proximal gradient algorithm with momentum remains obscure in nonconvex optimization. In this paper, we propose a novel proximal gradient algorithm with momentum and parameter restart for solving nonconvex and nonsmooth problems. Our algorithm is designed to 1) allow for adopting flexible parameter restart schemes that cover many existing ones; 2) have a global sub-linear convergence rate in nonconvex and nonsmooth optimization; and 3) have guaranteed convergence to a critical point and have various types of asymptotic convergence rates depending on the parameterization of local geometry in nonconvex and nonsmooth optimization. Numerical experiments demonstrate the convergence and effectiveness of our proposed algorithm.

Recurrent Neural Network (RNN) and its variations such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), have become standard building blocks for learning online data of sequential nature in many research areas, including natural language processing and speech data analysis. In this paper, we present a new methodology to significantly reduce the number of parameters in RNNs while maintaining performance that is comparable or even better than classical RNNs. The new proposal, referred to as Restricted Recurrent Neural Network (RRNN), restricts the weight matrices corresponding to the input data and hidden states at each time step to share a large proportion of parameters. The new architecture can be regarded as a compression of its classical counterpart, but it does not require pre-training or sophisticated parameter fine-tuning, both of which are major issues in most existing compression techniques. Experiments on natural language modeling show that compared with its classical counterpart, the restricted recurrent architecture generally produces comparable results at about 50% compression rate. In particular, the Restricted LSTM can outperform classical RNN with even less number of parameters.

We propose an adaptive signal sampling approach that dynamically adjusts the sampling rate to approximate the local Nyquist rate of the signal. The proposed adaptive sampling approach consists of a recurrent neural network-based change detector that detects the point of frequency change and a local Nyquist rate estimator based on a multi-rate signal processing scheme. We empirically demonstrate that our adaptive sampling approach significantly reduces the overall sampling rate for various types of signals and therefore improves the computational efficiency of subsequent signal processing.

Motivated by Fisher divergence, in this paper we present a new set of information quantities which we refer to as gradient information. These measures serve as surrogates for classical information measures such as those based on logarithmic loss, Kullback-Leibler divergence, directed Shannon information, etc. in many data-processing scenarios of interest, and often provide significant computational advantage, improved stability and robustness. As an example, we apply these measures to the Chow-Liu tree algorithm, and demonstrate remarkable performance and significant computational reduction using both synthetic and real data.

SARAH and SPIDER are two recently developed stochastic variance-reduced algorithms, and SPIDER has been shown to achieve a near-optimal first-order oracle complexity in smooth nonconvex optimization. However, SPIDER uses an accuracy-dependent stepsize that slows down the convergence in practice, and cannot handle objective functions that involve nonsmooth regularizers. In this paper, we propose SpiderBoost as an improved scheme, which allows to use a much larger constant-level stepsize while maintaining the same near-optimal oracle complexity, and can be extended with proximal mapping to handle composite optimization (which is nonsmooth and nonconvex) with provable convergence guarantee. In particular, we show that proximal SpiderBoost achieves an oracle complexity of O(min{n^{1/2}\epsilon^{-2},\epsilon^{-3}}) in composite nonconvex optimization, improving the state-of-the-art result by a factor of O(min{n^{1/6},\epsilon^{-1/3}}). We further develop a novel momentum scheme to accelerate SpiderBoost for composite optimization, which achieves the near-optimal oracle complexity in theory and substantial improvement in experiments.

We develop a methodology (referred to as kinetic prediction) for predicting time series undergoing unknown changes in their data generating distributions. Based on Kolmogorov-Tikhomirov's ε-entropy, we propose a concept called ε-predictability that quantifies the size of a model class (which can be parametric or nonparametric) and the maximal number of abrupt structural changes that guarantee the achievability of asymptotically optimal prediction. Moreover, for parametric distribution families, we extend the aforementioned kinetic prediction with discretized function spaces to its counterpart with continuous function spaces, and propose a sequential Monte Carlo-based implementation. We also extend our methodology for predicting smoothly varying data generating distributions. Under reasonable assumptions, we prove that the average predictive performance converges almost surely to the oracle bound, which corresponds to the case that the data generating distributions are known in advance. The results also shed some light on the so called “prediction-inference dilemma.” Various examples and numerical results are provided to demonstrate the wide applicability of our methodology.

The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent randomization improves the performance in terms of the required number of random features. In this paper, we are concerned with the randomized-feature approach in supervised learning for good generalizability. We propose the Energy-based Exploration of Random Features (EERF) algorithm based on a data-dependent score function that explores the set of possible features and exploits the promising regions. We prove that the proposed score function with high probability recovers the spectrum of the best fit within the model class. Our empirical results on several benchmark datasets further verify that our method requires smaller number of random features to achieve a certain generalization error compared to the state-of-the-art while introducing negligible pre-processing overhead. EERF can be implemented in a few lines of code and requires no additional tuning parameters.

The problem of sequential detection of anomalies in multimodal data is considered. The objective is to observe physical sensor data from CCTV cameras, and social media data from Twitter and Instagram to detect anomalous behaviors or events. Data from each modality is transformed to discrete time count data by using an artificial neural network to obtain counts of objects in CCTV images and by counting the number of tweets or Instagram posts in a geographical area. The anomaly detection problem is then formulated as a problem of quickest detection of changes in count statistics. The quickest detection problem is then solved using the framework of partially observable Markov decision processes (POMDP), and structural results on the optimal policy are obtained. The resulting optimal policy is then applied to real multimodal data collected from New York City around a 5K race to detect the race. The count data both before and after the change is found to be nonstationary in nature. The proposed mathematical approach to this problem provides a framework for event detection in such nonstationary environments and across multiple data modalities.

A macaque monkey is trained to perform two different kinds of tasks, memory aided and visually aided. In each task, the monkey saccades to eight possible target locations. A classifier is proposed for direction decoding and task decoding based on local field potentials (LFP) collected from the prefrontal cortex. The LFP time-series data is modeled in a nonparametric regression framework, as a function corrupted by Gaussian noise. It is shown that if the function belongs to Besov bodies, then the proposed wavelet shrinkage and thresholding based classifier is robust and consistent. The classifier is then applied to the LFP data to achieve high decoding performance. The proposed classifier is also quite general and can be applied for the classification of other types of time-series data as well, not necessarily brain data.

Machine learning systems learn from and make predictions by building models from observed data. Because large models tend to overfit while small models tend to underfit for a given fixed dataset, a critical challenge is to select an appropriate model (e.g. set of variables/features). Model selection aims to strike a balance between the goodness of fit and model complexity, and thus to gain reliable predictive power. In this paper, we study a penalized model selection technique that asymptotically achieves the optimal expected prediction loss (referred to as the limit of learning) offered by a set of candidate models. We prove that the proposed procedure is both statistically efficient in the sense that it asymptotically approaches the limit of learning, and computationally efficient in the sense that it can be much faster than cross validation methods. Our theory applies for a wide variety of model classes, loss functions, and high dimensions (in the sense that the models' complexity can grow with data size). We released a python package with our proposed method for general usage like logistic regression and neural networks.

In this work, we propose a new inference procedure for understanding non-stationary processes, under the framework of evolutionary spectra developed by Priestley. Among various frameworks of modeling non-stationary processes, the distinguishing feature of the evolutionary spectra is its focus on the physical meaning of frequency. The classical estimate of the evolutionary spectral density is based on a double-window technique consisting of a short-Fourier transform and a smoothing. However, smoothing is known to suffer from the so-called bias leakage problem. By incorporating Thomson's multitaper method that was originally designed for stationary processes, we propose an improved estimate of the evolutionary spectral density, and analyze its bias/variance/resolution tradeoff. As an application of the new estimate, we further propose a non-parametric rank-based stationarity test, and provide various experimental studies.

Nonlinear kernels can be approximated using finite-dimensional feature maps for efficient risk minimization. Due to the inherent trade-off between the dimension of the (mapped) feature space and the approximation accuracy, the key problem is to identify promising (explicit) features leading to a satisfactory out-of-sample performance. In this work, we tackle this problem by efficiently choosing such features from multiple kernels in a greedy fashion. Our method sequentially selects these explicit features from a set of candidate features using a correlation metric. We establish an out-of-sample error bound capturing the trade-off between the error in terms of explicit features (approximation error) and the error due to spectral properties of the best model in the Hilbert space associated to the combined kernel (spectral error). The result verifies that when the (best) underlying data model is sparse enough, i.e., the spectral error is negligible, one can control the test error with a small number of explicit features, that can scale poly-logarithmically with data. Our empirical results show that given a fixed number of explicit features, the method can achieve a lower test error with a smaller time cost, compared to the state-of-the-art in data-dependent random features.

To address order selection for an autoregressive model fitted to time series data, we propose a new information criterion. It has the benefits of the two well-known model selection techniques: the Akaike information criterion and the Bayesian information criterion. When the data are generated from a finite-order autoregression, the Bayesian information criterion is known to be consistent, and so is the new criterion. When the true order is infinity or suitably high with respect to the sample size, the Akaike information criterion is known to be efficient in the sense that its predictive performance is asymptotically equivalent to the best offered by the candidate models; in this case, the new criterion behaves in a similar manner. Different from the two classical criteria, the proposed criterion adaptively achieves either consistency or efficiency depending on the underlying true model. In practice, where the observed time series is given without any prior information about the model specification, the proposed order selection criterion is more flexible and reliable compared with classical approaches. Numerical results are presented, demonstrating the adaptivity of the proposed technique when applied to various data sets.

We consider the problem of generating pseudorandom matrices based on the similarity of their spectra to Wigner's semicircular law. We introduce the notion of an r-independent pseudo-Wigner matrix ensemble and prove the closeness of the spectra of its matrices to the semicircular density in the Kolmogorov distance. We give an explicit construction of a family of N × N pseudo-Wigner ensembles using dual BCH codes and show that the Kolmogorov complexity of the obtained matrices is of the order of log(N) bits for a fixed designed Kolmogorov distance precision. We compare our construction with the quasi-random graphs introduced by Chung et al. and demonstrate that the pseudo-Wigner matrices pass stronger randomness tests than the adjacency matrices of these graphs (lifted by the mapping 0 → 1 and 1 → -1) do. Finally, we provide numerical simulations verifying our theoretical results.

We consider the problem of generating symmetric pseudo-random sign (±1) matrices based on the similarity of their spectra to Wigner's semicircular law. Using binary m-sequences (Golomb sequences) of lengths n = 2m- 1, we give a simple explicit construction of circulant n x n sign matrices and show that their spectra converge to the semicircular law when n grows. The Kolmogorov complexity of the proposed matrices equals to that of Golomb sequences and is at most 2log2(n) bits.

We derive the tightest known bounds on η = 2ν, where ν is the growth rate of the logarithm of the number of independent sets on a hexagonal lattice. To obtain these bounds, we generalize a method proposed by Calkin and Wilf. Their original strategy cannot immediately be used to derive bounds for η, due to the difference in symmetry between square and hexagonal lattices, so we propose a modified method and an algorithm to derive rigorous bounds on η. In particular, we prove that 1.546440708536001 ≤ η ≤ 1.5513, which improves upon the best known bounds of 1.5463 ≤ η ≤ 1.5527 given by Nagy and Zeger. Our lower bound matches the numerical estimate of Baxter up to 9 digits after the decimal point, and our upper bound can be further improved by following our method.

We consider the problem of generating symmetric pseudo-random sign (±1) matrices based on the similarity of their spectra to Wigner's semicircular law. We introduce r-independent pseudo-Wigner ensembles and prove closeness of their spectra to the semicircular density in Kolmogorov distance. We give an explicit construction of a family of N × N pseudo-Wigner ensembles using dual BCH codes and show that the Kolmogorov complexity of the obtained matrices is of the order of log (N) bits for a fixed Kolmogorov distance precision. Finally, we provide numerical simulations verifying our theoretical results.

We consider the problem of generating symmetric pseudo-random sign (±1) matrices based on the similarity of their spectra to Wigner's semicircular law. Using binary m-sequences (Golomb sequences) of lengths n = 2m- 1, we give a simple explicit construction of circulant n × n sign matrices and show that their spectra converge to the semicircular law when n grows. The Kolmogorov complexity of the proposed matrices equals to that of Golomb sequences and is at most 2log2(n) bits.

We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance. A classical cross validation strategy is the leave-one-out cross validation (LOOCV) where one sample is left out for validation and training is done on the rest of the samples that are presented to the learner, and this process is repeated on all of the samples. LOOCV is rarely used in practice due to the high computational complexity. In this paper, we first develop a computationally efficient approximate LOOCV (ALOOCV) and provide theoretical guarantees for its performance. Then we use ALOOCV to provide an optimization algorithm for finding the regularizer in the empirical risk minimization framework. In our numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as well as our proposed framework for the optimization of the regularizer.

In this paper, we consider MIMO beamforming in the presence of Rayleigh product channels. Based on a derived largest eigenvalue distribution, the key performance metrics of the beamforming system are obtained, assuming perfect channel knowledge at the transmitter and receiver. Using the closed-form expressions, we gain insights into the behavior of MIMO beamforming systems in scenarios of practical interest.

The performance of MIMO radar has been a subject of intense study in the past decades. For such a system, however, the important phenomenon of phase transition has received little attention in the literature. In this paper, we study the phase transition on the target detection probability of a SNR maximizing detector. Such a detector declares a target to be present when the largest eigenvalue of the observed data matrix exceeds a threshold. In particular, we identify a critical value below and above which the limiting detection performance is described by the Tracy-Widom law and the Gaussian law, respectively. Under both laws, the scaling limits and asymptotic expansions of misdetection probability at the vanishing regime are derived using tools from random matrix theory.

We consider the problem of exact sparse signal recovery from a combination of linear and magnitude-only (phaseless) measurements. A k-sparse signal x ∈ ℂnis measured as r = Bx and y = |Cx |, where B ∈ ℂm1 ×nand C ∈ ℂm2 ×nare measurement matrices and |·| is the element-wise absolute value. We show that if max(2m1,1) + m2≥ 4k - 1, then a set of generic measurements are sufficient to recover every k-sparse x exactly, establishing the trade-off between the number of linear and magnitude-only measurements.

We consider the problem of energy-efficient base-station (BS) planning for green cellular network design. There exist a number of criteria for greenness in the literature, but we focus only on the energy-normalized throughput. We first model energy consumption for a heterogeneous network, consisting of macro and micro base-stations, and a network topology. Then, we propose an iterative algorithm for the green BS planning problem in the one-dimensional case: 1) We find the positions of base-stations that maximize the energy-normalized throughput for a fixed number of base-stations, and 2) find the optimal number of base-stations which maximizes the energy-normalized throughput. In this work, we evaluate the energy-normalized throughput accounting for adaptive modulation rates with power control, which is a common feature of modern communications systems. The convergence to a local maximum for the proposed algorithm is shown using computer simulations, and the corresponding energy-normalized throughputs are evaluated for a number of system model parameters.

This paper presents the analysis of the impact of circuit nonlinearities on the transmission rate of a communication system. We consider a general transmission model that includes transmitter- and receiver-side nonlinearities, environmental thermal noise and receiver chain noise. Using this model we establish lower and upper bounds for the capacity of nonlinear additive white Gaussian noise channels in various thermal noise and receiver noise regimes.

In this paper, we consider an undersampling system model of the form y = A(T(x, θ)) + n, where x is a k-sparse signal, T(·, · is a (possibly non-linear) function specified by a parameter vector θ and acting on x, A is a sensing matrix, and n is additive measurement noise. We consider an information theoretic decoder that aims to recover the sparse signal and the transformation parameter vector jointly, and study the achievability conditions for estimating the underlying signal within a specified ℓ2distortion for Gaussian sensing matrices. We compare the achievable distortion of the joint estimation process to that of the standard noisy compressed sensing model, where the sparse signal is directly measured with a sensing matrix with the same number of measurements. We also provide a numerical example to illustrate potential applications.

In this paper, we consider network coded (NCed) Hybrid-ARQ (HARQ) for multiple unicast flows. The main contribution of the paper is the derivation of throughput expressions for NCed HARQ with arbitrary number of users in identical i.i.d. channels amid packets for all users. We apply the result to Rayleigh fading channels and two packet combining schemes: incremental redundancy (IR) and chase combining (CC). We verify the analytical results with simulations and observe substantial SNR improvements over NCed ARQ and HARQ. The SNR gains in the moderate/high and low throughput regimes are mainly due to network coding and packet combining, respectively. For low/moderate SNRs, NCed HARQ with IR surpasses the CC performance. In addition, we introduce a novel re-transmission strategy that makes the network coding more efficient at low SNR.

Let A ∈ {-1,1}Na×n and B ∈ {-1,1}Nb×n be two matrices whose rows are drawn i.i.d. from the codewords of the binary codes Caand Cbof length n and dual distances d'aand d'b, respectively, under the mapping 0 → 1 and 1 → -1. It is proven that as n → ∞ with ya:=n/Na∈ (0,∞) and yb:=n/Nb∈ (0, ∞) fixed, the empirical spectral distribution of the matrix A B*/√{NaNb} resembles a universal distribution (closely related to the distribution function of the free multiplicative convolution of two members of the Marchenko-Pastur family of densities) in the sense of the Lévy distance, if the asymptotic dual distances of the underlying binary codes are large enough. Moreover, an explicit upper bound on the Lévy distance of the two distributions in terms of ya, yb, d'a, and d'bis given. Under mild conditions, the upper bound is strengthened to the Kolmogorov distance of the underlying distributions. Numerical studies on the empirical spectral distribution of the product of random matrices from BCH and Gold codes are provided, which verify the validity of this result.

We study the impact of nonlinearities on the transmission rate of a communication system by establishing lower and upper bounds on the channel capacity. Our analysis is based on a general transmission model that includes transmitter and receiver nonlinearities, environmental thermal noise, and receiver chain noise. Using this model, we derive lower and upper bounds for the channel capacity of additive white Gaussian noise channels, fading channels, and multiple input multiple output channels in various thermal noise and receiver noise regimes.

Given the sensitivity of the potential WSN applications and because of resource limitations, key management emerges as a challenging issue for WSNs. One of the main concerns when designing a key management scheme is the network scalability. Indeed, the protocol should support a large number of nodes to enable a large scale deployment of the network. In this paper, we propose a new scalable key management scheme for WSNs which provides a good secure connectivity coverage. For this purpose, we make use of the unital design theory. We show that the basic mapping from unitals to key pre-distribution allows us to achieve high network scalability. Nonetheless, this naive mapping does not guarantee a high key sharing probability. Therefore, we propose an enhanced unital-based key pre-distribution scheme providing high network scalability and good key sharing probability approximately lower bounded by 1-e-1≈ 0.632. We conduct approximate analysis and simulations and compare our solution to those of existing methods for different criteria such as storage overhead, network scalability, network connectivity, average secure path length and network resiliency. Our results show that the proposed approach enhances the network scalability while providing high secure connectivity coverage and overall improved performance. Moreover, for an equal network size, our solution reduces significantly the storage overhead compared to those of existing solutions.

This paper presents a comparative study of Schmidl-Cox (SC) and Morelli-Mengali (MM) algorithms for frequency offset estimation in OFDM, along with a new least squares (LS) and a new modified SC algorithm. All algorithms have comparable accuracy approaching asymptotically the Cramer-Rao bound. The complexity of the LS algorithm is between O(N) and O(N log N) operations, where N is the length of the training sequence, while the complexity of the SC algorithm is between O(N log N) and O(N2) operations, and the complexity of the MM algorithm is O(N2) operations. The modified version of the SC algorithm requires only one training sequence as opposed to two required by the original SC algorithm, and significantly reduced O(N log N) complexity. The sensitivity of the three algorithms to quantization of the arg function (the argument of a complex number) is analyzed and quantified. The analysis and simulation results demonstrate that while all considered algorithms can be used with coarse quantization of the arg function, the LS algorithm is least affected and the SC algorithm is most affected by this quantization error.

In this paper, we propose a new simple relaying strategy based on bit-interleaved convolutionally coded star-quadrature amplitude modulation (QAM) along with coherent/noncoherent detection. Star-QAM is composed of multiple concentric circles of phase-shift keying (PSK). Exploiting this property, a hard limiter is used to enhance power amplifier (PA) efficiency at the relay. Moreover, we show that the proposed approach retains differential detectability, which results in a significant reduction of receiver complexity with robustness against phase ambiguity. By analyzing our proposed cooperation with coherent/noncoherent detection in terms of asymptotic pairwise error probability (PEP), we show that the full diversity order can be achieved on the condition that the minimum free distance of the convolutional codes is larger than the predetermined value specified by the number of available relays. Furthermore, the effectiveness of the proposed scheme in terms of PA efficiency is confirmed by comparing the statistical distributions of the corresponding instantaneous signal powers. All the theoretical results agree with those obtained by computer simulations.

In this paper, we study the problem of distributed spectrum allocation under a vertical spectrum sharing scenario in a cognitive radio network. The secondary users share the spectrum licensed to the primary user by observing the activity statistics of the primary users, and regulate their transmission strategy in order to abide by the spectrum sharing etiquette. When the primary user is inactive in a subset of the available frequency bands, the secondary users share the spectrum using the recently proposed GADIA algorithm [1]. In this paper, we present analytical and numerical results on the performance of the GADIA algorithm in conjunction with the above-mentioned vertical spectrum sharing scenario. These results reveal near-optimal performance guarantees for the overall vertical spectrum sharing scenario.

In this paper, we propose a new cooperative wireless transmission in a scenario where the source salvages the energy during the relay's transmission considering the fact that the source does not need to retrieve the transmitted message. We also evaluate a direct wireless transmission with wireless energy transfer as a reference. We analyze the performance of these transmission techniques in terms of outage probability. Our analytical results reveal the advantage of energy salvage in combination with spatial diversity over the direct transmission even if the energy transfer efficiency is considerably low.

This study presents the analysis of a constellation partition (CP) algorithm for multiple-input multiple-output–maximum-likelihood detection (MIMO–MLD) systems. The authors consider an Nt by Nr MIMO system, where MLD algorithm is employed at the receiver side for MIMO signal detection. The authors show that for the case of orthogonal space-time block codes and MIMO beamforming (MIMO-BF) systems, the proposed CP algorithm achieves the same error-rate performance as the optimum MLD algorithm while cutting back significantly on the complexity of MIMO–MLD scheme. Thus complexity reduction is not achieved at the expense of the optimum MLD performance in these cases, unlike the existing simplified MLD schemes in the literature. For the case of MIMO-multiplexing system, however, it is observed that the complexity reduction is achieved at the expense of the optimum MLD performance, similar to the existing simplified MLD schemes. Thus the proposed algorithm is especially attractive in real-time implementation of space-time block code and MIMO-BF systems.

We study the capacity of discrete memoryless channels with synchronization errors and additive noise. We first show that with very large alphabets, their capacity can be achieved by independent and identically distributed input sources, and establish proven tight lower and upper capacity bounds. We also derive tight numerical capacity bounds for channels where the synchronization between the input and output is partly preserved, for instance using incorruptible synchronization markers. Such channels include channels with duplication errors, channels that only insert or delete zeros, and channels with bitshift errors studied in magnetic recording. Channels with small alphabets and corrupted by synchronization errors have an infinite memory. Revisiting the theoretical work of Dobrushin and adapting techniques used to compute capacity bounds for finite-state source/channel models, we compute improved numerical capacity lower bounds for discrete memoryless channels with small alphabets, synchronization errors, and memoryless noise. An interesting and some- what surprising result is that as long as the input sequences are not completely deleted, the capacity of channels corrupted by discrete timing errors is always nonzero even if all the symbols are corrupted.

In this paper, we propose jointly optimized linear transceiver algorithms called interference aware-coordinated beamforming (IA-CBF) for a two-cell system where each base station is equipped with multiple transmit antennas. To generalize IA-CBF to more than two-cell scenarios, a new beam-switching mechanism combined with IA-CBF is proposed. For a two-cell system, we derive a minimum-mean-square-error-type IA-CBF algorithm based on a lower bound on the achievable sum rate. We propose optimal (under an assumption of zero other-cell interference) and suboptimal transmit/receive beamforming vectors through zero-forcing IA-CBF algorithms. We also investigate the optimality of the proposed IA-CBF algorithms with respect to the number of receive antennas. Numerical results confirm that the proposed system with two transmit/receive antennas achieves the full degrees of freedom (a.k.a. multiplexing gain) of the two-cell multiple-input multiple-output channel while showing a better sum rate performance than competitive solutions such as non-cooperative eigen-beamforming and interference nulling. A three-dimensional ray tracing tool is also used to evaluate the proposed multi-cell IA-CBF algorithm.

In this work, we propose a QPSK OFDM code that has both low peak to average power ratio and good bit error rate performance. We consider a Time-frequency Turbo Block code. This code is composed of the cosets of the first-order binary Reed-Muller (RM) code in the frequency domain, and Bose-Chaudhuri-Hocquenghem (BCH) code in the time domain. Our contributions are two-fold: 1) we introduce two encoding schemes that guarantee a PAPR of 3 dB for QPSK constellation, 2) we propose moderate complexity decoding procedures for these schemes. We evaluate the performance of the proposed schemes for the additive white Gaussian noise channel. Our code performs only slightly worse than the best known Turbo codes but has significantly lower PAPR. This gives a significant total power gain over the best known Turbo codes.

In this paper, we investigate the service coverage for interweaving cognitive radio networks with cooperative relays in shadowed areas within the metropolitan region. We highlight the influence of relays on the primary and secondary systems in severe shadow fading channels. The outage probability is utilized as a comprehensive performance metric to characterize the coverage quality for the cognitive radio system with relays. We show that cooperative spectrum sensing among the secondary transmitter and relays can improve the vacant spectrum detection probability and the false alarm probability, thereby making the additionally introduced interference by relays to be constrained within a tolerable limit. Overall, our analysis unveils that both the spectrum utilization rate and the spatial coverage can be significantly improved through the use of the relays, and there exists the spatial selectivity phenomena for both the primary and secondary systems.

We investigate spectrum sensing by energy detection based on two different objective functions: a Bayesian sensing cost or the network weighted sum capacity. The Bayesian cost is a traditional detection measure which aims at minimizing a combination of the miss-detection and false-alarm probabilities, while the capacity objective is a communication measure which aims at maximizing the network throughput. Fading-dependent optimal sensing thresholds for each objective are derived in closed-form for different cases of location side information. To make sensing more robust to channel fading, we also propose fading-independent sub-optimal thresholds. Results show that location side information helps improve performance when using the threshold designed for that performance measure. However, the Bayesian-based threshold does not utilize the side information well in improving the network sum capacity. On the other hand, the capacity-based threshold captures the benefit of side information in both the capacity and Bayesian cost measures. Furthermore, it helps to significantly improve the network throughput. The proposed sensing schemes with location side information can also be generalized to a network with multiple cognitive users in a simple and distributed manner.

In this paper, we present an advanced solution for the long standing problem of large peak to average power ratio (PAPR) in orthogonal frequency division multiplexing (OFDM) systems. Although the design of low PAPR codewords has been extensively studied and the existence of asymptotically good codes with low PAPR is also proven, still no code has been constructed to satisfy all requirements. The main goal of the paper is to develop a coding scheme that not only generates low PAPR codewords, but it also performs relatively close to the Shannon limit. We achieve this goal by implementing a time-frequency turbo block coded OFDM. In this scheme, we design the frequency domain component to have a tightly bounded PAPR. The time domain component code is designed to obtain good performance while the decoding algorithm has reasonable complexity. Through comparative performance evaluation we show that utilizing the proposed method, we achieve considerable improvement in terms of PAPR while we slightly loose the performance compared to capacity achieving codes with similar overall block length.

In this paper, we study the air-to-ground cellular systems with no frequency reuse. We provide an analysis of the inter-cell interference under idealized assumptions: 3D hexagonal cell planning, a line of sight channel model with no shadowing and idealized antenna patterns. Based on this model, we derive approximate bounds for the inter to intra-cell interference ratio for the air-to-ground link. In addition, we provide upper bounds on the interference and the outage probability for the ground-to-air link. Simulation results are provided to validate the accuracy of the analytical results.

LetCbe a binary linear block code of lengthn, dimensionkand minimum Hamming distancedover GF(2)n. Letd⊥denote the minimum Hamming distance of the dual code ofCover GF(2)n. Let ε:GF(2)n→{-1,1}nbe the component-wise mapping ε(vi):=(-1)vi, forv=(v1,v2,...,vn) ∈ GF(2)n. Finally, forpn, let \mmbΦCbe a p × n random matrix whose rows are obtained by mapping a uniformly drawn set of sizepof the codewords ofCunder ε. It is shown that ford⊥large enough andy:=p/n∈ (0,1) fixed, asn→∞ the empirical spectral distribution of the Gram matrix of [1/(√n)]\mmbΦCresembles that of a random i.i.d. Rademacher matrix (i.e., the Marchenko-Pastur distribution). Moreover, an explicit asymptotic uniform bound on the distance of the empirical spectral distribution of the Gram matrix of [1/(√n)]\mmbΦCto the Marchenko-Pastur distribution as a function ofyandd⊥is presented.

We consider beam selection using a fixed beamforming network (FBN) at a base station withMarray antennas. In our setting, a Butler matrix is deployed at the RF stage to formMbeams, and then the best beam is selected for transmission. We introduce some properties of the noncentral chi-square distribution and prove the resulting properties of the beam selection gain verifying that beam selection is superior to antenna selection in Rician channels with anyK-factors. Furthermore, we find asymptotically tight stochastic bounds of the beam selection gain, which yield approximate closed form expressions of the expected selection gain and the ergodic capacity. Beam selection has the order of growth of the ergodic capacity Θ(logM) regardless of user location in contrast to Θ(log(logM)) for antenna selection.

Two distinct, but overlapping, networks that operate at the same time, space, and frequency is considered. The first network consists ofnrandomly distributed primary users, which form an ad hoc network. The second network again consists ofmrandomly distributed ad hoc secondary users or cognitive users. The primary users have priority access to the spectrum and do not need to change their communication protocol in the presence of the secondary users. The secondary users, however, need to adjust their protocol based on knowledge about the locations of the primary users to bring little loss to the primary network's throughput. By introducing preservation regions around primary receivers, a modified multihop routing protocol is proposed for the cognitive users. Assumingm=nβwith β >; 1, it is shown that the secondary network achieves almost the same throughput scaling law as a stand-alone network while the primary network throughput is subject to only a vanishingly small fractional loss. Specifically, the primary network achieves the sum throughput of ordern1/2and, for any δ >; 0, the secondary network achieves the sum throughput of orderm1/2-δwith an arbitrarily small fraction of outage. Thus, almost all secondary source-destination pairs can communicate at a rate of orderm-1/2-δ.

In a bi-directional relay channel, two nodes wish to exchange independent messages over a shared wireless half-duplex channel with the help of a relay. In this paper, we derive achievable rate regions for four new half-duplex protocols and compare these to four existing half-duplex protocols and outer bounds. In time, our protocols consist of either two or three phases. In the two phase protocols, both users simultaneously transmit during the first phase and the relay alone transmits during the second phase, while in the three phase protocol the two users sequentially transmit followed by a transmission from the relay. The relay may forward information in one of four manners; we outline existing amplify and forward (AF), decode and forward (DF), lattice based, and compress and forward (CF) relaying schemes and introduce the novel mixed forward scheme. The latter is a combination of CF in one direction and DF in the other. We derive achievable rate regions for the CF and Mixed relaying schemes for the two and three phase protocols. We provide a comprehensive treatment of eight possible half-duplex bi-directional relaying protocols in Gaussian noise, obtaining their relative performance under different SNR and relay geometries.

This paper analyzes the impact and benefits of infrastructure support in improving the throughput scaling in networks ofnrandomly located wireless nodes. The infrastructure uses multiantenna base stations (BSs), in which the number of BSs and the number of antennas at each BS can scale at arbitrary rates relative ton. Under the model, capacity scaling laws are analyzed for both dense and extended networks. Two BS-based routing schemes are first introduced in this study: an infrastructure-supported single-hop (ISH) routing protocol with multiple-access uplink and broadcast downlink and an infrastructure-supported multihop (IMH) routing protocol. Then, their achievable throughput scalings are analyzed. These schemes are compared against two conventional schemes without BSs: the multihop (MH) transmission and hierarchical cooperation (HC) schemes. It is shown that a linear throughput scaling is achieved in dense networks, as in the case without help of BSs. In contrast, the proposed BS-based routing schemes can, under realistic network conditions, improve the throughput scaling significantly in extended networks. The gain comes from the following advantages of these BS-based protocols. First, more nodes can transmit simultaneously in the proposed scheme than in the MH scheme if the number of BSs and the number of antennas are large enough. Second, by improving the long-distance signal-to-noise ratio (SNR), the received signal power can be larger than that of the HC, enabling a better throughput scaling under extended networks. Furthermore, by deriving the corresponding information-theoretic cut-set upper bounds, it is shown under extended networks that a combination of four schemes IMH, ISH, MH, and HC is order-optimal in all operating regimes.

Coronary magnetic resonance imaging (MRI) is a noninvasive imaging modality for diagnosis of coronary artery disease. One of the limitations of coronary MRI is its long acquisition time due to the need of imaging with high spatial resolution and constraints on respiratory and cardiac motions. Compressed sensing (CS) has been recently utilized to accelerate image acquisition in MRI. In this paper, we develop an improved CS reconstruction method, Bayesian least squares-Gaussian scale mixture (BLS-GSM), that uses dependencies of wavelet domain coefficients to reduce the observed blurring and reconstruction artifacts in coronary MRI using traditional l(1) regularization. Images of left and right coronary MRI was acquired in 7 healthy subjects with fully-sampled k-space data. The data was retrospectively undersampled using acceleration rates of 2, 4, 6, and 8 and reconstructed using l(1) thresholding, l(1) minimization and BLS-GSM thresholding. Reconstructed right and left coronary images were compared with fully-sampled reconstructions in vessel sharpness and subjective image quality (1-4 for poor-excellent). Mean square error (MSE) was also calculated for each reconstruction. There were no significant differences between the fully sampled image score versus rate 2, 4, or 6 for BLS-GSM for both right and left coronaries (=N.S.). However, for l(1) thresholding significant differences were observed for rates higher than 2 and 4 for right and left coronaries respectively. l(1) minimization also yields images with lower scores compared to the reference for rates higher than 4 for both coronaries. These results were consistent with the quantitative vessel sharpness readings. BLS-GSM allows acceleration of coronary MRI with acceleration rates beyond what can be achieved with l(1) regularization.

Dual Bose-Ray-Chaudhuri (BCH) codes, despite their favorable peak to average power ratio (PAPR) properties, have not been used in coded orthogonal frequency division multiplexing (OFDM) systems. This is due to unavailability of a practical decoder and large performance gap to the Shannon limit. In this paper, we propose an advanced approach to solve the aforementioned problems. We construct a new code with favorable PAPR properties based on dual BCH codes and develop the associated maximum a posteriori (MAP) decoding algorithm. By exploiting this code as the frequency domain constituent code in a time-frequency turbo structure, we reduce the gap between the performance and Shannon limit while the bounded PAPR of OFDM symbols is guaranteed. By comparative performance evaluation we illustrate that the performance of this system is comparable with that of capacity approaching codes while it has 7 dB lower PAPR.

In this paper, an exponential model is proposed to approximate the error performance of Turbo codes within low and intermediate SNR region. Our analysis unveils that, the proposed exponential model offers a simple but effective method to approximate the waterfall region performance of Turbo codes, which is generally intricate for analytical bounding techniques. By combining the proposed exponential model with the well known union bound estimate, the paper derives a combined performance model to extend the performance estimate within moderate to high SNR region. Moreover, it is shown that, only the free distance term of the weight spectrum is not always enough for a reasonable estimate of the error floor region performance, especially when the free distance multiplicity is small. Numerical results are presented to validate the applicability of the proposed performance model.

We consider the compressive sensing of a sparse or compressible signal x ∈ ℝM. We explicitly construct a class of measurement matrices, referred to as the low density frames, and develop decoding algorithms that produce an accurate estimate x even in the presence of additive noise. Low density frames are sparse matrices and have small storage requirements. Our decoding algorithms for these frames can be implemented in O(Mdvdc) complexity, where dcand dvare the row and column weight of the frame respectively. Simulation results are provided, demonstrating that our approach significantly outperforms state-of-the-art recovery algorithms for numerous cases of interest.

In this paper, we consider network coded (NCed) Hybrid-ARQ (HARQ) for multiple unicast flows. The main contribution of the paper is the derivation of a throughput expression for NCed HARQ with arbitrary number of users in i.i.d. channels. We apply the result to Rayleigh fading channels for incremental redundancy (IR) and chase combining (CC) based NCed HARQ. We verify the analytical approach with simulations and observe substantial SNR (or energy) improvements over regular ARQ with network coding as well as classical (H)ARQ. The SNR gains in the high and low throughput regimes are mainly due to the network coding and HARQ aspects, respectively. For low SNRs, NCed HARQ with IR surpass the CC performance.

Capacity scaling laws are analyzed in an underwater acoustic network with n regularly located nodes. A narrow-band model is assumed where the carrier frequency is allowed to scale as a function of n. In the network, we characterize an attenuation parameter that depends on the frequency scaling as well as the transmission distance. A cut-set upper bound on the throughput scaling is then derived in extended networks. Our result indicates that the upper bound is inversely proportional to the attenuation parameter, thus resulting in a highly power-limited network. Furthermore, we describe an achievable scheme based on the simple nearest-neighbor multi-hop (MH) transmission. It is shown under extended networks that the MH scheme is order-optimal as the attenuation parameter scales exponentially with √n (or faster). Finally, these scaling results are extended to a random network realization.

In this paper, we study the number of measurements required to recover a sparse signal in CMwith L nonzero coefficients from compressed samples in the presence of noise. We consider a number of different recovery criteria, including the exact recovery of the support of the signal, which was previously considered in the literature, as well as new criteria for the recovery of a large fraction of the support of the signal, and the recovery of a large fraction of the energy of the signal. For these recovery criteria, we prove that O(L) (an asymptotically linear multiple of L) measurements are necessary and sufficient for signal recovery, whenever L grows linearly as a function of M. This improves on the existing literature that is mostly focused on variants of a specific recovery algorithm based on convex programming, for which O(L log(M - L)) measurements are required. In contrast, the implementation of our proof method would have a higher complexity. We also show that O(L log(M - L)) measurements are required in the sublinear regime (L - o(M)). For our sufficiency proofs, we introduce a Shannon-theoretic decoder based on joint typicality, which allows error events to be defined in terms of a single random variable in contrast to previous information-theoretic work, where comparison of random variables are required. We also prove concentration results for our error bounds implying that a randomly selected Gaussian matrix will suffice with high probability. For our necessity proofs, we rely on results from channel coding and rate-distortion theory.

In this paper, the problem of distributed dynamic frequency allocation is considered for a canonical communication network, which spans several networks such as cognitive radio networks and digital subscriber lines (DSLs). A greedy asynchronous distributed interference avoidance (GADIA) algorithm for horizontal spectrum sharing has been proposed that achieves performance close to that of a centralized optimal algorithm. The convergence of the GADIA algorithm to a near-optimal frequency allocation strategy is proved and several asymptotic performance bounds have been established for various spatial configurations of the network nodes. Furthermore, the near-equilibrium dynamics of the GADIA algorithm has been studied using the Glauber dynamics, by identifying the problem with the antiferromagnetic inhomogeneous long-range Potts model. Using the near-equilibrium dynamics and methods from stochastic analysis, the robustness of the algorithm with respect to time variations in the activity of network nodes is studied. These analytic results along with simulation studies reveal that the performance is close to that of an optimum centralized frequency allocation algorithm. Further simulation studies confirm that our proposed algorithm outperforms the iterative water-filling algorithm in the low signal-to-interference-plus-noise ratio (SINR) regime, in terms of achieved sum rate, complexity, convergence rate, and robustness to time-varying node activities.

In a protocol assisted switched diversity (PASD) system, the time separated repetition blocks are accumulated in order to increase decoding reliability. In this paper, we consider transmit antenna selection (TAS) protocols in a PASD system. To minimize system complexity we assume the feedback information is limited to the SNR of the current pair of antennas. We propose three TAS protocols : the random TAS (R-TAS) protocol which selects antennas randomly, the threshold TAS (T-TAS) protocol which keeps the current selection if the predetermined SNR threshold level requirement is met and the minimum number of transmission TAS (MNT-TAS) protocol which determines the selection of a new antenna set or keeping the current set based on the evaluation of expectation of the number of necessary remaining transmissions. We analyze the average number of transmissions of three proposed protocols in a Rayleigh fading channel and compare numerical evaluations to simulation results under different system conditions.

We propose a precoding strategy which controls amplitude and phase of receiving signals to improve throughput for two-stage bidirectional relaying. We consider the case when the nodes know channel state information (CSI) and can adopt adaptive modulation techniques. We introduce a novel scheme termed adaptive modulation and network coding (AMNC), which jointly optimizes modulations and network coding based on the CSI. For dynamic bit loading and power allocation, we propose a practical time-sharing method called the segmented precoding, in which a packet is split into several sub-packets, and a set of modulation and network coding is optimized in conjunction with amplitude and phase controls for each sub-packet. It is demonstrated that our proposed scheme can offer a significant improvement of achievable throughput for two-way relaying.

In this paper, the problem of resource allocation in IS-856 uplink in the presence of time-delay is studied. A set of nonlinear adaptive controllers are designed to stabilize the wireless network and use the system resources efficiently. The controllers obtained are then modified properly to retain network stability and performance in the presence of time-delay. Simulation results are presented to show the effectiveness of the proposed approach.

In this paper, data-rate assignment in IS-856 uplink (reverse link) is studied. The problem is first formulated in an interference model framework, and then a dynamic control strategy is developed for efficient rate assignment. In the first step, the controller is designed for the special case when the number of users in the network is fixed. Then, the minimum time required for a dynamic network (where the number of users is subject to change) to achieve a desired performance is obtained. The simulation results are presented to elucidate the effectiveness of the proposed approach.

In this paper, we present a forwarding strategy for two-stage bidirectional relaying in which trellis-coded modulation (TCM) is employed. We reveal that adaptive network coding cannot resolve distance shortening occurred at specific channel conditions when a certain TCM is used. To overcome this issue, we introduce an improved amplify-and-forward (AF) scheme termed pseudo AF (PAF). The proposed strategy adaptively switches network coding and PAF according to the channel information. Computer simulations demonstrate that the proposed approach can improve throughput performance.

We investigate on multi-dimensional QAM constellations optimized by sphere packing with the known densest lattices. We propose a greedy design method assisted by the sphere detection. It is demonstrated that the optimized constellations can significantly increase the squared minimum distance in comparison to the conventional QAM constellations. In addition, we analyze the optimized QAMs through the use of an extrinsic information transfer (EXIT) chart for iterative decoding.

In a bi-directional relay channel, two nodes wish to exchange independent messages over a shared wireless half-duplex channel with the help of relays. Recent work has considered information theoretic limits of the bi-directional relay channel with a single relay. In this work we consider bi-directional relaying with multiple relays. We derive achievable rate regions and outer bounds for half-duplex protocols with multiple decode and forward relays and compare these to the same protocols with amplify and forward relays in an additive white Gaussian noise channel. We consider three novel classes of half-duplex protocols: the (m, 2) 2 phase protocol with m relays, the (m, 3) 3 phase protocol with m relays, and general (m, t) Multiple Hops and Multiple Relays (MHMR) protocols, where m is the total number of relays and 3 < t ¿ m + 2 is the number of temporal phases in the protocol. Finally, we provide a comprehensive treatment of the MHMR protocols with decode and forward relaying and amplify and forward relaying in Gaussian noise, obtaining their respective achievable rate regions, outer bounds and relative performance at different SNRs. The (m,m+2) DF MHMR protocol achieves the largest rate region under simulated channel conditions.

We consider on the move satellite systems with geostationary satellites. Using the antenna pointing distribution of Weerackody and Gonzalez, we derive tight upper and lower bounds of the interference between adjacent satellites from/to the on-the-move platforms. We then compute the distribution of the interference and the outage probability of the system. Simulation results are provided demonstrating that the analytical estimates are accurate.

We study two distinct, but overlapping, networks which operate at the same time, space and frequency. The first network consists of n randomly distributed primary users, which form either an ad hoc network, or an infrastructure supported ad hoc network in which l additional base stations support the primary users. The second network consists of m randomly distributed secondary or cognitive users. The primary users have priority access to the spectrum and do not change their communication protocol in the presence of secondary users. The secondary users, however, need to adjust their protocol based on knowledge about the locations of the primary users so as not to harm the primary network's scaling law. Base on percolation theory, we show that surprisingly, when the secondary network is denser than the primary network, both networks can simultaneously achieve the same throughput scaling as a standalone ad hoc network.

A novel downlink transmission rate-control and feedback reduction strategy for closed-loop multiple-input multiple-output (MIMO) multiple-input multiple-output wireless systems is presented. Unlike conventional systems that use signal to interference plus noise ratio at the receiver as an indicator of channel quality, we propose using instantaneous MIMO capacity as an indicator for the downlink transmission rate-control. A set of instantaneous capacity thresholds is first chosen such that the expected weighted capacity loss because of thresholding effects are minimised. While computing the thresholds, we also consider the quality of service and weight function to meet different traffics and user needs. Then a set of codebooks can be constructed minimising the overall capacity loss with given quality of service constraint. Simulation results show that, with only four data rate-control bits, our algorithm gives only 12% capacity loss in 4×4 MIMO systems and almost twice better than the current IS-856 standard in single-input single-output systems. In case of 5-bit feedback scenario, the proposed algorithm outperforms conventional systems by minimising instantaneous capacity loss.

We investigate modulation schemes optimized for two-way wireless relaying systems, for which network coding is employed at the physical layer. We consider network coding based on denoise-and-forward (DNF) protocol, which consists of two stages: multiple access (MA) stage, where two terminals transmit simultaneously towards a relay, and broadcast (BC) stage, where the relay transmits towards the both terminals. We introduce a design principle of modulation and network coding, considering the superposed constellations during the MA stage. For the case of QPSK modulations at the MA stage, we show that QPSK constellations with an exclusive-or (XOR) network coding do not always offer the best transmission for the BC stage, and that there are several channel conditions in which unconventional 5-ary constellations lead to a better throughput performance. Through the use of sphere packing, we optimize the constellation for such an irregular network coding. We further discuss the design issue of the modulation in the case when the relay exploits diversity receptions such as multiple-antenna diversity and path diversity in frequency-selective fading. In addition, we apply our design strategy to a relaying system using higher-level modulations of 16QAM in the MA stage. Performance evaluations confirm that the proposed scheme can significantly improve end-to-end throughput for two-way relaying systems.

It has recently been shown that significant reduction in the peak to mean envelope power (PMEPR) can be obtained by altering the sign of each subcarrier in a multicarrier system with n subcarriers. However, finding the best sign not only requires a search over 2npossible signs but also may lead to a substantial rate loss for small size constellations. In this paper, we first propose a greedy algorithm to choose the signs based on p-norm minimization and prove that the resulting PMEPR is guaranteed to be less than c log n where c is a constant independent of n for any n. This approach has lower complexity in each iteration compared to the derandomization approach of while achieving similar PMEPR reduction. We further improve the performance of the proposed algorithm by enlarging the search space using pruning. Simulation results show that PMEPR of a multicarrier signal with 128 subcarriers can be reduced to within 1.6 dB of the PMEPR of a single carrier system. In the second part of the paper, we address the rate loss by proposing a block coding scheme in which only one sign vector is chosen for K different modulating vectors. The sign vector can be computed using the greedy algorithm in n iterations. We show that the multi-symbol encoding approach can reduce the rate loss by a factor of K while achieving the PMEPR of c logKn, i.e., only logarithmic growth in K. Simulation results show that the rate loss can be made smaller than %10 at the cost of only 1 db increase in the resulting PMEPR for a system with 128 subcarriers.

We consider beamforming in a cognitive network with multiple primary users and a secondary user sharing the same spectrum. Each primary and secondary user consists of a transmitter and a receiver. In particular, we assume that the secondary transmitter hasNtantennas and transmits data to its single-antenna receiver using beamforming. The beamformer is designed to maximize the cognitive signal-to-interference ratio (CSIR). Using mathematical tools from random matrix theory, we derive both lower and upper bounds on the average interference created by the cognitive transmitter at the primary receivers and the average CSIR of the cognitive user. We further analyze and prove the convergence of these two performance measures asymptotically as the number of antennasNtor primary usersNpincreases. Specifically, we show that the average interference per primary receiver converges toE[d-α], the expected value of the path loss in the network, whereas the average CSIR decays as1/cwhenc = Np/Nt→∞. In the special case ofNt≥ Npthe lower bound of the average total interference approaches 0 and the upper bound of the average CSIR approachesNtE[d->-α]/σC2whereσC2is the noise variance at the cognitive receiver.

For a multiple-antenna system, we find a simple and accurate expression for the asymptotic distribution of the antenna selection gain when the transmitter selects the transmit antenna with the strongest channel. We use this to estimate the underlying channel capacity distributions and obtain the approximate ergodic capacity. This estimate is compared with upper and lower bounds. This analysis demonstrates that unlike multiple-input/multiple-output (MIMO) systems, the channel for antenna selection systems hardens at a slower rate, and thus a significant multiuser scheduling gain can exist-Theta (1/logm) for channel selection as opposed to Theta (1/radicm) for MIMO, where m is the number of transmit antennas.

We propose a low-complexity equalizer whose performance approaches that of the optimal maximum-likelihood estimators in wideband multiple-input multiple-output (MIMO) channels. The proposed algorithm makes use of a bit-flipping refinement procedure preceded by a frequency-domain equalizer and is based on local-optima searching algorithms. Through performance evaluations, it is demonstrated that the proposed equalizer can perform well when a large number of diversity branches are available in severely dispersive fading channels.

We study a cognitive network consisting of a single primary transmitter and multiple secondary, or cognitive, users. The primary transmitter, located at the center of the network, communicates with primary receivers within a disc called the primary exclusive region (PER). Inside the PER, no cognitive users may transmit, in order to guarantee an outage probability for the primary receivers within. Outside the PER, uniformly distributed cognitive users may transmit, provided they are at a certain protected radius from a primary receiver. We analyze the aggregated interference from the cognitive transmitters to a primary receiver within the PER. Based on this interference and the outage guarantee, we derive bounds on the radius of the PER, showing its interdependence on the receiver protected distance and other system parameters. We also extend the analysis to allowing the cognitive users to scale their power according to the distance from the primary transmitter. These studies provide a closed-form, theoretical analysis of such a network geometry with PER, which may be relevant in the upcoming spectrum sharing actions.

We consider a cognitive network consisting of n cognitive users uniformly distributed with constant density among primary users. Each user has a single transmitter and a single receiver, and the primary and cognitive users transmit concurrently. The cognitive users use single-hop transmission in two scenarios: (i) with constant transmit power, and (ii) with transmit power scaled according to the distance to a designated primary transmitter. We show that, in both cases, the cognitive users can achieve a throughput scaled linearly with the number of users n. The first scenario requires the cognitive users to have the transmitter-receiver (Tx-Rx) distance bounded, but it can be arbitrarily large. Then with high probability, any network realization has the throughput scaling linearly with n. The second scenario allows the cognitive Tx-Rx distance to grow with the network at a feasible exponent as a function of the path loss and the power scaling factors. In this case, the average network throughput grows at least linearly with n and at most as n log(n). These results suggest that single-hop transmission may be a suitable choice for cognitive transmission.

In this paper, we consider a cognitive network in which a single primary transmitter communicates with primary receivers within an area of radius RO, called the primary exclusive region (PER). Inside this region, no cognitive users may transmit. Outside the PER, provided that the cognitive transmitters are at a minimal distance isinpfrom a primary receiver, they may transmit concurrently with the primary user. We determine bounds on the primary exclusive radius ROand the guard band isinpto guarantee an outage performance for the primary user. Specifically, for a desired rate COand an outage probability beta, the probability that the primary user's rate falls below COis less than beta. This performance guarantee holds even with an arbitrarily large number of cognitive users uniformly distributed with constant density outside the primary exclusive region.

We investigate on the design of modulation schemes suited for two-way wireless relaying systems that apply network coding at the physical layer. We consider network coding based on denoise-and-forward (DNF), which consists of two stages: multiple access (MA) stage and broadcast (BC) stage. For the case of QPSK constellation in the MA stage, we introduce the modulation-related problems in DNF. We propose two approaches to solve those issues. One uses only QPSK constellations at the BC stage. The other allows the use of unconventional 5-ary modulations, optimized according to the channel condition. The performance evaluation shows that a significant improvement in end-to-end throughput can be achieved, in particular for Nakagami-Rice fading channels.

We consider beamforming in a cognitive network with multiple primary users and secondary users sharing the same spectrum. In particular, we assume that each secondary transmitter has Nt antennas and transmits data to its single- antenna receiver using beamforming. The beamformer is designed to maximize the cognitive user's signal-to-interference ratio (SIR), defined as the ratio of the received signal power at the desired cognitive receiver to the total interference created at all the primary receivers. Using mathematical tools from random matrix theory, we derive both lower and upper bounds on the average interference at the primary receivers and the average SIR of the cognitive user. We further analyze and prove the convergence of these two performance measures asymptotically as the number of antennas Ntor primary users Ntincreases. Specifically, the average interference per primary receiver converges to the expected value of the path loss in the network whereas the average SIR of the secondary user decays as 1/c when c = Np/Ntrarr infin. In the special case of Nt= Np, the average total interference approaches 0 and the average SIR approaches infin.

Interference in cellular terrestrial systems has been extensively studied in the literature mainly through the use of simulations. The interference for CDMA cellular air-to-ground systems has been similarly evaluated through extensive simulations. However, the air-to-ground systems are simpler to analyze than terrestrial systems. In this note, we provide analytic bounds for inter-cell interference in air-to-ground cellular systems with no frequency reuse. We consider the standard 3D hexagonal cell plan and a line of sight channel model with no shadowing and no fading. Simulation results demonstrate that our approximations are extremely tight.

We analyze the benefits of infrastructure support in improving the throughput scaling in networks of n randomly located wireless nodes. The infrastructure uses multi-antenna base stations (BSs), in which the number of BSs and the number of antennas at each BS can scale at arbtrary rates relative to n. We introduce two multi-antenna BS-based routing protocols and analyze their throughput scaling laws. Two conventional schemes not using BSs are also shown for comparison. In dense networks, we show that the BS-based routing schemes do not improve the throughput scaling. In contrast, in extended networks, we show what our BS-based routing schemes can, under certain network conditions, improve the throughput scaling significantly.

Orthogonal Frequency Division Multiplexing (OFDM) is an attractive technique for high data rate transmissions in wireless and wireline systems. However OFDM signals suffer from high peak-to-mean envelope power (PMEPR). In this paper we consider the problem of reducing this high PMEPR using reserved peak reduction subcarriers as considered by Tellado and Cioffi [9]. We analyze the fundamental tradeoff between PMEPR reduction and the rate loss due to reserved subcarriers. We first provide a lower bound on the complementary cumulative distribution of PMEPR using infinitely many reserved subcarriers. We then show that the problem of minimizing the maximum of the absolute value of the signal using reserved subcarriers can be stated as a convex linear matrix inequality problem. While the problem is proved to be convex, its complexity is cubic in the number of OFDM subcarriers n which can be prohibitive when n is large. We then propose a suboptimal Greedy algorithm based on p-norm minimization that chooses only bipolar values for each subcarrier that has less complexity and more PMEPR reduction than the previously proposed algorithm in [8]. Our results provide the best tradeoff possible between PMEPR reduction and the associated rate loss and also leads to a practical algorithm to approach those limits.

We consider a wireless transmission scenario, when a base station is endowed with a fixed beamforming network, where M antennas are employed at the base station to point beams to predetermined azimuthal angles. In our setting, a Butler matrix is deployed at the RF stage to form M beams, and then the best beam is selected for transmission. We derive the distribution of the beam selection gain for this scenario under a Rician channel assumption as a function of both the azimuthal location of the remote unit and the Rician A'-factor. Using some key properties of the noncentral chi-square distribution, we prove that beam selection outperforms antenna selection.

In this paper, we analyze the performance of a communication system that employs Protocol Assisted Switched Diversity (PASD) antennas under Rayleigh fading channel conditions. The PASD system accumulates time-displaced blocks, each with the same information but using different antennas, and then combines their symbols using maximal ratio combining (MRC). In the PASD system, quality of service (QoS) is ensured by comparing the available signal-to-noise ratio (SNR) at the output of the combiner against a preset threshold value. Our analysis shows that this method reduces the number of transmissions significantly as the channel deteriorates as compared with ARQ without memory while it can satisfy the symbol error rate (SER) requirement for most of blocks. It is also shown that deploying more than one antenna is important to stabilize the system when the channel becomes static.

We study a cognitive network consisting of multiple cognitive users communicating in the presence of a single primary user. The primary user is located at the center of the network, and the cognitive users are uniformly distributed within a circle around the primary user. Assuming a constant cognitive user density, the radius of this circle will increase with the number of users. We consider a scheme in which the primary transmitter sends a beacon signaling its own transmission. The cognitive users, upon receiving this beacon, stay silent. Because of channel fading, however, there is a non-zero probability that a cognitive user misses the beacon and hence, with a certain activity factor, transmits concurrently with the primary user. Given the location of the primary receiver, we are interested in the total interference caused by the cognitive users to this receiver. In particular, we provide closed-form bounds on the mean and variance of the interference, and relate them to the outage probability on the primary user. These analytical results can help in the design of a cognitive network with beacon.

In this paper, we construct a family of block orthogonal Golay sequences that have low peak-to-mean envelope power ratio (PMEPR) as well as block wise orthogonal properties. We then present an application of the sequences to channel estimation of multiple-input multiple-output orthogonal frequency division multiplexing (MIMO-OFDM) systems. We compare the performance of the proposed algorithm with that of a frequency division multiplexing (FDM) piloting algorithm, and investigate the effect of co-channel interference (CCI) on the channel estimation performance.

In coded bidirectional cooperation, two nodes wish to exchange messages over a shared half-duplex channel with the help of a relay. In this correspondence, we derive performance bounds for this problem for each of three decode-and-forward protocols. The first protocol is a two phase protocol where both users simultaneously transmit during the first phase and the relay alone transmits during the second. In this protocol, our bounds are tight. The second protocol considers sequential transmissions from the two users followed by a transmission from the relay while the third protocol is a hybrid of the first two protocols and has four phases. In the latter two protocols the bounds are not identical. Numerical evaluation shows that in some cases of interest our bounds do not differ significantly. Finally, in the Gaussian case with path loss, we derive achievable rates and compare the relative merits of each protocol. This case is of interest in cellular systems. Surprisingly, we find that in some cases, the achievable rate region of the four phase protocol contains points that are outside the outer bounds of the other two protocols.

Antenna selection strategies for multiple-input multiple-output (MIMO) wireless systems is presented. In our scheme, both transmit and receive antenna selection is done at the receiver, significantly reducing feedback information to the transmitter. Row/column probability density function (pdf) is used for antenna selection to reduce computational complexity without performance degradation. Unlike other schemes, we used two different antenna selection algorithms both for high SNR and low SNR regime, achieving additional performance gain in comparison with the single antenna selection algorithm. Simulation results show that our scheme nearly approaches to the optimal closed-loop capacity (known as water-filling capacity).

We consider approximations of signals by the elements of a frame in a complex vector space of dimension N and formulate both the noiseless and the noisy sparse representation problems. The noiseless representation problem is to find sparse representations of a signal r given that such representations exist. In this case, we explicitly construct a frame, referred to as the Vandermonde frame, for which the noiseless sparse representation problem can be solved uniquely using O(N2) operations, as long as the number of non-zero coefficients in the sparse representation of r is isinN for some 0 les isin les 0.5, thus improving on a result of Candes and Tao [3]. We also show that isin les 0.5 cannot be relaxed without violating uniqueness. The noisy sparse representation problem is to find sparse representations of a signal r satisfying a distortion criterion. In this case, we establish a lower bound on the trade-off between the sparsity of the representation, the underlying distortion and the redundancy of any given frame.

For a multiple antenna system, we compute the asymptotic distribution of antenna selection gain when the transmitter selects the transmit antenna with the strongest channel. We use this to asymptotically estimate the underlying channel capacity distributions, and demonstrate that unlike multiple- input/multiple-output (MIMO) systems,the channel for antenna selection systems hardens at a slower rate, and thus a significant multiuser scheduling gain can exist. Additionally, even without this scheduling gain, it is demonstrated that transmit antenna selection systems outperform open loop MIMO systems at low signal-to-interference-plus-noise ratio (SINR) regimes, particularly for small number of receive antennas. This may have some implications on wireless system design, because most of the users in modern wireless systems have low SINRs.

We consider sparse representations of signals with at most L nonzero coefficients using a frame F of size M in CN. For any F, we establish a universal numerical lower bound on the average distortion of the representation as a function of the sparsity epsiv = L/N of the representation and redundancy (tau - 1) = M/N - 1 of F. In low dimensions (e.g., N = 6, 8.10), this bound is much stronger than the analytical and asymptotic bounds given in another of our papers. In contrast, it is much less straightforward to compute. We then compare the performance of randomly generated frames to this numerical lower bound and to the analytical and asymptotic bounds given in the aforementioned paper. In low dimensions, it is shown that randomly generated frames perform about 2 dB away from the theoretical lower bound, when the optimal sparse representation algorithm is used. In higher dimensions, we evaluate the performance of randomly generated frames using the greedy orthogonal matching pursuit (OMP) algorithm. The results indicate that for small values of epsiv, OMP performs close to the lower bound and suggest that the loss of the suboptimal search using orthogonal matching pursuit algorithm grows as a function of epsiv. In all cases, the performance of randomly generated frames hardens about their average as N grows, even when using the OMP algorithm.

We compare three approaches for modeling the ultra-wideband (UWB) indoor channel delay profile, including the one adopted by the IEEE 802.15.3a Task Group. We do this using a large database we collected (and have reported on previously) spanning numerous indoor environments. Where appropriate, we recalculate model parameters so as to be compatible with the database. We test the models against the database and against each other by computing certain statistical `attributes' of the ensemble of channel delay profiles, e.g., the probability distribution, across the ensemble, of the root-mean-square (rms) delay spread. We show that each of the modeling approaches yields reasonable agreement with the database for most or all of the attributes tested, and we discuss the relative merits of the three approaches

We propose a set of distributed rate control algorithms for the reverse-link of CDMA systems using a pricing mechanism. In the derivation, we assume that the base station broadcasts a real-valued feedback in order to control the reverse activity level. In addition, we assume that the access terminals are able to transmit data at any rate within a finite range, and that the transmission power is a linear function of the transmission rate. For a specific utility function, we show that the proposed algorithm achieves social optimality, satisfies an interference power constraint, and exhibits desirable short- and long-term fairness behaviors. We then modify this algorithm so that it can be applied to the case when the transmission power is a convex function of the transmission rate. Next, we consider the case that the feedback parameter is a single reverse activity bit, and the underlying rates are elements of a discrete set of admissible rates. This scenario is similar to that of the IS-856 standard. We further modify our algorithms so that they can be applied to this system setup, and verify the performance using numerical simulations. It is demonstrated that our algorithms exhibit short- and long-term fairness behavior for this realistic scenario.

In this paper, we construct a differential spatial multiplexing method that can trade-off spatial diversity for increased transmission rate in differential multi-input multi-output (MIMO) systems. To achieve a desired multiplexing gain while providing simple encoding and decoding, Gram-Schmidt algorithm is used at the transmitter to construct the unitary transmission matrix. In addition, by applying optimum scaling factors to generate transmission matrix, the symbol error rates can be minimized. Because the transmission matrix itself is not the information we are interested in, the equivalent channel parameters for each transmitted information symbols are calculated and the linear signal model for the information symbols is derived. Since that model is exactly the same as that of the coherent spatial multiplexing, all kinds of coherent detection schemes can be directly applied. The advantage of the proposed scheme over the differential space-time block codes (STBC) and the effects of scaling factor are investigated by computer simulation. In high transmission rate regimes, the proposed differential spatial multiplexing method can outperform the differential STBC having the same transmit rate in the signal to noise ratios (SNRs) we are interested in.

We propose a new distributed rate control algorithm for the reverse link of CDMA systems. Specifically, when the statistics of packet arrivals are known locally to each access terminal (AT), this new algorithm aims to provide a means to control the relative delay that each AT experiences. For this algorithm, assuming that the arrival process is Poisson and stationary, it is shown through simulations that all ATs experience an equal amount of average delay when the system is underloaded, and the same packet drop probability when the system is over-loaded. Furthermore, we propose a modification to the algorithm that gives priority to some desired ATs, and reduces their relative average delay. Finally, simulation results indicate that the algorithm achieves similar short-term average delay even when the packet arrival process is not stationary.

Smart antenna technology can greatly improve the coverage radius of wireless systems by focusing the transmitted signal (beam) toward desired users. However, beamforming has an important side effect as it creates a region, herein referred to as the complementary region, where some users in the network cannot sense the directional signals (beams). We have referred to this as the hidden beam problem in Tarokh et al.. From a physical layer perspective, beamforming reduces cochannel interference (CCI). However, in systems that use medium access-control techniques such as the carrier-sense multiple access (CSMA) protocol in wireless local area networks, during a busy period of the channel, the users in the complementary region may wrongly decide that the medium is idle and transmit packets. This may cause CCI, unnecessary retransmissions, subsequent backoff, and increased network latency. We introduced the concept of complementary beamforming (CBF) to address this issue in Tarokh et al.. In this paper, we propose two new methods, called "subspace complementary beamforming " (SCBF) and "complementary superposition beamforming" (CSBF). The SCBF uses dummy data to ensure a controlled level of received energy in all directions of eigenvectors unused by downlink beamforming. The technique works in multipath fading channels. The approach achieves similar results to those reported in Tarokh et al.. However, it enables the second technique (CSBF) which can also send data content in the complementary beam. This allows passive nodes in the network to receive "broadcast" information, while the active nodes are engaged in the exchange of user-specific data. It is shown that CBF effects can be achieved simply by increasing the transmit power of only one of antenna elements when space-division multiple access is not applied. Simulations are provided in order to quantify overall network performance improvements when using CBF, confirming significant throughput and delay performance enhancements.

We construct nonuniform phase-shift keying (PSK) constellations that provide unequal error protection for multiclass traffic such as compressed voice and video data. Then closed-form expressions expression for the exact bit-error rate (BER) of the nonuniform PSK constellations is derived in multiple receive-antenna systems over Rayleigh fading channels. Based on this BER analysis, we optimize the nonuniform PSK constellations such that the BERs of all the bits of each class are equalized or such that the total transmission power is minimized subject to the average BER constraints. In particular, we demonstrate that even for transmitting single-class traffic, the optimized nonuniform PSK constellations can be better than the conventional uniform PSK. Finally, we extend the nonuniform PSK constellations to space-time coded communications systems with multiple transmit and multiple receive antennas.

Cognitive radio promises a low-cost, highly flexible alternative to the classic single-frequency band, single-protocol wireless device. By sensing and adapting to its environment, such a device is able to fill voids in the wireless spectrum and can dramatically increase spectral efficiency. In this paper, the cognitive radio channel is defined as a two-sender, two-receiver interference channel in which sender 2 obtains the encoded message sender 1 plans to transmit. We consider two cases: in the genie-aided cognitive radio channel, sender 2 is noncausally presented the data to be transmitted by sender 1 while in the causal cognitive radio channel, the data is obtained causally. The cognitive radio at sender 2 may then choose to transmit simultaneously over the same channel, as opposed to waiting for an idle channel as is traditional for a cognitive radio. Our main result is the development of an achievable region which combines Gel'fand-Pinkser coding with an achievable region construction for the interference channel. In the additive Gaussian noise case, this resembles dirty-paper coding, a technique used in the computation of the capacity of the Gaussian multiple-input multiple-output (MIMO) broadcast channel. Numerical evaluation of the region in the Gaussian noise case is performed, and compared to an inner bound, the interference channel, and an outer bound, a modified Gaussian MIMO broadcast channel. Results are also extended to the case in which the message is causally obtained.

Costa has proved that for noncausally known Gaussian interference at a power constrained transmitter communicating over an additive white Gaussian noise channel there is no capacity loss when compared to a scenario where interference is not present. For the case of a transmitter communicating over a quasistatic (i.e., nonergodic) fading channel, his method does not apply. In this correspondence, we derive upper and lower bounds on the capacity of compound channels with side information at the transmitter, first for finite alphabet channels and then, based on this result, for channels on standard alphabets (this includes real alphabets). For the special case of a degenerate compound channel with only one possible realization, our bounds are equivalent to the well-known capacity with side-information formula of Gel'fand and Pinsker. For the quasistatic fading channel, when fading is Ricean, we suggest a scheme based on our lower bound for which the performance is found to be relatively good even for moderate K-factor. As K/spl rarr//spl infin/, the uncertainty on the channel vanishes and our scheme obtains the performance of dirty paper coding, namely that the interference is perfectly mitigated. As K/spl rarr/0, the proposed scheme treats the interferer as additional noise. These results may be of importance for the emerging field of cognitive radios where one user may be aware of another user's intended message to a common receiver, but is unaware of the channel path gains.

The performance of two-phase collaborative communication protocols is studied for wireless networks. All the communication nodes in the cluster are assumed to share the same channel and transmit or receive collaboratively in a quasi-static Rayleigh flat-fading environment. In addition to small-scale fading, the effect of large-scale path loss is also considered. Based on a decode-and-forward approach, we consider various variable-rate two-phase protocols that can achieve full diversity order and analyze the effect of node geometry on their performance in terms of the outage probability of mutual information. For the single-relay node case, it is shown that if the collaborator node is close to the source node, a protocol based on space-time coding (STC) can achieve good diversity gain. Otherwise, a protocol based on receiver diversity performs better. These protocols are also compared with one based on fixed-rate repetition coding and their performance tradeoffs with node geometry are studied. The second part deals with multiple relays. It is known that with N relays an asymptotic diversity order of N+1 is achievable with STC-based protocols in the two-phase framework. However, in the framework of collaborative STC, those relay nodes which fail to decode remain silent (this event is referred to as a node erasure). We show that this node erasure has the potential to considerably reduce the diversity order and point out the importance of designing the STC to be robust against such node erasure

The performance of collaborative beamforming is studied using the theory of random arrays within the framework of wireless sensor networks. With the application to ad hoc networks in mind, two scenarios, one denoted closed-loop and the other open-loop, are considered. Associated with these scenarios, the effects of phase jitter and location estimation errors on the average beam pattern are analyzed.

In order to improve the throughput of the opportunistic beamforming, we generalize the opportunistic beamforming by using multiple random weighting vectors at each time slot. The base station chooses the best weighting vector and performs the opportunistic beamforming with this optimum vector. For the case of equally strong independent fast Rayleigh fading channels, we analytically approximate the throughput of the proposed scheme and obtain the optimum number of random weighting vectors per time slot. Numerical results demonstrate that the proposed scheme considerably improves the throughput compared to the conventional opportunistic beamforming for a low/realistic number of users.

The use of the spatial dimension is known to greatly increase the reliability of quasi-static (non-ergodic) wireless channels. In this paper we demonstrate that most of this gain can also he achieved through collaborative communications with single-antenna/multiple-antenna nodes when there is one receiving agent. In particular, for the single antenna case, we consider communication to take place between clusters of nearby nodes. We show the existence of collaborative codes for communications for which the intra-cluster negotiation penalty is in principle small and almost all the diversity gain of traditional space-time codes may be realized. For example, for single transmitter/receiver nodes with two collaborators that have as little as 10 dB path loss advantage over the receiver, the penalty for collaboration over traditional space-time systems is negligible.

We consider an online, processor sharing system with a single server in a packet-switched network where the processor multiplexes packets from multiple input ports onto the same output link. Under a fluid model, in which incoming data is treated as infinitely divisible, the multiplexer allocates its output capacity to serve fractions of packets from all input ports simultaneously (which we call a fluid policy). However, due to the packet nature of the network traffic, a multiplexer scheduler must approximate this fluid policy by a schedule in which only whole packets are sent (which we call a packetized policy). We assume that at each time instant, the aggregate service rate demanded by a fluid policy is no greater than the output link capacity. Then, for a single server with N input ports, and a scheduler using speedup s of at least 2 and a lookahead window of L/spl ges/0 time steps, we prove bounds on worst-case maximum additional delay D. In particular, we show that ([Ne/sup -s/]-L-2)/sup +//spl les/D min {([Ne/sup 1-s/]-L)/sup +/, [Ne/sup -s/]}. In the case when no lookahead is used (L=0), a tighter bound can be obtained: ([Ne/sup -s/]-2)/sup +//spl les/D/spl les/[Ne/sup -s/}.

A cognitive radio can sense the transmission of other users in its environment and possibly extract the corresponding messages. It can use this information to transmit over the same channel while reducing interference from, and to other users. In this paper, we define inter/intra-cluster competitive, cooperative, and cognitive behavior in wireless networks. We define intercluster cognitive behavior as simultaneous transmissions by two or more clusters in which some clusters know the messages to be transmitted by other clusters, and so can act as relays or use a Gel'fand-Pinsker coding-like technique to mitigate interference. We construct an achievable region for the inter-cluster behavior of two multiple access channels. In the Gaussian case, we compare our achievable region to that of competitive behavior as well as that of cooperative behavior

The use of the spatial dimension is known to greatly increase the reliability of quasi-static (i.e., nonergodic) wireless channels. In this paper, it is demonstrated that most of this gain can also be achieved through collaborative communications with single-antenna/multiple-antenna nodes when there is one receiving agent. In particular, for the single-antenna case, communication is considered to take place between clusters of nearby nodes. The existence of collaborative codes for which the intra-cluster negotiation penalty is, in principle, small (and almost all the diversity gain of traditional space-time codes may be realized) is shown. For example, for a single transmitter node with two collaborators and one receiver node, if the collaborators have as little as a 10-dB path loss advantage over the receiver, the penalty for collaboration over traditional space-time systems is negligible.

In order to improve the throughput of the opportunistic beamforming, the authors generalize the opportunistic beamforming by using multiple random weighting vectors at each time slot. The base station chooses the best weighting vector and performs the opportunistic beamforming with this optimum vector. For the case of equally strong independent fast Rayleigh fading channels, the throughput of the proposed scheme is analytically approximated and the optimum number of random weighting vectors per time slot is obtained. Numerical results demonstrate that the proposed scheme considerably improves the throughput compared to the conventional opportunistic beamforming for a low/realistic number of users.

We present an OFDM frequency synchronization scheme. The scheme uses periodic OFDM symbols, similar to the algorithms proposed previously by Morelli and Mengali (1999) and Schmidl and Cox (1997). The proposed scheme attains considerably higher accuracy than the scheme by Schmidl and Cox requiring a similar computational load. Compared to the scheme by Morelli and Mengali, the proposed algorithm attains a somewhat inferior accuracy but at a significantly reduced computational complexity, i.e O(N) versus O(N/sup 2/) operations for N-tone OFDM. In addition to that, the scheme proposed here is considerably less sensitive to the accuracy of the involved computations than the other two schemes.

We develop differential space time block codes (STBC) using non-constant modulus constellations for four transmit antennas. The proposed method improves on the conventional differential STBC techniques because of the larger minimum distances of non-constant modulus constellations. The transmitted signals are modulated using pulse amplitude modulation (PAM). Encoding is similar to that of the conventional differential STBC, while the receiver is different. The signal at the receiver is divided by the estimated channel power and then decoded using a conventional QAM decoder which treats pairs of transmitted PAM symbols transmitted from two transmit antennas as one QAM symbol. For transmission rates greater than 2 bits/channel use, the proposed method outperforms the conventional differential STBC.

We consider a type of on-line, traffic scheduling problem in input queued, crossbar switches. The input to a problem, at each time step, is a set of desired traffic rates. These traffic rates in general cannot be exactly achieved since they assume arbitrarily small fractions of packets can be transmitted at each time step. The goal of the traffic scheduling problem is to closely approximate the given sequence of traffic rates by a sequence of switch uses in which only whole packets are sent. The focus of this paper is bounding the costs incurred in using such an approximation, in terms of the additional buffer size required. We establish universal bounds on the additional buffer size due to sending only whole packets; these bounds do not depend on the particular distribution of the input traffic, require no speedup, and guarantee 100% throughput. Specifically, for an N /spl times/ N input queued, crossbar switch, an on-line, packetizing algorithm is presented that guarantees 100% throughput with a buffer requirement of (N + 1)/sup 2//4 packets per input port with no speedup. The algorithm can he improved to run in O(N log N) time, using a fast algorithm for edge-coloring bipartite multigraphs. In the reverse direction, it is shown for an N /spl times/ N input queued, crossbar switch, that any on-line, packetizing algorithm with no speedup requires a buffer size of N/e - 2 packets per input port. We also extend the main packetizing algorithm in this paper to a general class of switch architectures.

The performance of collaborative beamforming is analyzed using the theory of random arrays. The statistical average and distribution of the beam patterns of randomly generated phased arrays are derived in the framework of wireless ad hoc sensor networks. Each sensor node is assumed to have a single isotropic antenna and nodes in the cluster collaboratively transmit the signal such that the signal in the target direction is coherently added in the far-field region. The distribution of the maximum beam pattern sidelobe is also analyzed and a closed-form bound is derived based on the Gaussian approximation method.

This paper describes the results of frequency-domain channel sounding in residential environments. It consists of detailed characterization of complex frequency responses of ultra-wideband (UWB) signals having a nominal center frequency of 5 GHz. A path loss model as well as a second-order autoregressive model is proposed for frequency response generation of the UWB indoor channel. Probability distributions of the model parameters for different locations are presented. Also, time-domain results such as root mean square delay spread and percent of captured power are presented.

Wireless data traffic is expected to grow over the next few years and the technologies that will provide data services are still being debated. One possibility is to use multiple antennas at base stations and terminals to get very high spectral efficiencies in rich scattering environments. Such multiple-input/multiple-output (MIMO) channels can then be used in conjunction with scheduling and rate-feedback algorithms to further increase channel throughput. This paper provides an analysis of the expected gains due to scheduling and bits needed for rate feedback. Our analysis requires an accurate approximation of the distribution of the MIMO channel mutual information. Because the exact distribution of the mutual information in a Rayleigh-fading environment is difficult to analyze, we prove a central limit theorem for MIMO channels with a large number of antennas. While the growth in average mutual information (capacity) of a MIMO channel with the number of antennas is well understood, it turns out that the variance of the mutual information can grow very slowly or even shrink as the number of antennas grows. We discuss implications of this "channel-hardening" result for data and voice services, scheduling, and rate feedback. We also briefly discuss the implications when shadow fading effects are included.

In this study we provide a physical layer based analysis of the coexistence issues of ultra-wideband (UWB) with other devices in the same spectrum. Specifically, we have focused on the UWB interference to and from devices using the wireless local area network (WLAN) standard IEEE 802.11a. Our results indicate that a UWB interferer operating at the peak allowable power density induces minimal interference into such WLAN devices in line-of sight (LOS) scenarios, even at close range. However, in the non-LOS (NLOS) case, a UWB interferer can severely affect the data-rate sustainable by 802.11a systems. Moreover, 802.11a interference into UWB systems is shown to reduce the signal-to-interference ratio (SIR) by as much as 36 dB when the interferer is within LOS of the UWB receiver.

We present a statistical model for the multipath intensity profile (MIP) that can be used to construct an impulse response of a residential wireless channel. The model incorporates relevant parameters such as distance, global rms delay spread and the correlation between received multipaths. It describes the statistics of the MIP in detail. This model, together with a suitable path loss model, can be used to simulate residential wireless channels for performance evaluation of various communication systems. The MIP model is based on over 300,000 indoor frequency responses measured in a 1.25GHz bandwidth centered on 5-GHz, and simulation shows that it captures very well the main statistical properties of the measured data.

The propose differential space time block codes (STBC) using quadrature amplitude modulation (QAM), which can not be utilized in the conventional differential STBC. Since QAM constellations have a larger minimum distance compared with phase shift keying (PSK), the proposed method has the advantage of SNR gain compared with conventional differential STBC. The QAM signals are encoded in a manner similar to that of conventional differential STBCs. To decode QAM signals, the signals received are normalized by the channel power estimated forgoing training symbols, and then decoded with a conventional QAM decoder. When the transmission rate is more than 3 bits/channel use in time varying channels, the simulation results demonstrate that the proposed method with the channel power estimation outperforms the conventional differential STBC.

We describe a statistical model for the multipath intensity profile (MIP) in residential environments. The work is based on over 300,000 frequency response measurements at 712 locations in 23 homes. We show that, for non-line-of-sight (NLS) path, the MIP is well-modeled as a decaying exponential vs. delay, multiplied by a lognormal process over delay; and that a similar model applies to line-of-sight (LOS) paths. Moreover, we present simple statistical descriptions for the parameters of these functions.

We consider a multiple antenna system when combined array processing with space-time coding is used. We present variable rate space-time block codes for two, three, and four transmit antennas and optimize the transmit power so that the average bit-error rate (BER) is minimized. Numerical results show that this optimum power allocation scheme provides significant gain over the equal power allocation scheme. We then classify all the variable rate space-time block codes having the same code rates and identify the unique code that achieves the lowest BER. We explicitly compute the performance of the variable rate codes over a Rayleigh-fading channel. The proposed variable rate space-time block codes are useful for unequal error protection in multiple transmit antenna systems.

Space-time coding is a bandwidth and power efficient method of communication over fading channels that realizes the benefits of multiple transmit antennas. Specific codes have been constructed using design criteria derived for quasi-static flat Rayleigh or Rician fading, where channel state information is available at the receiver. It is evident that the practicality of space-time codes will be greatly enhanced if the derived design criteria remain valid in the absence of perfect channel state information. It is even more desirable that the design criteria not be unduly sensitive to frequency selectivity and to the Doppler spread. This paper presents a theoretical study of these issues beginning with the effect of channel estimation error. Here it is assumed that a channel estimator extracts fade coefficients at the receiver and for constellations with constant energy, it is proved that in the absence of ideal channel state information the design criteria for space-time codes is still valid. The analysis also demonstrates that standard channel estimation techniques can be used in conjunction with space-time codes provided that the number of transmit antennas is small. We also derive the maximum-likelihood detection metric in the presence of channel estimation errors. Next, the effect of multiple paths on the performance of space-time codes is studied for a slowly changing Rayleigh channel. It is proved that the presence of multiple paths does not decrease the diversity order guaranteed by the design criteria used to construct the space-time codes. Similar results hold for rapid fading channels with or without multiple paths. The conclusion is that the diversity order promised by space-time coding is achieved under a variety of mobility conditions and environmental effects.

We present a new construction of 16-QAM Golay sequences of length n = 2/sup m/. The number of constructed sequences is (14 + 12m)(m!/2)4/sup m+1/. When employed as a code in an orthogonal frequency-division multiplexing (OFDM) system; this set of sequences has a peak-to-mean envelope power ratio (PMEPR) of 3.6. By considering two specific subsets of these sequences, we obtain new codes with PMEPR bounds of 2.0 and 2.8 and respective code sizes of (2 + 2m)(m!/2)4/sup m+1/ and (4 + 4m)(m!/2)4/sup m+1/. These are larger than previously known codes for the same PMEPR bounds.

We consider the problem of reduction of computation cost by introducing redundancy in the number of ports as well as in the input and output sequences of computation modules. Using our formulation, the classical "communication scenario" is the case when a computation module has to recompute the input sequence at a different location or time with high fidelity and low bit-error rates. We then consider communication with different computational cost objective than that given by bit-error rate. An example is communication over deep submicrometer very-large scale integration (VLSI) buses where the expected energy consumption per communicated information bit is the cost of computation. We treat this scenario using tools from information theory and establish fundamental bounds on the achievable expected energy consumption per bit in deep submicrometer VLSI buses as a function of their utilization. Some of our results also shed light on coding schemes that achieve these bounds. We then prove that the best tradeoff between the expected energy consumption per bit and bus utilization can be achieved using codes constructed from typical sequences of Markov stationary ergodic processes. We use this observation to give a closed-form expression for the best tradeoff between the expected energy consumption per bit and the utilization of the bus. This expression, in principle, can be computed using standard numerical methods. The methodology developed here naturally extends to more general computation scenarios.

For an (N/sub 1/, N/sub 2/) interleaver of block size L, we prove that L/spl ges/N/sub 1/N/sub 2/. Conversely, we prove that whenever L/spl ges/N/sub 1/N/sub 2/, an (N/sub 1/, N/sub 2/) interleaver of block size L exists. Explicit algebraic constructions are provided. It is also proven that rectangular interleavers are not optimal.

We consider transmission using N transmit and reception using M receive antennas in a wireless environment assuming that neither the transmitter nor the receiver knows the channel coefficients. For the scenario that the transmission employs noncoherent T /spl times/ N unitary space-time codes and for a block-fading channel model where the channel is static during T channel uses and varies from T channel uses to the other, we establish the bound r /spl les/ min(T-N, N) on the diversity advantage rM provided by the code. In order to show that the requirement r /spl les/ min(T-N, N) cannot be relaxed, for any given R, N, T, and r /spl les/ min(T-N, N), we then construct unitary T /spl times/ N space-time codes of rate R that guarantee diversity advantage rM. Two constructions are given that are also amenable to simple encoding and noncoherent maximum-likelihood (ML) decoding algorithms.

We present a quadrature phase-shift keying (QPSK) coding scheme for multicarrier communications. The scheme supports simple encoding and extremely simple maximum-likelihood (ML) decoding algorithms for orthogonal frequency-division multiplexing (OFDM). Moreover, all the codewords have peak-to-mean envelope power ratio (PMEPR) of at most 6 dB.

We explicitly construct multiple transmit antenna differential encoding/decoding schemes based on generalized orthogonal designs. These constructions generalize the two transmit antenna differential detection scheme that we proposed before (Tarokh and Jafarkhani 2000).

Using a realization of 16-QAM as set addition of two scaled versions of the 4-PSK constellation, we construct 16-QAM sequences having low peak-to-mean envelope power ratios (PMEPR) from 4-PSK Golay sequences. Various upper bounds on the peak envelope powers of these sequences are established. Transmission using these sequences provides twice as much rate as transmission using complementary Golay sequences with a maximum PMEPR of 3.6 (5.56 dB).

We present a transmission scheme for exploiting diversity given by two transmit antennas when neither the transmitter nor the receiver has access to channel state information. The new detection scheme can use equal energy constellations and encoding is simple. At the receiver, decoding is achieved with low decoding complexity. The transmission provides full spatial diversity and requires no channel state side information at the receiver. The scheme can be considered as the extension of differential detection schemes to two transmit antennas.

For any code C defined over an equal energy constellation, it is first shown that at any time instance, the problem of determining codewords of C with high peak-to-average power ratios (PAPR) in a multicarrier communication system is intimately related to the problem of minimum-distance decoding of C. Subsequently, a method is proposed for computing the PAPR by minimum-distance decoding of C at many points of time. Moreover an upper bound on the error between this computed value and the true one is derived. Analogous results are established for codes defined over arbitrary signal constellations. As an application of this computational method, an approach for reducing the PAPR of C proposed by Jones and Wilkinson (1996) is revisited. This approach is based on introducing a specific phase shift to each coordinate of all the codewords where phase shifts are independent of the codewords and known both to the transmitter and the receiver. We optimize the phase shifts offline by applying our method for computing the PAPR for the coding scenario proposed by the ETSI BRAN Standardization Committee. Reductions of order 4.5 dB can be freely obtained using the computed phase shifts. Examples are provided showing that most of the gain is preserved when the computed optimal phase shifts are rounded to quantenary phase-shift keying (PSK), 8-PSK, and 16-PSK type phase shifts.

The first lower bound on the peak-to-average power ratio (PAPR) of a constant energy code of a given length n, minimum Euclidean distance and rate is established. Conversely, using a nonconstructive Varshamov-Gilbert style argument yields a lower bound on the achievable rate of a code of a given length, minimum Euclidean distance and maximum PAPR. The derivation of these bounds relies on a geometrical analysis of the PAPR of such a code. Further analysis shows that there exist asymptotically good codes whose PAPR is at most 8 log n. These bounds motivate the explicit construction of error-correcting codes with low PAPR. Bounds for exponential sums over Galois fields and rings are applied to obtain an upper bound of order (log n)/sup 2/ on the PAPRs of a constructive class of codes, the trace codes. This class includes the binary simplex code, duals of binary, primitive Bose-Chaudhuri-Hocquenghem (BCH) codes and a variety of their nonbinary analogs. Some open problems are identified.

The authors note a few misprints exist in the final version of the above-named paper [ibid., vol. 45, pp. 1456–1467, July 1999]. In Definitions 3.4.1, 4.1.1, and 5.4.1, also in the last line of the proof of Theorem 3.5.2.

Space-time block coding (STBC) is an effective transmit diversity scheme. It was presented previously by in terms of coding and decoding by Alamouti (see IEEE J. Select. Areas Commun., vol.16, no.8, p.1451-58, 1998) and by Tarokh, Jafarkhani and Calderbank (see IEEE Trans. Inform. Theory, 1999). Here, STBC is examined from a different perspective; namely the radiation patterns resulting from simultaneous transmission from multiple transmit antennas.

We present a transmission scheme for exploiting diversity given by two transmit antennas when neither the transmitter nor the receiver has access to channel state information. The new detection scheme can use PSK constellations and encoding is simple. At the receiver, decoding is achieved with low decoding complexity. The transmission provides full spatial diversity and requires no channel state side information at the receiver. The scheme can be thought as the extension of differential detection schemes to two transmit antennas.

We document the performance of space-time block codes, which provide a new paradigm for transmission over Rayleigh fading channels using multiple transmit antennas. Data is encoded using a space-time block code, and the encoded data is split into n streams which are simultaneously transmitted using n transmit antennas. The received signal at each receive antenna is a linear superposition of the n transmitted signals perturbed by noise. Maximum likelihood decoding is achieved in a simple way through decoupling of the signals transmitted from different antennas rather than joint detection. This uses the orthogonal structure of the space-time block code and gives a maximum likelihood decoding algorithm which is based only on linear processing at the receiver. We review the encoding and decoding algorithms for various codes and provide simulation results demonstrating their performance. It is shown that using multiple transmit antennas and space-time block coding provides remarkable performance at the expense of almost no extra processing.

We present a construction of multiple description trellis-coded quantizers. We use the tensor product of trellises to build a trellis which is applicable to multiple description coding. The problems of index assignment and set partitioning for the resulting trellis are considered. The Viterbi algorithm provides the best path for encoding and the design procedure utilizes a generalized Lloyd algorithm. The encoding process simultaneously generates all the transmitted sequences. Furthermore, the complexity of the scheme is almost independent of the rate. The quantizer provides remarkable performance with little encoding complexity.

Space-time coding is a bandwidth and power efficient method of communication over fading channels that realizes the benefits of multiple transmit antennas. Specific codes have been constructed using design criteria derived for quasi-static flat Rayleigh or Rician fading, where channel state information is available at the receiver. It is evident that the practicality of space-time codes will be greatly enhanced if the derived design criteria remain valid in the absence of perfect channel state information. It is even more desirable that the design criteria not be unduly sensitive to frequency selectivity and to the Doppler spread. This paper presents a theoretical study of these issues beginning with the effect of channel estimation error. Here it is assumed that a channel estimator extracts fade coefficients at the receiver and for constellations with constant energy, it is proved that in the absence of ideal channel state information the design criteria for space-time codes is still valid. The analysis also demonstrates that standard channel estimation techniques can be used in conjunction with space-time codes provided that the number of transmit antennas is small. We also derive the maximum-likelihood detection metric in the presence of channel estimation errors. Next, the effect of multiple paths on the performance of space-time codes is studied for a slowly changing Rayleigh channel. It is proved that the presence of multiple paths does not decrease the diversity order guaranteed by the design criteria used to construct the space-time codes. Similar results hold for rapid fading channels with or without multiple paths. The conclusion is that the diversity order promised by space-time coding is achieved under a variety of mobility conditions and environmental effects.

We propose successively refinable trellis-coded quantizers for progressive transmission. (Progressive transmission is an essential component of image and multimedia browsing systems.) A new trellis structure which is scalable is used in the design of our trellis-coded quantizers. A hierarchical set partitioning is developed to preserve successive refinability. Two algorithms for designing trellis-coded quantizers which provide embedded bit streams are provided. The computational complexity of the proposed schemes is compared with that of trellis-coded quantization. Simulation results show excellent performances for memoryless sources.

We introduce space-time block coding, a new paradigm for communication over Rayleigh fading channels using multiple transmit antennas. Data is encoded using a space-time block code and the encoded data is split into n streams which are simultaneously transmitted using n transmit antennas. The received signal at each receive antenna is a linear superposition of the n transmitted signals perturbed by noise. Maximum-likelihood decoding is achieved in a simple way through decoupling of the signals transmitted from different antennas rather than joint detection. This uses the orthogonal structure of the space-time block code and gives a maximum-likelihood decoding algorithm which is based only on linear processing at the receiver. Space-time block codes are designed to achieve the maximum diversity order for a given number of transmit and receive antennas subject to the constraint of having a simple decoding algorithm. The classical mathematical framework of orthogonal designs is applied to construct space-time block codes. It is shown that space-time block codes constructed in this way only exist for few sporadic values of n. Subsequently, a generalization of orthogonal designs is shown to provide space-time block codes for both real and complex constellations for any number of transmit antennas. These codes achieve the maximum possible transmission rate for any number of transmit antennas using any arbitrary real constellation such as PAM. For an arbitrary complex constellation such as PSK and QAM, space-time block codes are designed that achieve 1/2 of the maximum possible transmission rate for any number of transmit antennas. For the specific cases of two, three, and four transmit antennas, space-time block codes are designed that achieve, respectively, all, 3/4, and 3/4 of maximum possible transmission rate using arbitrary complex constellations. The best tradeoff between the decoding delay and the number of transmit antennas is also computed and it is shown that many of the codes presented here are optimal in this sense as well.

The information capacity of wireless communication systems may be increased dramatically by employing multiple transmit and receive antennas. The goal of system design is to exploit this capacity in a practical way. An effective approach to increasing data rate over wireless channels is to employ space-time coding techniques appropriate to multiple transmit antennas. These space-time codes introduce temporal and spatial correlation into signals transmitted from different antennas, so as to provide diversity at the receiver, and coding gain over an uncoded system. For large number of transmit antennas and at high bandwidth efficiencies, the receiver may become too complex whenever correlation across transmit antennas is introduced. This paper dramatically reduces encoding and decoding complexity by partitioning antennas at the transmitter into small groups, and using individual space-time codes, called the component codes, to transmit information from each group of antennas. At the receiver, an individual space-time code is decoded by a novel linear processing technique that suppresses signals transmitted by other groups of antennas by treating them as interference. A simple receiver structure is derived that provides diversity and coding gain over uncoded systems. This combination of array processing at the receiver and coding techniques for multiple transmit antennas can provide reliable and very high data rate communication over narrowband wireless channels. A refinement of this basic structure gives rise to a multilayered space-time architecture that both generalizes and improves upon the layered space-time architecture proposed by Foschini (see Bell Labs Tech. J., vol.1, no.2, 1996).

We present a lower bound on the probability of symbol error for maximum-likelihood decoding of lattices and lattice codes on a Gaussian channel. The bound is tight for error probabilities and signal-to-noise ratios of practical interest, as opposed to most existing bounds that become tight asymptotically for high signal-to-noise ratios. The bound is also universal; it provides a limit on the highest possible coding gain that may be achieved, at specific symbol error probabilities, using any lattice or lattice code in n dimensions. In particular, it is shown that the effective coding gains of the densest known lattices are much lower than their nominal coding gains. The asymptotic (as n/spl rarr//spl infin/) behavior of the new bound is shown to coincide with the Shannon (1948) limit for Gaussian channels.

We propose successively refinable trellis coded quantizers which are suitable for progressive transmission. A new trellis structure which is scalable is used in the design of our trellis coded quantizers. A hierarchical set partitioning is used to preserve successive refillability. Two algorithms for designing trellis coded quantizers which provide embedded bit streams are provided. The computational complexity of the proposed schemes is compared with that of trellis coded quantization. Simulation results show good performance for memoryless sources.

We present a construction of multiple description trellis coded quantizers. We use the tensor product of trellises to build a trellis which is applicable to multiple description coding. The problems of index assignment and set partitioning for the resulting trellis are considered. The Viterbi algorithm provides the best path for encoding and the design procedure utilizes a generalized Lloyd algorithm. The encoding process simultaneously generates all the transmitted sequences. Furthermore, the complexity of the scheme is almost independent of the rate. The quantizer provides remarkable performance with little encoding complexity.

This paper presents the theory and practice of a new advanced modem technology suitable for high-data-rate wireless communications and presents its performance over a frequency-flat Rayleigh fading channel. The new technology is based on space-time coded modulation (STCM) with multiple transmit and/or multiple receive antennas and orthogonal pilot sequence insertion (O-PSI). In this approach, data is encoded by a space-time (ST) channel encoder and the output of the encoder is split into N streams to be simultaneously transmitted using N transmit antennas. The transmitter inserts periodic orthogonal pilot sequences in each of the simultaneously transmitted bursts. The receiver uses those pilot sequences to estimate the fading channel. When combined with an appropriately designed interpolation filter, accurate channel state information (CSI) can be estimated for the decoding process. Simulation results of the proposed modem, as applied to the IS-136 cellular standard, are presented. We present the frame error rate (FER) performance results as a function of the signal-to-noise ratio (SNR) and the maximum Doppler frequency, in the presence of timing and frequency offset errors. Simulation results show that for a 10% FER, a 32-state eight-phase-shift keyed (8-PSK) ST code with two transmit and two receive antennas can support data rates up to 55.8 kb/s on a 30-kHz channel, at an SNR of 11.7 dB and a maximum Doppler frequency of 180 Hz. Simulation results for other codes and other channel conditions are also provided. We also compare the performance of the proposed STCM scheme with delay diversity schemes and conclude that STCM can provide significant SNR improvement over simple delay diversity.

We consider the design of channel codes for improving the data rate and/or the reliability of communications over fading channels using multiple transmit antennas. Data is encoded by a channel code and the encoded data is split into n streams that are simultaneously transmitted using n transmit antennas. The received signal at each receive antenna is a linear superposition of the n transmitted signals perturbed by noise. We derive performance criteria for designing such codes under the assumption that the fading is slow and frequency nonselective. Performance is shown to be determined by matrices constructed from pairs of distinct code sequences. The minimum rank among these matrices quantifies the diversity gain, while the minimum determinant of these matrices quantifies the coding gain. The results are then extended to fast fading channels. The design criteria are used to design trellis codes for high data rate wireless communication. The encoding/decoding complexity of these codes is comparable to trellis codes employed in practice over Gaussian channels. The codes constructed here provide the best tradeoff between data rate, diversity advantage, and trellis complexity. Simulation results are provided for 4 and 8 PSK signal sets with data rates of 2 and 3 bits/symbol, demonstrating excellent performance that is within 2-3 dB of the outage capacity for these channels using only 64 state encoders.

We revisit space-time codes for a mobile communication system that employs multiple antennas at the base station and optional antenna diversity at the mobile station. The realistic case when the channel state is not completely known is considered. It is assumed that the channel estimator extracts the fade coefficients using orthogonal pilot tones. Mismatch analysis is then carried out. It is proved that in the absence of ideal channel state information the design criteria for space-time codes developed in Tarokh et al. (1997) is still valid for the equal energy constellation case. Using our derivation, it is observed that channel estimation techniques commonly used over rapidly fading channels can be used in conjunction with space-time codes provided that the number of transmit antennas is small.

We consider the design of channel codes for improving the data rate and/or the reliability of communications over fading channels using multiple transmit antennas. Here, data is encoded by a channel code and the encoded data is split into n streams that are simultaneously transmitted using n transmit antennas. The received signal at each receive antenna is a linear superposition of the n transmitted signals. We derive performance criteria for designing channel codes under the assumption that the fading is slow and frequency non-selective. Performance is shown to be determined by diversity gain quantified by ranks and coding gain quantified by determinants of certain matrices that are constructed from the code sequences.

We consider the design of channel codes for improving the data rate and/or the reliability of communications using multiple transmit antennas over a fading channel. It is assumed that the transmitter does not know the channel but seeks to choose a codebook that guarantees a diversity gain of r/sub 1/ when there is no mobility and a diversity gain of r/sub 2//spl ges/r/sub 1/ when the channel is fast fading. A solution to this problem is unveiled in this paper. Here, the encoded data is split into n streams that are simultaneously transmitted using n transmit antennas. The signal received at each receive antenna is a superposition of the faded versions of the n transmitted signals. We derive performance criteria for designing codes having the aforementioned properties. Performance is shown to be determined by diversity advantage quantified by a rank/distance and coding advantage quantified by a determinant/product criterion. The criteria is used to design codes for both slow and rapid fading channels. The constructed codes have remarkable performance in low signal to noise ratios and are suitable for improving the frequency reuse factor under a variety of mobility conditions.

It is proven that for every random variable with a countably infinite set of outcomes and finite entropy there exists an optimal prefix code which can be constructed from Huffman codes for truncated versions of the random variable, and that the average lengths of any sequence of Huffman codes for the truncated versions converge to that of the optimal code. Also, it is shown that every optimal infinite code achieves Kraft's inequality with equality.

Unlike block codes, n-dimensional lattices can have minimal trellis diagrams with an arbitrarily large number of states, branches, and paths. In particular, we show by a counterexample that there is no f(n), a function of n, such that all rational lattices of dimension n have a trellis with less than f(n) states. Nevertheless, using a theorem due to Hermite, we prove that every integral lattice /spl Lambda/ of dimension n has a trellis T, such that the total number of paths in T is upper-bounded by P(T)/spl les/n!(2//spl radic/3)/sup n2(n-1/2)/V(/spl Lambda/)/sup n-1/ where V(n) is the volume of /spl Lambda/. Furthermore, the number of states at time i in T is upper-bounded by |S/sub i/|/spl les/(2//spl radic/3)/sup i2(n-1)/V(/spl Lambda/)/sup 2i2/n/. Although these bounds are seldom tight, these are the first known general upper bounds on trellis complexity of lattices.

An inequality relating the trellis complexity of lattices to their dimension and Hermite parameter is established. Using this inequality, a conjecture of Forney is proved indicating that the trellis complexity of the densest lattice packings in $\mathbb{R}^n $ grows exponentially as a function of their coding gain.

The best possible tradeoff between the coding gain and trellis complexity for lattices is studied. Three trellis complexity functions are defined for lattices as a measure of minimum trellis decoding complexity per dimension required for achieving a coding gain /spl gamma/. The properties of these functions are studied from an analytic perspective. It is also shown that the trellis decoding complexity per dimension is lower-bounded by an explicit power of /spl gamma/.

For pt.I see ibid., vol. 42, no.6, p.1796-1802, 1996. Every rational lattice has a finite trellis diagram which can be employed for maximum-likelihood decoding over the additive white Gaussian noise channel via the Viterbi algorithm. For an arbitrary rational lattice L with gain /spl gamma/, the average number of states (respectively, branches) in any given trellis diagram of L is bounded below by a function of /spl gamma/. It is proved that this function grows exponentially in /spl gamma/. In the reverse direction, it is proved that given /spl isin/>0, for arbitrarily large values of /spl gamma/, there exist lattices of gain /spl gamma/ with an average number of branches and states less than exp(/spl gamma//sup (1+/spl isin//)). Trellis diagrams of block codes obtained from truncated convolutional codes are employed to show that, inside the trellis model, the problem of decoding lattices is not much harder than exponential.