Sparse Bayesian Learning (SBL) is a popular sparse signal recovery method, and various algorithms exist under the SBL paradigm. In this paper, we introduce a novel re-parameterization that allows the iterations of existing algorithms to be viewed as special cases of a unified and general mapping function. Furthermore, the re-parameterization enables an interesting beamforming interpretation that lends insights to all the considered algorithms. Utilizing the abstraction allowed by the general mapping viewpoint, we introduce a novel neural network architecture for learning improved iterative update rules under the SBL framework. Our modular design of the architecture enables the model to be independent of the size of the measurement matrix and provides us a unique opportunity to test the generalization capabilities across different measurement matrices. We show that the network when trained on a particular parameterized dictionary generalizes in many ways hitherto not possible; different measurement matrices, both type and dimension, and number of snapshots. Our numerical results showcase the generalization capability of our network in terms of mean square error and probability of support recovery across sparsity levels, different signal-to-noise ratios, number of snapshots and multiple measurement matrices of different sizes.

(Covariance matrix reconstruction has been the most widely used guiding objective in gridless direction-of-arrival (DoA) estimation for sparse linear arrays. Many semidefinite programming (SDP)-based methods fall under this category. Although deep learning-based approaches enable the construction of more sophisticated objective functions, most methods still rely on covariance matrix reconstruction. In this paper, we propose new loss functions that are invariant to the scaling of the matrices and provide a comparative study of losses with varying degrees of invariance. The proposed loss functions are formulated based on the scale-invariant signal-to-distortion ratio between the target matrix and the Gram matrix of the prediction. Numerical results show that a scale-invariant loss outperforms its non-invariant counterpart but is inferior to the recently proposed subspace loss that is invariant to the change of basis. These results provide evidence that designing loss functions with greater degrees of invariance is advantageous in deep learning-based gridless DoA estimation.

This paper introduces new and practically relevant non-Gaussian priors for the Sparse Bayesian Learning (SBL) framework applied to the Multiple Measurement Vector (MMV) problem. We extend the Gaussian Scale Mixture (GSM) framework to model prior distributions for row vectors, exploring the use of shared and different hyperparameters across different measurements. We propose Expectation Maximization (EM) based algorithms to estimate the parameters of the prior density along with the hyperparameters. To promote sparsity more effectively in a non-Gaussian setting, we show the importance of incorporating learning of the parameters of the mixing density. Such an approach effectively utilizes the common support notion in the MMV problem and promotes sparsity without explicitly imposing a sparsity-promoting prior, indicating the methods’ robustness to model mismatches. Numerical simulations are provided to compare the proposed approaches with the existing SBL algorithm for the MMV problem.

We consider the problem of adaptive sensing for multi-path channel estimation in millimeter-wave (mmWave) communications system with single RF chain. Current adaptive sensing approaches either focus on estimating the single dominant path or assume apriori knowledge of the number of multi-path components. A key challenge in Bayesian adaptive sensing is the choice of prior which captures the geometric structure of mmWave channels and leads to tractable adaptive sensing methods. In this work, we propose to learn a flexible empirical prior through supervised training that allows for test-time adaptations to the specific channel environment. We realize our framework using a novel recurrent neural network architecture which learns the prior implicitly as part of the recurrent update rule. At test time, our approach alternates between adaptively designing the beamformer to acquire measurement, estimating the channel, and refining the hyperparameters of the learned empirical prior based on history. We demonstrate our approach produces interpretable beamformers and generalizes to channel configurations with fewer multi-path components than it was trained upon.

We present a general Bernoulli Gaussian scale mixture based approach for modeling priors that can represent a large class of random signals. For inference, we introduce belief propagation (BP) to multi-snapshot signal recovery based on the minimum mean square error estimation criteria. Our method relies on intra-snapshot messages that update the signal vector for each snapshot and inter-snapshot messages that share probabilistic information related to the common sparsity structure across snapshots. Despite the very general model, our BP method can efficiently compute accurate approximations of marginal posterior PDFs. Preliminary numerical results illustrate the superior convergence rate and improved performance of the proposed method compared to approaches based on sparse Bayesian learning (SBL).

We address the problem of the beam split (B-SP) caused by the spatial wideband (SW) effect in intelligent reflecting surface (IRS) aided wideband systems. The SW effect arises when the signal delay across the IRS aperture is comparable to the system sampling time. This leads to a B-SP, wherein the IRS phase shifters fail to form a beam at the desired user equipment (UE) over the complete bandwidth (BW) allotted to that UE, in turn reducing the system throughput. This paper proposes a distributed IRS design that aids in naturally combating the SW/BSP effects by parallelizing the spatial delays across the multiple IRSs. Specifically, we propose to split a single large IRS into multiple smaller IRSs and distribute them over the geographical area. Then, by specifying the maximum permissible number of elements at each IRS, we ensure that the B-SP remains within a specified tolerance level. Finally, we derive the sum-rate and show that the proposed solution yields a nonzero rate over the whole BW of operation and circumvents the SW/B-SP effects. We verify our findings numerically and illustrate the low complexity of the proposed method compared to state-of-the-art techniques.

This letter addresses the problem of estimating block sparse signal with unknown group partitions in a multiple measurement vector (MMV) setup. We propose a Bayesian framework by applying an adaptive total variation (TV) penalty on the hyper-parameter space of the sparse signal. The main contributions are two-fold. 1) We extend the TV penalty beyond the immediate neighbor, thus enabling better capture of the signal structure. 2) A dynamic framework is provided to learn the regularization weights for the TV penalty based on the statistical dependencies between the entries of tentative blocks, thus eliminating the need for fine-tuning. The superior performance of the proposed method is empirically demonstrated by extensive computer simulations with the state-of-art benchmarks. The proposed solution exhibits both excellent performance and robustness against sparsity model mismatch.

We examine the problem of uplink cell-free access point (AP) placement in the context of optimal throughput. In this regard, we formulate two main placement problems, namely the sum rate and minimum rate maximization problems, and discuss the challenges associated with solving the underlying optimization problems with the help of some simple scenarios. As a practical solution to the AP placement problem, we suggest a vector quantization (VQ) approach. The suitability of the VQ approach to cell-free AP placement is investigated by examining three VQ-based solutions. First, the standard VQ approach, that is the Lloyd algorithm (using the squared error distortion function) is described. Second, the tree-structured VQ (TSVQ), which performs successive partitioning of the distribution space is applied. Third, a probability density function optimized VQ (PDFVQ) procedure is outlined, enabling efficient, low complexity, and scalable placement, and is aimed at a massive distributed multiple-input-multiple-output scenario. While the VQ-based solutions do not explicitly solve the cell-free AP placement problems, numerical experiments show that their sum and minimum rate performances are good enough, and offer a good starting point for gradient-based optimization methods. Among the VQ solutions, PDFVQ, with its distinct advantages, offers a good trade-off between sum and minimum rates.

One-bit massive MIMO has gained much attention in the areas of wireless communication and sensing. Among the various receiver designs, the maximum-likelihood-based receivers achieve state-of-the-art performance. Through this work we provide both analytical insight into the likelihood formulation, and develop a one-bit MIMO receiver, motivated specifically from this analysis. In particular, (i) Properties of the original Gaussian CDF based likelihood function are analyzed, culminating in an improved gradient descent (GD) algorithm for one-bit MIMO. (ii) This improved GD update rule is further enhanced through an accelerated GD method, improving convergence performance. (iii) The likelihood analysis is extended to an effective surrogate function for the Gaussian CDF, i.e., the logistic regression (LR). The presented analytical framework for the CDF also serves as a robust mathematical model to explain the enhanced performance of the LR, when utilized as a surrogate likelihood. (iv) Detection from a finite M-QAM constellation is incorporated by introducing a Gaussian denoiser to project the detected symbols onto the M-QAM subspace. This is implemented as a novel, unfolded, DNN architecture for one-bit detection. Through our experimental validation we demonstrate results on par with the current state-of-the-art methods for one-bit MIMO detection.

Deep neural networks (DNNs) have greatly benefited direction of arrival (DoA) estimation methods for speech source localization in noisy environments. However, their localization accuracy is still far from satisfactory due to the vulnerability to nonspeech interference. To improve the robustness against interference, we propose a DNN based normalized time-frequency (T-F) weighted criterion which minimizes the distance between the candidate steering vectors and the filtered snapshots in the T-F domain. Our method requires no eigendecomposition and uses a simple normalization to prevent the optimization objective from being misled by noisy filtered snapshots. We also study different designs of T-F weights guided by a DNN. We find that duplicating the Hadamard product of speech ratio masks is highly effective and better than other techniques such as direct masking and taking the mean in the proposed approach. However, the best-performing design of T-F weights is criterion-dependent in general. Experiments show that the proposed method outperforms popular DNN based DoA estimation methods including widely used subspace methods in noisy and reverberant environments.

We present a Light-Weight Sequential Sparse Bayesian Learning (LWS-SBL) algorithm as an alternative to the orthogonal matching pursuit (OMP) algorithm for the general sparse signal recovery problem. The proposed approach formulates the recovery problem under the Type-II estimation framework and the stochastic maximum likelihood objective. We compare the computational complexity for the proposed algorithm with OMP and highlight the main differences. For the case of parametric dictionaries, a gridless version is developed by extending the proposed sequential SBL algorithm to locally optimize grid points near potential source locations and it is empirically shown that the performance approaches Cramer-Rao bound.´ Numerical results using the proposed approach demonstrate the support recovery performance improvements in different scenarios at a small computational price when compared to the OMP algorithm.

Multi-user massive MIMO signal detection from one-bit received measurements strongly depends on the wireless channel. To this end, majority of the model and learning-based approaches address detector design for the rich-scattering, homogeneous Rayleigh fading channel. Our work proposes detection for one-bit massive MIMO for the lower diversity mmWave channel. We analyze the limitations of the current state-of-the-art gradient descent (GD)-based joint multiuser detection of one-bit received signals for the mmWave channels. Addressing these, we introduce a new framework to ensure equitable per-user performance, in spite of joint multi-user detection. This is realized by means of: (i) a parametric deep learning system, i.e., the mmW-ROBNet, (ii) a constellation-aware loss function, and (iii) a hierarchical detection training strategy. The experimental results corroborate this proposed approach for equitable per-user detection.

We address the uplink small-cell access point (AP) placement problem for optimal throughput, while considering load balancing (LB) among the APs. To consider LB and consequently incorporate fairness in user spectral access, i.e., the frequency of user-to-AP communications, we modify the Lloyd algorithm from vector quantization so that delays incurred by the existence of a large number of users in a cell are accounted for in the AP placement process. Accordingly, we present two methods, the first of which involves the incorporation of weights proportional to the cell occupancy, hence called the Occupancy Weighted Lloyd algorithm (OWLA). The second method adds a new step to the Lloyd algorithm, which involves re-assigning users from higher to lower occupancy cells, and the adoption of a distance threshold to cap the throughput lost in the assignment process. This formulated Lloyd-type algorithm is called the Cell Equalized Lloyd Algorithm-α (CELA-α) where α is a factor that allows for throughput and spectrum access delay trade-off. Extensive simulations show that both CELA-α and OWLA algorithms provide significant gains, in comparison to the standard Lloyd algorithm, in 95%-likely user spectral access. For the α values considered in this paper, CELA-α achieves gains up to 20.83%, while OWLA yields a gain of 12.5%. Both algorithms incur minimal throughput losses of different degrees, and the choice of using one algorithm over the other for AP placement depends on system LB as well as throughput requirements.

We propose a data-driven partial differential equation (PDE) identification scheme based on ℓ1-norm minimization which can identify spatially-dependent PDEs from measurements. Spatially-dependent PDEs refers to that the terms in the PDEs vary across space. In reality a physical system is often governed by spatially-dependent PDEs because the properties of the medium can be various across space, and the proposed method is the first data-driven spatially-dependent PDEs identification scheme. In addition, our method is efficient owing to its non-iterative nature and efficient implementation by coordinate descent.1

A deep neural network using rectified linear units represents a continuous piecewise linear (CPWL) function and vice versa. Recent results in the literature estimated that the number of neurons needed to exactly represent any CPWL function grows exponentially with the number of pieces or exponentially in terms of the factorial of the number of distinct linear components. Moreover, such growth is amplified linearly with the input dimension. These existing results seem to indicate that the cost of representing a CPWL function is expensive. In this paper, we propose much tighter bounds and establish a polynomial time algorithm to find a network satisfying these bounds for any given CPWL function. We prove that the number of hidden neurons required to exactly represent any CPWL function is at most a quadratic function of the number of pieces. In contrast to all previous results, this upper bound is invariant to the input dimension. Besides the number of pieces, we also study the number of distinct linear components in CPWL functions. When such a number is also given, we prove that the quadratic complexity turns into bilinear, which implies a lower neural complexity because the number of distinct linear components is always not greater than the minimum number of pieces in a CPWL function. When the number of pieces is unknown, we prove that, in terms of the number of distinct linear components, the neural complexities of any CPWL function are at most polynomial growth for low-dimensional inputs and factorial growth for the worst-case scenario, which are significantly better than existing results in the literature.

In this work, we addressthe MIMO semi-blindchannel estimation problem. We propose an eigenvalue decomposition based technique to significantly reduce the dimensionality of the EM based algorithm when the imposed prior on the data is Gaussian, greatly lowering the computational complexity. In addition to that, we apply the Minimum Power Distortionless Response (MPDR) decoupling principle to derive a tractable EM algorithm that uses the actual discrete prior of the data symbols. Our results show that the proposed MPDR based algorithm has superior performance over other EM based algorithms in both low and high SNR regions. The results also show that a faster version of the algorithm can be obtained by initializing it using the eigenvalue decomposition based Gaussian algorithm.

In this paper, we explore the small-cell uplink access point (AP) placement problem in the context of throughput optimality and provide solutions while taking into consideration inter-cell interference (ICI). First, we briefly review the vector quantization (VQ) approach and related single user throughput-optimal formulations for AP placement. Then, we investigate the small-cell case with multiple users and expose the limitations of mean squared error based VQ for solving this problem. While the Lloyd algorithm from the VQ approach is found not to strictly solve the small-cell case, based on the tractability and quality of the resulting AP placement, we deem it suitable as a simple and appropriate framework to solve more complicated problems. Accordingly, to minimize ICI and consequently enhance achievable throughput, we design two Lloyd-type algorithms, namely the Interference Lloyd algorithm and the Inter-AP Lloyd algorithm, both of which incorporate ICI in their distortion functions. Simulation results show that both of the proposed algorithms provide superior 95%-likely rate over the traditional Lloyd algorithm and the Inter-AP Lloyd algorithm yields a significant increase of up to 36.34% in achievable rate over the Lloyd algorithm.

Sparse signal recovery algorithms like sparse Bayesian learning work well but the complexity quickly grows when tackling higher dimensional parametric dictionaries. In this work we propose a novel Bayesian strategy to address the two dimensional harmonic retrieval problem, through remodeling and reparameterization of the standard data model. This new model allows us to introduce a block sparsity structure in a manner that enables a natural pairing of the parameters in the two dimensions. The numerical simulations demonstrate that the inference algorithm developed (H-MSBL) does not suffer from source identifiability issues and is capable of estimating the harmonic components in challenging scenarios, while maintaining a low computational complexity.

Block-sparse signal recovery without knowledge of block sizes and boundaries, such as those encountered in multi-antenna mmWave channel models, is a hard problem for compressed sensing (CS) algorithms. We propose a novel Sparse Bayesian Learning (SBL) method for block-sparse recovery based on popular CS based regularizers with the function input variable related to total variation (TV). Contrary to conventional approaches that impose the regularization on the signal components, we regularize the SBL hyperparameters. This iterative TV-regularized SBL algorithm employs a majorization-minimization approach and reduces each iteration to a convex optimization problem, enabling a flexible choice of numerical solvers. The numerical results illustrate that the TV-regularized SBL algorithm is robust to the nature of the block structure and able to recover signals with both block-patterned and isolated components, proving useful for various signal recovery systems.

Models recently used in the literature proving residual networks (ResNets) are better than linear predictors are actually different from standard ResNets that have been widely used in computer vision. In addition to the assumptions such as scalar-valued output or single residual block, the models fundamentally considered in the literature have no nonlinearities at the final residual representation that feeds into the final affine layer. To codify such a difference in nonlinearities and reveal a linear estimation property, we define ResNEsts, i.e., Residual Nonlinear Estimators, by simply dropping nonlinearities at the last residual representation from standard ResNets. We show that wide ResNEsts with bottleneck blocks can always guarantee a very desirable training property that standard ResNets aim to achieve, i.e., adding more blocks does not decrease performance given the same set of basis elements. To prove that, we first recognize ResNEsts are basis function models that are limited by a coupling problem in basis learning and linear prediction. Then, to decouple prediction weights from basis learning, we construct a special architecture termed augmented ResNEst (A-ResNEst) that always guarantees no worse performance with the addition of a block. As a result, such an A-ResNEst establishes empirical risk lower bounds for a ResNEst using corresponding bases. Our results demonstrate ResNEsts indeed have a problem of diminishing feature reuse; however, it can be avoided by sufficiently expanding or widening the input space, leading to the above-mentioned desirable property. Inspired by the densely connected networks (DenseNets) that have been shown to outperform ResNets, we also propose a corresponding new model called Densely connected Nonlinear Estimator (DenseNEst). We show that any DenseNEst can be represented as a wide ResNEst with bottleneck blocks. Unlike ResNEsts, DenseNEsts exhibit the desirable property without any special architectural re-design.

SSR methods that majorize the regularized objective function during the optimization process. We show that introducing the majorizers leads to the same algorithm as simply using the gradient update of the regularized objective function, as is done in existing approaches. Different from the past works, the reweighting formulation naturally leads to an affine scaling transformation (AST) strategy, which effectively introduces a diagonal weighting on the gradient, giving rise to new algorithms that demonstrate improved convergence properties. Interestingly, setting the regularization coefficient to zero in the proposed AST-based framework leads to the Sparsity-promoting LMS (SLMS) and Sparsity-promoting Normalized LMS (SNLMS) algorithms, which exploit but do not strictly enforce the sparsity of the system response if it already exists. The SLMS and SNLMS realize proportionate adaptation for convergence speedup should sparsity be present in the underlying system response. In this manner, we develop a new way for rigorously deriving a large class of proportionate algorithms, and also explain why they are useful in applications where the underlying systems admit certain sparsity, e.g., in acoustic echo and feedback cancellation.

While deep neural networks (DNNs) have achieved state-of-the-art results in many fields, they are typically over-parameterized. Parameter redundancy, in turn, leads to inefficiency. Sparse signal recovery (SSR) techniques, on the other hand, find compact solutions to overcomplete linear problems. Therefore, a logical step is to draw the connection between SSR and DNNs. In this paper, we explore the application of iterative reweighting methods popular in SSR to learning efficient DNNs. By efficient, we mean sparse networks that require less computation and storage than the original, dense network. We propose a reweighting framework to learn sparse connections within a given architecture without biasing the optimization process, by utilizing the affine scaling transformation strategy. The resulting algorithm, referred to as Sparsity-promoting Stochastic Gradient Descent (SSGD), has simple gradient-based updates which can be easily implemented in existing deep learning libraries. We demonstrate the sparsification ability of SSGD on image classification tasks and show that it outperforms existing methods on the MNIST and CIFAR-10 datasets.

In this paper we analyse the performance of Sparse Bayesian Learning (SBL) in an environment with correlated sources. We provide two new perspectives to understand SBL's strategy for handling correlated sources. Using a Minimum Power Distortionless Response (MPDR) beamformer-based perspective of SBL, it is shown that the measured signal covariance strucure based on uncorrelated sources assumption used by SBL, infact provides immunity to source correlation. The MPDR based perspective is extended to formulate SBL's misspecified model. This misspecified model is compared to the underlying true data distribution using Kullback-Leibler (KL) divergence metric and it is shown that SBL attempts to find an uncorrelated source covariance matrix that best fits the data. Theory on performance of misspecified models for the two source case is studied to gain further insights into the problem. Numerical results are provided to demonstrate SBL's robustness as well as to quantify the model misfit when sources are correlated.

Accurate and efficient methods for Direction of Arrival (DOA) estimation play an important role in mmWave channel estimation methods. This estimation procedure can potentially be affected by the different RF and analog components in the communication system. Such components add an unknown, nonlinear distortion to the received signal. We address the DOA estimation problem for a general nonlinear distortion of the received signal. Two different angle recovery scenarios are considered: with and without the use of pilot symbols. The analysis focuses on the transformation of the signal subspace, resulting from the communication system nonlinearities. We prove that under certain conditions on the nonlinearity, the signal subspace is preserved. The derived result implies that the angles can be recovered without calibrating or inverting the nonlinearity.

In this letter, we propose a novel conjugate gradient (CG) adaptive filtering algorithm for online estimation of system responses that admit sparsity. Specifically, the Sparsity-promoting Conjugate Gradient (SCG) algorithm is developed based on iterative reweighting methods popular in the sparse signal recovery area. We propose an affine scaling transformation strategy within the reweighting framework, leading to an algorithm that allows the usage of a zero sparsity regularization coefficient. This enables SCG to leverage the sparsity of the system response if it already exists, while not compromising the optimization process. Simulation results show that SCG demonstrates improved convergence and steady-state properties over existing methods.

Massive machine-type communications (mMTC) or massive Internet of things (IoT) is one of the key application scenarios of fifth generation (5G) and beyond cellular networks. Since initial access procedures of legacy systems are not suitable for massive connectivity due to increased collision probability and prohibitive overhead, it is of great interest to design an efficient grant-free access protocol. In this paper, we propose an uncoordinated access protocol for massive connectivity, which leverages a massive number of antennas at the base station (BS), which is expected to be widely deployed in cellular networks. With the proposed scheme consisting of a sparse frame structure and receiver processing based on sparse dictionary learning, a massive number of IoT devices can transmit data without any prior scheduling process. Unlike existing schemes that necessitate significant overhead for preamble signals, the overhead required for the proposed scheme is negligible, which is attractive in terms of resource utilization and transmit power consumption. Numerical results verify that the proposed scheme can be utilized to facilitate massive connectivity in realistic cellular-based IoT scenarios.

In this paper, we propose a new Bayesian model to solve the Robust PCA problem - recovering the underlying low-rank matrix and sparse matrix from their noisy compositions. We first derive and analyze a new objective function, which is proven to be equivalent to the fundamental minimizing "rank+sparsity" objective. To solve this objective, we develop a concise Sparse Bayesian Learning (SBL) method that has minimum assumptions and effectively deals with the crux of the problem. The concise modeling allows simple and effective Empirical Bayesian inference via MAP-EM. Simulation studies demonstrate the superiority of the proposed method over the existing state-of-the-art methods. The efficacy of the method is further verified through a text extraction image processing task.

Utilizing the array processing minimum power distortionless response (MPDR) beamformer framework, we present a new perspective on the sparse Bayesian learning (SBL) algorithm used in sparse signal recovery. In addition to providing more insight into the SBL algorithm, this new perspective allows us to extend the algorithm to more general non-Gaussian priors. Finally, we use the connection between the MPDR and the LMMSE estimator to lower the complexity of the algorithm using a generalized approximate message passing (GAMP) based LMMSE estimator. The result is a GAMP based algorithm with improved convergence properties.

Acoustic feedback control continues to be a challenging problem due to the emerging form factors in advanced hearing aids (HAs) and hearables. In this paper, we present a novel use of well-known all-pass filters in a network to perform frequency warping that we call "freping." Freping helps in breaking the Nyquist stability criterion and improves adaptive feedback cancellation (AFC). Based on informal subjective assessments, distortions due to freping are fairly benign. While common objective metrics like the perceptual evaluation of speech quality (PESQ) and the hearing-aid speech quality index (HASQI) may not adequately capture distortions due to freping and acoustic feedback artifacts from a perceptual perspective, they are still instructive in assessing the proposed method. We demonstrate quality improvements with freping for a basic AFC (PESQ: 2.56 to 3.52 and HASQI: 0.65 to 0.78) at a gain setting of 20; and an advanced AFC (PESQ: 2.75 to 3.17 and HASQI: 0.66 to 0.73) for a gain of 30. From our investigations, freping provides larger improvement for basic AFC, but still improves overall system performance for many AFC approaches.

This paper investigates semi-blind channel estimation in massive multiple-input multiple-output (MIMO) systems using different priors on data symbols. We derive two tractable expectation-maximization (EM) based channel estimation algorithms; one based on a Gaussian prior and the other one based on a Gaussian mixture model (GMM) for the unknown data symbols. The numerical results show that the semiblind estimation schemes provide better channel estimates compared with the estimation based on training sequences only. The EM algorithm with a Gaussian prior provides superior channel estimates compared to the EM algorithm with a GMM prior in low signal-to-noise ratio (SNR) regime. However, the latter one outperforms the EM algorithm with Gaussian prior as the SNR or as the number antennas at the base station (BS) increases. Furthermore, the performance of the semi-blind estimators become closer to the genie-aided maximum likelihood estimator based on known data symbols as the number of antennas at the BS increases.

Relative Impulse Responses (ReIRs) have several applications in speech enhancement, noise suppression and source localization for multi-channel speech processing in reverberant environments. Noise is usually assumed to be white Gaussian during the estimation of the ReIR between two microphones. We show that the noise in this system identification problem is instead dependent upon the microphone measurements and the ReIR itself. We then present modifications that incorporate this new noise model into three prevalent methods: Least Squares, Non-Stationary Frequency Domain and Sparse Bayesian Learning based approaches. We demonstrated improvements with an experimental study using real-world measurements in various noise environments.

State-of-the-art noise power spectral density (PSD) estimation techniques for speech enhancement utilize the so-called speech presence probability (SPP). However, in highly non-stationary environments, SPP-based techniques could still suffer from inaccurate estimation, leading to significant amount of residual noise or speech distortion. In this paper, we propose to improve speech enhancement by deploying the bone-conduction (BC) sensor, which is known to be relatively insensitive to the environmental noise compared to the regular air-conduction (AC) microphone. A strategy is suggested to utilized the BC sensor characteristics for assisting the AC microphone in better SPP-based noise estimation. To our knowledge, no previous work has incorporated the BC sensor in this noise estimation aspect. Consequently, the proposed strategy can possibly be combined with other BC sensor assisted speech enhancement techniques. We show the feasibility and potential of the proposed method for improving the enhanced speech quality by both objective and subjective tests.

This paper addresses the problem of uplink (UL) and downlink (DL) channel estimation in frequency-division duplexing (FDD) massive multiple-input multiple-output (MIMO) systems. By utilizing the sparse recovery and compressive sensing algorithms, we are able to improve the accuracy of the UL/DL channel estimation and reduce the number of UL/DL pilot symbols. Such successful channel estimation builds upon the assumption that the channel can be sparsely represented under some basis/dictionary. Previous works model the channel using some predefined basis/dictionary; while in this paper, we present a dictionary learning-based channel model such that a dictionary is learned from comprehensively collected channel measurements. The learned dictionary adapts specifically to the cell characteristics and promotes a more efficient and robust channel representation, which in turn improves the performance of the channel estimation. Furthermore, we extend the dictionary learning-based channel model into a joint UL/DL learning framework by observing the reciprocity of the angle of arrival/angle of departure between the UL/DL transmissions and propose a joint channel estimation algorithm that combines the UL and DL received training signals to obtain a more accurate channel estimate. In other words, the DL training overhead, which is a bottleneck in FDD massive MIMO system, can be reduced by utilizing the information from simpler UL training.

Proactive caching will enable the 5G networks of the future to meet the challenge of continuously increasing wireless data traffic. Thanks to proactive caching, we are observing a shift from standard cellular networks, with one base station providing wireless connectivity to all the users in the cell, to heterogeneous networks, where several small base stations can assist the macro base station. Going one step further, users can also share their local content via device-to-device communications, avoiding multiple requests to the base station. In this complex network scenario, we propose a proactive caching policy to exploit all these communication opportunities and reduce congestion on the backhaul link, with the goal of minimizing the system cost, e.g., in terms of energy or bandwidth wastage. We provide a closed-form expression for the average system cost in the case of user mobility. In this scenario, we also consider heterogeneous file popularity profiles for different users. We present a robust optimization framework and show significant performance gains compared to a static caching policy, in which the caching decisions do not depend on the evolution of the network scenario.

In this paper, we present a novel multimodal sparse dictionary learning algorithm based on a hierarchical sparse Bayesian framework. The framework allows for enforcing joint sparsity across dictionaries without restricting the actual entries to be equal. We show that the proposed method is able to learn dictionaries of higher quality than existing approaches. We validate our claims with extensive experiments on synthetic data as well as real-world data.

In this paper we address the problem of learning shared sparse representation across several tasks. Assuming that the tasks share a common set of relevant features across all tasks is highly restrictive. This acts as a motivation to look for a generalized model which will be able to learn any correlation structure present between the tasks. We propose a generalized scale mixture distribution, the Multivariate Power Exponential Scale Mixture (M-PESM), as a joint sparsity promoting prior and derive a unified framework which consists of many of the popular Multitask Learning algorithms. Our proposed unified model also has the ability to learn any present correlation structure between tasks which leads to a more robust framework.

We study the problem of low-rank and sparse decomposition from possibly noisy observations. We propose a novel objective function with nuclear norm on the low-rank term and ℓ0-`norm' on the sparse term, as well as ℓ1-norm on the additive noise term. When there is no dense inlier noise, the proposed method shares the same theoretical guarantee as the Principal Component Pursuit (PCP), i.e., it can recover the low-rank component and sparse component exactly with high probability. Simulations in the noisy case demonstrate that the proposed method outperforms existing state-of-the-art methods. Results on a surveillance video application further verify the effectiveness of the proposed method.

In this letter, we consider the problem of joint blind source separation of multiple datasets simultaneously using an Independent vector analysis (IVA) framework. In particular we propose a new paradigm of reweighted algorithms for IVA by employing a source prior from a multivariate generalized scale mixture distribution family. In addition, our proposed reweighted algorithms can also exploit second-order statistical information across datasets by learning intrasource correlation matrix of each source component vector (SCV) along with higher order statistics. Experimental results are provided to show the efficacy of our proposed algorithms in achieving reliable source separation for both the cases, i.e., when there is correlation present within an SCV, and also when the sources are uncorrelated, i.e., no second order dependencies across datasets.

This paper studies the joint beamforming and power control in a multiuser multi-input single-output network by utilizing the only statistical channel distribution information. Such information consists of slowly varying covariance matrices in the beamforming network that can be employed to reduce instantaneous feedback overhead in transmission. Utilizing solely the statistical channel information, we study how to minimize the maximum outage probability under a weighted sum power constraint that guarantees max-min fairness to all users. This problem is, however, generally hard to solve due to the nonconvexity and nonlinear coupling between beamformer and power variables. First, assuming a fixed beamformer set, we use the nonlinear Perron-Frobenius theory to design a decentralized algorithm with provable geometrically fast convergence rate to compute the optimal power. Then, for the general case, we examine a certainty-equivalent margin counterpart with outage-mapped thresholds that incorporate the statistical channel information. We show that a network duality for this certainty-equivalent problem can be useful to decouple the coupling between the beamformer and power variables. This nonlinear Perron-Frobenius theory motivated approach yields a feasible beamformer and power allocation that are near-optimal as compared to Monte Carlo averaging simulations.

Cell-free Massive multiple-input multiple-output (MIMO) comprises a large number of distributed low-cost low-power single antenna access points (APs) connected to a network controller. The number of AP antennas is significantly larger than the number of users. The system is not partitioned into cells and each user is served by all APs simultaneously. The simplest linear precoding schemes are conjugate beamforming and zero-forcing. Max-min power control provides equal throughput to all users and is considered in this paper. Surprisingly, under max-min power control, most APs are found to transmit at less than full power. The zero-forcing precoder significantly outperforms conjugate beamforming. For zero-forcing, a near-optimal power control algorithm is developed that is considerably simpler than exact max-min power control. An alternative to cell-free systems is small-cell operation in which each user is served by only one AP for which power optimization algorithms are also developed. Cell-free Massive MIMO is shown to provide five- to ten-fold improvement in 95%-likely per-user throughput over small-cell operation.

In this paper we present a novel Hierarchical Bayesian approach to estimate Relative Impulse Response (ReIR) using short, noisy and reverberant microphone recordings. The information contained in ReIRs between two microphones is useful for a wide range of multichannel speech processing applications such as speaker localization, speech enhancement, etc. It has been shown in several previous works that the Relative Transfer Function (RTF) corresponding to a given ReIR is dynamic and depends on the environment, microphone positions and target position. This acts as the main motivation of this work, as we develop a structured sparse Bayesian learning algorithm to estimate ReIR using very short recordings, which will be robust to changes in the environment. An extensive experimental study with real-world recordings has also been conducted to show the efficacy of our proposed approach over other competing approaches.

Methods for generating multizone soundfield have a wide range of applications. The desired dominant frequency bands to be generated and perceived, however, are not the same for different applications. Previous work proposed a low complexity algorithm for multizone wideband sound field generation using a frequency variable dictionary in a Lasso-LS optimization. This work demonstrates the flexibility of the novel algorithm in imposing variable number of active speakers to provide the desired reproduction accuracy at variable frequency bands. The deployment of this technique can lead to the customization of multizone sound system design based on both signal frequency content and listeners' perception.

The increase in wireless data traffic is encouraging a shift from standard cellular networks, with one base station providing wireless connectivity to all the users in the cell, to heterogeneous networks, where several small base stations can assist the macro base station in providing service. Furthermore, with device-to-device communications the users can also share local content without multiple requests to the base station. In this complex network scenario, we propose a proactive caching policy to exploit all these communication opportunities and reduce congestion at the backhaul link. The goal is to minimize the system cost, in terms of energy or bandwidth wastage. We provide a closed form expression for the average system cost in the case in which we consider user mobility and different classes of user's interests. We present a robust optimization framework, and we show significant performance gains compared to a static caching policy.

In this paper, we present a novel Bayesian approach to recover simultaneously block sparse signals in the presence of outliers. The key advantage of our proposed method is the ability to handle non-stationary outliers, i.e. outliers which have time varying support. We validate our approach with empirical results showing the superiority of the proposed method over competing approaches in synthetic data experiments as well as the multiple measurement face recognition problem.

This paper presents an approach for multizone wideband sound field generation using an efficient harmonic nested (EHN) dictionary for sparse loudspeakers' placement and weight. Effectively, the nested arrays provide a priori knowledge of prospective loudspeaker locations based on the frequency bands of interest. The nested arrays are then further optimized in the Lasso stage to form an efficient loudspeakers' location dictionary. The final loudspeaker locations and weightings are estimated by a two-stage Lasso-LS pressure matching optimization. In the first stage Lasso algorithm, the center band frequencies of octave bands from 1 kHz to 8 kHz were used to select active loudspeakers. A second stage then optimizes reproduction using all selected loudspeakers on the basis of a regularized LS algorithm. The results demonstrate that the proposed approach provides a solution for the multizone sound system with the mean squared error (MSE) under -30 dB across the targeted frequency range (500 Hz- 16 kHz) using a linear array e.g. 13 loudspeakers. While, the single-stage LS approach generates the MSE peaks of -10 dB and -9 dB at 9 kHz within the active and silent zones respectively using an identical number of loudspeakers and array length.

We address the problem of downlink channel estimation in frequency division duplex (FDD) Massive MIMO system when downlink training duration is limited. Leveraging the concept of Compressive Sensing (CS), downlink channel could be estimated with limited training duration if channel can be sparsely represented. In this paper, we develop a better dictionary under which the downlink channel can be more sparsely represented, thus improving the performance of recovery in the compressive sensing process. We develop a method for learning such a dictionary from channel measurements, which capture information about communication environment as well as the property of antennas. We develop methods to learn the dictionary that best models the data, thus adapting to the environment and antenna property, while simultaneously inducing sparsity. Also we compare different dictionaries and provide insights into reasons why a learned dictionary outperforms others.

This work addresses the problem of sequential recovery of temporally correlated sparse vectors with common support from noisy under-determined linear measurements. The Kalman sparse Bayesian learning (SBL) algorithm is an efficient tool for solving the problem when the temporal correlation is modeled using a first order autoregressive model. However, this method processes the input data in a batch mode, which results in high latency. We propose two online SBL algorithms which operate on the observations in a serial fashion. They are sequential expectation maximization (EM) schemes, implemented using fixed lag smoothing and sawtooth lag smoothing. The online algorithms require significantly lower computational and memory resources compared to their offline counterparts. Also, estimates of the sparse vectors become available after a fixed delay from the time observations arrive. Using Monte Carlo simulations, we illustrate that the mean square error and support recovery performance of the proposed algorithms is very close to the offline Kalman SBL algorithm.

We propose two novel methods of service delay control for multiuser wireless systems that employ a scheduling scheme based on the Cumulative Density Function (CDF) of user channels in correlated fading environments. The first method allows for different hard delay deadlines for the users. The second method relaxes the deadline enforcement into “soft” deadlines in order to take better advantage of multiuser diversity. Both methods offer excellent performance while still preserving the temporal fair resource allocation property of the CDF scheduling policy.

This paper considers random multiple access in a network where only a small portion of users have data to forward and transmit packets in each time slot because the user activity ratio is not high in practice. For this reason, the access point (AP) has to not only identify the users who transmitted but also decode the received data codewords. Exploiting the sparsity of transmitting users, Lasso, which is a well-known practical compressed sensing algorithm, is applied for efficient user identification. The compressed sensing algorithm enables the AP to handle more users than the conventional random multiple access schemes do. We develop distributed scheduling methods for maximizing the system sum throughput, and we analyze the corresponding optimal throughput for three different cases of channel knowledge, i.e., the channel state information at the transmitter (CSIT), the channel state information at the receiver (CSIR), and the imperfect channel state information at the receiver (ImCSIR). We also derive the closed-form expressions of asymptotically optimal scheduling parameters and the corresponding maximum sum throughput for each CSI assumption. The results show the effects of system parameters on the sum throughput and provide useful insights on using compressed sensing for throughput maximization in random multiple access schemes.

In this work, simple and practical CDF scheduling methods are developed that preserve the virtues of CDF scheduling, namely fairness and effective use of multiuser diversity. They are the accelerated extended CDF scheduling (AeCS), modified non-parametric CDF scheduling (modified NPCS), and the CDF scheduling with optimized Quantizers (CSwQ) methods. The developed methods do not need a priori knowledge of the channel distribution and can work effectively in systems that support discrete number of transmission rates and are constrained by limited feedback resources. They exploit the limited feedback resource for their scheduling decisions and transmission rate selections and learn the distribution of channel quality information of each user, as needed, to best exploit multiuser diversity. The CSwQ method is shown to have desirable properties in terms of both system throughput and power savings. Analytical results for the system throughput and power savings of the CSwQ method are developed that are valid for a finite number of observation samples. Then, through numerical simulations, the developed methods are shown to effectively save transmit power and achieve high system throughput. In addition, they converge rapidly with the number of data samples to the ideal case wherein perfect knowledge of the channel CDF is assumed.

We consider the resource allocation problem in cellular networks which support device-to-device communications (D2D). For systems that enable D2D via only orthogonal resource sharing, we propose and analyze two resource allocation policies that guarantee access fairness among all users, while taking advantage of multi-user diversity and local D2D communications, to provide marked improvements over existing cellular-only policies. The first policy, the Cellular Fairness Scheduling (CFS) Policy, provides the simplest D2D extension to existing cellular systems, while the second policy, the D2D Fairness Scheduling (DFS) Policy, harnesses maximal performance from D2D-enabled systems under the orthogonal sharing setting. For even higher spectral efficiency, cellular systems with D2D can schedule the same frequency resource for more than one D2D pairs. Under this non-orthogonal sharing environment, we propose a novel group scheduling policy, the Group Fairness Scheduling (GFS) Policy, that exploits both spatial frequency reuse and multiuser diversity to deliver dramatic improvements to system performance with perfect fairness among the users, regardless of whether they are cellular or D2D users.

In this paper, it is shown that an appropriate model for voiced speech is an all-pole filter excited by a block sparse excitation sequence. The modeling approach is generalized in a novel manner to deal with a wide spectrum of speech signal; voiced speech, unvoiced speech and mixed excitation speech. In this context, the input sequence to the all-pole model is modeled as a suitable weighted linear combination of a block sparse signal and white noise. We develop the corresponding estimation procedure to reconstruct the generalized input sequence and model parameters via sparse Bayesian learning methods employing the Expectation-Maximization based procedure. Rigorous experiments have been performed to show the efficacy of our proposed model for the speech modeling task. By imposing a block sparse structure on the input sequence, the problems associated with the commonly used Linear Prediction approach is alleviated leading to a more robust modeling scheme.

This paper investigates the filter bank (FB) based selection diversity combining as well as linear equalization for single carrier (SC)transmissions over frequency selective channels. In contrast to the multicarrier carrier (MC) transmissions, e.g., OFDM, the FB based approach avoids the use of cyclic prefix (CP) or guard band and offers a number of superior properties such as synchronization, and peak to average power ratio (PAPR), etc. However, due to the lack of practical diversity techniques and cost effective equalizers, the broadband SC signal can hardly be deployed under highly dispersive channel environment. We propose a practical FB based selection diversity based on non-maximally decimated filter banks with perfect reconstruction support (PR-NMDFB) with its companion linear equalizer in the FB transformed domain. We shall give detailed minimum mean square error (MMSE) and bit error rate (BER) analysis and compare it to the optimal maximum ratio combining (MRC) solution.

We consider the applications of blind and semi-blind interference alignment in multicell scenarios, specifically in clustered small cells. As a first step, two simple straight forward extensions of blind interference alignment are examined and it is observed that neither of them is uniformly superior. Then, we propose exploiting the location information of the users and base stations in the cluster to enhance the performance of fully blind schemes for any given user distribution scenario. Our aim is to group suitable users that can be served at the same time to minimize the supersymbol length for each cluster. Since the defined problem is NP-hard, we propose a heuristic algorithm that can provide an effective solution without too much complexity. By numerical simulations, we show that the proposed semi blind algorithm, Top.BIA, uniformly performs better than pure blind interference alignment schemes for any possible user distribution scenario.

In modern heterogeneous wireless networks, the task of supporting fairness along with user priorities and concurrently achieving the highest possible system throughput is desirable and challenging. Herein, a class of practical cumulative distribution function (CDF) scheduling algorithms are developed to achieve these goals. These algorithms are used when the channel fading model is unknown. The mapping from channel quality information (CQI) to the real CDF is unknown but is constructed exploiting the order statistics of the CQI sequence. The constructed CDF mapping methods are shown to converge to the actual CDF. Specifically, one algorithm uses the expected value of the ordered CDF scheduling while others called Non-parametric CDF scheduling (NPCS) algorithms reconstructs the CDF with an extra interpolation step. By collecting a moderate number of CQI data, the algorithms almost achieve the system throughput of CDF scheduling as if the CDF is known. Throughout the work, CDF scheduling algorithms, supported by simulations, are shown to be able to effectively support fairness and frequently outperform, and are potential alternatives to, the well known Proportional Fair (PF) scheduling method.

We consider the problem of controlling the user service delays for a system that employs a scheduling scheme based on Cumulative Density Functions (CDF) of user channels in a correlated Rayleigh fading environment. We first formulate and solve this control problem as a Markov Decision Process (MDP) on the system state space formed by the user channel conditions and delay times. The MDP formulation, however, has prohibitively high complexity for systems with typical number of users. We then propose an approximation to the MDP formulation that can achieve performance close to MDP optimal solution but has much lower complexity for large systems.

In this work, we address the recovery of block sparse vectors with intra-block correlation, i.e., the recovery of vectors in which the correlated nonzero entries are constrained to lie in a few clusters, from noisy underdetermined linear measurements. Among Bayesian sparse recovery techniques, the cluster Sparse Bayesian Learning (SBL) is an efficient tool for block-sparse vector recovery, with intrablock correlation. However, this technique uses a heuristic method to estimate the intra-block correlation. In this paper, we propose the Nested SBL (NSBL) algorithm, which we derive using a novel Bayesian formulation that facilitates the use of the monotonically convergent nested Expectation Maximization (EM) and a Kalman filtering based learning framework. Unlike the cluster-SBL algorithm, this formulation leads to closed-form EM updates for estimating the correlation coefficient. We demonstrate the efficacy of the proposed NSBL algorithm using Monte Carlo simulations.

Predicting cognitive performance of subjects from their magnetic resonance imaging (MRI) measures and identifying relevant imaging biomarkers are important research topics in the study of Alzheimer's disease. Traditionally, this task is performed by formulating a linear regression problem. Recently, it is found that using a linear sparse regression model can achieve better prediction accuracy. However, most existing studies only focus on the exploitation of sparsity of regression coefficients, ignoring useful structure information in regression coefficients. Also, these linear sparse models may not capture more complicated and possibly nonlinear relationships between cognitive performance and MRI measures. Motivated by these observations, in this work we build a sparse multivariate regression model for this task and propose an empirical sparse Bayesian learning algorithm. Different from existing sparse algorithms, the proposed algorithm models the response as a nonlinear function of the predictors by extending the predictor matrix with block structures. Further, it exploits not only inter-vector correlation among regression coefficient vectors, but also intra-block correlation in each regression coefficient vector. Experiments on the Alzheimer's Disease Neuroimaging Initiative database showed that the proposed algorithm not only achieved better prediction performance than state-of-the-art competitive methods, but also effectively identified biologically meaningful patterns.

This paper studies the joint beamforming and power control in a multiuser distributed antenna uplink network, wherein the number of users and the number of separately located antennas grow large with ratio being bounded. We consider the SINR fairness problem under individual power constraint and present a distributed iterative algorithm. This algorithm, though converging to the instantaneous optimal solution, requires instantaneous power update. In order to design a low complexity algorithm that achieves optimality in the asymptotic sense, we leverage the large system structure to derive an asymptotic solution requiring only channel statistics. In this algorithm, the asymptotic power is slowly updated and the asymptotic beamformer can be obtained in a non-iterative manner. The convergence property of the proposed algorithm is also studied.

Opportunistic spectrum access in a cognitive radio network has been a challenge due to the dynamic nature of spectrum availability and possible collisions between the primary user (PU) and the secondary user (SU). To maximize the spectrum utilization, we propose a spectrum access strategy where SU's packets are interleaved with periodic sensing to detect PU's return. We formulate the sensing/probing/access process as a maximum rate-of-return problem in the optimal stopping theory framework and show that the optimal channel access strategy is a pure threshold policy. We consider a realistic channel and system model by taking into account channel fading and sensing errors. We jointly optimize the rate threshold and the packet transmission time to maximize the average throughput of SU, while limiting interference to PU.

In a multicell multiuser MIMO downlink employing random beamforming as the transmission scheme, the heterogeneous large scale channel effects of intercell and intracell interference complicate analysis of distributed scheduling based systems. In this paper, we extend the analysis in [1] and [2] to study the aforementioned challenging scenario. The cumulative distribution function (CDF)-based scheduling policy utilized in [1] and [2] is leveraged to maintain fairness among users and simultaneously obtain multiuser diversity gain. The closed form expression of the individual sum rate for each user is derived under the CDF-based scheduling policy. More importantly, with this distributed scheduling policy, we conduct asymptotic (in users) analysis to determine the limiting distribution of the signal-to-interference-plus-noise ratio, and establish the individual scaling laws for each user.

In this paper, we consider the power allocation problem that minimizes the outage probability for Rayleigh slow fading channels with equiprobable QAM inputs. We focus on the uniform power allocation with thresholding (UPAT) policy that assigns nonzero constant power only to a subset of the subchannels. This simple suboptimal policy can significantly alleviate the feedback overhead and the complexity compared to the optimal mercury/water-filling (MWF) solution. Through asymptotic analysis and numerical simulations, we first show that the optimal UPAT, namely, the UPAT with the optimal threshold, performs close to MWF if the constellation size M is large enough that log2M #8811; R, where R is the fixed target transmission rate. This condition log2M #8811; R turns out to define a natural system operating point. As we show through numerical results, if log2M #8776; R, both MWF and the optimal UPAT perform poorly due to having too small M and their performance can be significantly improved by using a larger M. From these results, we conclude that for a given target transmission rate, the optimal UPAT performs close to MWF as long as the constellation size is chosen appropriately not to limit the performance.

In this paper, we show that optimizing the feedback strategy in a limited feedback multiuser system can significantly improve system performance. Herein, we consider a limited feedback multiuser system, in which the fairness among users are guaranteed by the application of the cumulative distribution function (CDF) based scheduling technique. As the users are diverse in types, priorities and channel statistics, adjusting feedback strategy as well as dynamically distributing feedback resources among the users helps the system to exploit multiuser diversity gain. The system performance optimization is formulated in which we consider feedback as a resource to optimize. Then, the optimization is decoupled into two sub-problems, namely optimization of quantizers and allocation of feedback bits for the users. Optimizing feedback usage is shown to be significantly beneficial, both analytically and with the help of simulation results. Finally, in order to address the practical aspect, we propose an algorithm to efficiently solve these two sub-problems in an optimal manner followed by a low complexity sub-optimal approach which offers good tradeoff between complexity and performance.

We consider a multiuser downlink system where the base station (BS) only has imperfect channel quality information (CQI) due to both feedback delay and estimation error at the receiver. Using coherent detection, the signal portion due to delay is accommodated while the estimation error is random and can be considered as noise. A thresholding feedback scheme is used to reduce the overall amount of feedback which is typically set by the system. Then, a fixed rate and a variable rate transmission scheme are analyzed which are then optimized to obtain the highest average system goodput. By using variable rate with optimized parameters, system goodput increases significantly in comparison with the fixed rate strategy. The feedback delay is shown to affect system performance with the severity increasing over time. Finally, the match between analysis and simulation are verified by the experiments.

In this paper we present a solution to the problem of tracking and separation of a mixture of concurrent sources in a reverberant environment where the number of sources is unknown and varies with time: new sources can appear and existing sources can disappear or undergo silence periods. In order to deal with this challenging problem, we synergistically combine two key ideas, one in the front end and the other at the back end. In the front end we employ independent component analysis (ICA) to demix the mixtures and the state coherence transform (SCT) to represent the signals in a direction of arrival (DOA) detection framework. By exploiting the frequency sparsity of the sources, ICA/SCT is even effective when the number of simultaneous sources is greater than the number of sensors therefore allowing for minimal number of sensors to be used. At the back end, the probability hypothesis density (PHD) filter is incorporated in order to track the multiple DOAs and determine the number of sources. The PHD filter is based on random finite sets (RFS) where the multi-target states and the number of targets are integrated to form a set-valued variable with uncertainty in the number of sources. A Gaussian mixture implementation of the PHD filter (GM-PHD) is utilized that solves the data association problem intrinsically, hence providing distinct DOA tracks. The distinct tracks also make the separation task possible by going back and rearranging the outputs of the ICA stage. The tracking and separation capabilities of the proposed method is demonstrated using simulations of multiple sources in reverberant environments.

This paper studies the performance limits in the support recovery of sparse signals based on multiple measurement vectors (MMV). An information-theoretic analytical framework inspired by the connection to the single-input multiple-output multiple-access channel communication is established to reveal the performance limits in the support recovery of sparse signals with fixed number of nonzero entries. Sharp sufficient and necessary conditions for asymptotically successful support recovery are derived in terms of the number of measurements per vector, the number of nonzero rows, the measurement noise level, and the number of measurement vectors. Through the interpretations of the results, the benefit of having MMV for sparse signal recovery is illustrated, thus providing a theoretical foundation to the performance improvement enabled by MMV as observed in many existing simulation results. In particular, it is shown that the structure (rank) of the matrix formed by the nonzero entries plays an important role in the performance limits of support recovery.

In multiuser multiple-input-multiple-output (MIMO) networks, receivers decode multiple concurrent signals using successive interference cancellation (SIC). With SIC, a weak target signal can be deciphered in the presence of stronger interfering signals. However, this is only feasible if each strong interfering signal satisfies a signal-to-noise-plus-interference ratio (SINR) requirement. This necessitates the appropriate selection of a subset of links that can be concurrently active in each receiver's neighborhood; in other words, a subtopology consisting of links that can be simultaneously active in the network is to be formed. If the selected subtopologies are of small size, the delay between the transmission opportunities on a link increases. Thus, care should be taken to form a limited number of subtopologies. We find that the problem of constructing the minimum number of subtopologies such that SIC decoding is successful with a desired probability threshold is NP-hard. Given this, we propose MUSIC, a framework that greedily forms and activates subtopologies in a way that favors successful SIC decoding with a high probability. MUSIC also ensures that the number of selected subtopologies is kept small. We provide both a centralized and a distributed version of our framework. We prove that our centralized version approximates the optimal solution for the considered problem. We also perform extensive simulations to demonstrate that: 1) MUSIC forms a small number of subtopologies that enable efficient SIC operations; the number of subtopologies formed is at most 17% larger than the optimum number of topologies, discovered through exhaustive search (in small networks); 2) MUSIC outperforms approaches that simply consider the number of antennas as a measure for determining the links that can be simultaneously active. Specifically, MUSIC provides throughput improvements of up to four times, as compared to such an approach, in various topological settings. The improvements can be directly attributable to a significantly higher probability of correct SIC based decoding with MUSIC.

This paper investigates three open problems in random beamforming based communication systems: the scheduling policy with heterogeneous users, the closed form sum rate, and the randomness of multiuser diversity with selective feedback. By employing the cumulative distribution function based scheduling policy, we guarantee fairness among users as well as obtain multiuser diversity gain in the heterogeneous scenario. Under this scheduling framework, the individual sum rate, namely the average rate for a given user multiplied by the number of users, is of interest and analyzed under different feedback schemes. Firstly, under the full feedback scheme, we derive the closed form individual sum rate by employing a decomposition of the probability density function of the selected user's signal-to-interference-plus-noise ratio. This technique is employed to further obtain a closed form rate approximation with selective feedback in the spatial dimension. The analysis is also extended to random beamforming in a wideband OFDMA system with additional selective feedback in the spectral dimension wherein only the best beams for the best-L resource blocks are fed back. We utilize extreme value theory to examine the randomness of multiuser diversity incurred by selective feedback. Finally, by leveraging the tail equivalence method, the multiplicative effect of selective feedback and random observations is observed to establish the individual rate scaling.

This paper studies joint beamforming and power control in a coordinated multicell downlink system that serves multiple users per cell to maximize the minimum weighted signal-to-interference-plus-noise ratio. The optimal solution and distributed algorithm with geometrically fast convergence rate are derived by employing the nonlinear Perron-Frobenius theory and the multicell network duality. The iterative algorithm, though operating in a distributed manner, still requires instantaneous power update within the coordinated cluster through the backhaul. The backhaul information exchange and message passing may become prohibitive with increasing number of transmit antennas and increasing number of users. In order to derive asymptotically optimal solution, random matrix theory is leveraged to design a distributed algorithm that only requires statistical information. The advantage of our approach is that there is no instantaneous power update through backhaul. Moreover, by using nonlinear Perron-Frobenius theory and random matrix theory, an effective primal network and an effective dual network are proposed to characterize and interpret the asymptotic solution.

This paper investigates how multiuser dimensions can effectively be exploited for target degrees of freedom (DoF) in interfering broadcast channels (IBC) consisting of K-transmitters and their user groups. First, each transmitter is assumed to have a single antenna and serve a single user in its user group where each user has receive antennas less than K. In this case, a K-transmitter single-input multiple-output (SIMO) interference channel (IC) is constituted after user selection. Without help of multiuser diversity, K-1 interfering signals cannot be perfectly removed at each user since the number of receive antennas is smaller than or equal to the number of interferers. Only with proper user selection, non-zero DoF per transmitter is achievable as the number of users increases. Through geometric interpretation of interfering channels, we show that the multiuser dimensions have to be used first for reducing the DoF loss caused by the interfering signals, and then have to be used for increasing the DoF gain from its own signal. The sufficient number of users for the target DoF is derived. We also discuss how the optimal strategy of exploiting multiuser diversity can be realized by practical user selection schemes. Finally, the single transmit antenna case is extended to the multiple-input multiple-output (MIMO) IBC where each transmitter with multiple antennas serves multiple users.

Opportunistic spectrum access in a cognitive radio network has been a challenge due to the dynamic nature of spectrum availability and possible collisions between the primary user (PU) and the secondary user (SU). To maximize the spectrum utilization, we propose a spectrum access strategy where SU's packets are interleaved with periodic sensing to detect PU's return. Similar to earlier works on distributed opportunistic scheduling (DOS), we formulate the sensing/probing/access process as a maximum rate-of-return problem in the optimal stopping theory framework and show that the optimal channel access strategy is a pure threshold policy. We consider a realistic channel and system model by taking into account channel fading and sensing errors. We jointly optimize the rate threshold and the packet transmission time to maximize the average throughput of SU, while limiting interference to PU. Our numerical results show that significant throughput gains can be achieved with the proposed scheme compared to other well-known schemes. Our work sheds light on designing DOS protocols for cognitive radio with optimal transmission time that takes into account the dynamic nature of PUs.

We develop channel-aware opportunistic spectrum access strategies for cognitive radio networks consisting of multiple secondary users. In addition to balancing between the cost of foregoing weaker transmission opportunities in pursuit of channels that offer better data rates, we consider collisions between primary users (PUs) and secondary users (SUs), and among the SUs themselves. We derive strategies for both cooperative setting where SUs maximize their sum total of throughputs, as well as non-cooperative, game theoretic setting where each SU tries to maximize its own throughput. We show that the optimal schemes for both scenarios are pure threshold policies, where each SU decides to use or skip transmission opportunities by comparing the channel qualities to a fixed threshold. In the non-cooperative case, we establish the existence of Nash equilibrium and develop best response strategies that can converge to equilibria, with SUs relying only on their local observations. We study the tradeoff between maximal throughput in cooperative setting and fairness in the non-cooperative setting, and schemes based on utility functions and pricing that mitigate this tradeoff. Lastly, we analyze the issue of interference of SUs to the PUs. We show numerical results illustrating the schemes and analysis.

Alzheimer’s disease (AD) is the most common form of de-mentia that causes progressive impairment of memory and other cognitive functions. Multivariate regression models have been studied in AD for revealing relationships between neuroimaging measures and cognitive scores to understand how structural changes in brain can influence cognitive sta-tus. Existing regression methods, however, do not explic-itly model dependence relation among multiple scores de-rived from a single cognitive test. It has been found that such dependence can deteriorate the performance of these methods. To overcome this limitation, we propose an effi-cient sparse Bayesian multi-task learning algorithm, which adaptively learns and exploits the dependence to achieve improved prediction performance. The proposed algorithm is applied to a real world neuroimaging study in AD to pre-dict cognitive performance using MRI scans. The effective-ness of the proposed algorithm is demonstrated by its supe-rior prediction performance over multiple state-of-the-art competing methods and accurate identification of compact sets of cognition-relevant imaging biomarkers that are con-sistent with prior knowledge. 1.

This paper studies joint beamforming and power control in a multiuser MISO interference network with statistical channel information. Such information consists of the slow-varying covariance matrices in the beamforming network, and can be employed to reduce instantaneous feedback needs. With the outage event induced by the utilization of statistical channel information, we optimize signal transmission strategies to minimize the maximum outage probability under weighted sum power constraint to achieve outage balancing in the interference network. Under the condition of fixed beamformer, we use nonlinear Perron-Frobenius theory to present a decentralized algorithm with provable geometrically fast convergence rate to compute the optimal power. Since the joint beamformer and power optimization problem is non-convex, we examine its certainty-equivalent margin counterpart. By leveraging nonlinear Perron-Frobenius theory and the established network duality, we present a near-optimal decentralized algorithm to jointly optimize the beamformer and power. The algorithm converges quickly and the convergence rate of the algorithm is proven to be geometrical.

This paper studies efficient signal transmission strategies in a multiuser MIMO network where multiple source nodes form a large virtual MIMO array to perform cooperative beamforming. In order to reduce instantaneous feedback needs, we investigate joint power control and cooperative beamforming techniques to maximize the minimum weighted average SINR based on statistical channel information, which consists of the covariance matrices in the beamforming network. Since the joint optimization problem over power, transmit beamformer and receive beamformer is non-convex, we analyze two sub-problems: the joint optimization of power and transmit beamformer under fixed receive beamformer, and the joint optimization of power and receive beamformer under fixed transmit beamformer. For both sub-problems, we use nonlinear Perron-Frobenius theory to characterize the optimal solution and present decentralized algorithms with provable geometrically fast convergence rate. By combining the developed techniques and the network duality, we provide an effective iterative algorithm for the joint optimization problem.

In this paper, we propose a robust data-aided carrier frequency offset (CFO) tracking algorithm for orthogonal frequency division multiplexing (OFDM) system. The problem is solved as a sequence of estimation and correction steps. We find the major contributors to the CFO measurement uncertainty are the additive white noise together with the inter-carrier interference (ICI). Our derivation results show the noise variance can be reduced via averaging while the ICI introduced uncertainty can be decreased by iterative CFO compensations. These considerations lead us to a Type-2 correction loop. Theoretical analysis and simulation results show our proposed algorithm is robust and able to compensate and track comparably large CFO.

In this work, we develop limited feedback techniques that utilize both Channel State Information (CSI) and Channel Distribution Information (CDI) for communication in multiuser MIMO beamforming networks with SINR constraints on the links. We minimize power in the network using a CDI-based algorithm, and then use limited CSI feedback to improve on this scheme by reducing power consumption further and transmit opportunistically. We study three cases of the CSI feedback channel-with one bit of bandwidth, with infinite bits, and with B bits. We develop feedback techniques for the one-bit and infinite-bit cases, and then derive the optimal quantizer for the B-bit case. Our results show that significant power reduction can be achieved using a small number of bits.

Spatial and multiuser diversity are two types of diversity techniques for delivering reliable high-date-rate services. Spectral diversity comes from opportunistic scheduling in the frequency domain enabled by the OFDMA technique, and is influenced by partial feedback design. By employing the best-M partial feedback strategy, we provide a unified view of spatial, spectral, and multiuser diversity through asymptotic (in users) analysis. We examine the tail behavior of the distribution of the received channel quality information (CQI) at the scheduler to prove the type of convergence as well as to derive the asymptotic approximations for the average spectral efficiency under partial feedback. We investigate the application of our analysis to different spatial diversity schemes. Our derived results can be used to quickly determine the minimum required partial feedback in a general multiuser MIMO-OFDMA system.

We study the problem of support recovery of block-sparse signals, where nonzero entries occur in clusters, via random noisy measurements. By drawing analogy between the problem of block-sparse signal recovery and the problem of communication over Gaussian multi-input and single-output multiple access channel, we derive the sufficient and necessary condition under which exact support recovery is possible. Based on the results, we show that block-sparse signals can reduce the number of measurements required for exact support recovery, by at least `1/(block size)', compared to conventional or scalar-sparse signals. The minimum gain is guaranteed by increased signal to noise power ratio (SNR) and reduced effective number of entries (i.e., not individual elements but blocks) that are dominant at low SNR and at high SNR, respectively. When the correlation between the elements of each nonzero block is low, a larger gain than `1/(block size)' is expected due to, so called, diversity effect, especially in the moderate and low SNR regime.

In this paper we study the recovery of block sparse signals and extend conventional approaches in two important directions; one is learning and exploiting intra-block correlation, and the other is generalizing signals' block structure such that the block partition is not needed to be known for recovery. We propose two algorithms based on the framework of block sparse Bayesian learning (bSBL). One algorithm, directly derived from the framework, requires a priori knowledge of the block partition. Another algorithm, derived from an expanded bSBL framework using the generalization method, can be used when the block partition is unknown. Experiments show that they have superior performance to state-of-the-art algorithms.

Modern heterogeneous networks have an inherent heterogeneous structure such as user densities and large scale channel effects that motivates this work on heterogeneous partial feedback design. In this paper, we address the partial feedback design issue in an OFDMA-based macro-pico cellular system employing the best-M partial feedback strategy. We consider the scenario with one picocell inside a macrocell, and investigate a scheduling policy which tracks the small scale channel effects in order to guarantee fairness among users as well as exploiting the multiuser diversity. We derive a closed form expression for the average spectral efficiency of the network with best-M partial feedback for the identified representative scenarios. We carry out asymptotic analysis using extreme value theory to derive an approximation to the average spectral efficiency in order to determine the minimum required partial feedback with minimal loss in system performance.

We present a unified analytical framework to jointly examine spatial, spectral, and multiuser diversity in a general multiuser MIMO OFDMA system. Spectral diversity originates from frequency domain opportunistic scheduling enabled by the OFDMA technique, and is influenced by partial feedback design. We propose a heterogeneous partial feedback design method which adapts users' feedback resources to their frequency domain channel statistics. We investigate this adaptive feedback design by employing the best-M partial feedback strategy, and derive closed form expressions for the average spectral efficiency and bit error rate with different spatial diversity schemes under the generalized Nakagami-m fading channels. We also examine the interplay among the diversity effects using our heterogeneous feedback design as well as the channel fading effect through numerical results.

In this letter, we investigate the sum rate performance of a generic one-pico-inside OFDMA network with opportunistic scheduling policy and the best-M partial feedback design. By a detailed examination of the statistical property of the selected user's signal-to-interference-plus-noise ratio and its interplay with intercell interference, we derive the exact closed form expression for the sum rate under partial feedback. The general expression incorporates the special interference limited and noise limited scenarios as special cases. The result explains the exact interaction with partial feedback and the approach can be generalized and applied to many related problems.

A distributed adaptive opportunistic routing scheme for multihop wireless ad hoc networks is proposed. The proposed scheme utilizes a reinforcement learning framework to opportunistically route the packets even in the absence of reliable knowledge about channel statistics and network model. This scheme is shown to be optimal with respect to an expected average per-packet reward criterion. The proposed routing scheme jointly addresses the issues of learning and routing in an opportunistic context, where the network structure is characterized by the transmission success probabilities. In particular, this learning framework leads to a stochastic routing scheme that optimally “explores” and “exploits” the opportunities in the network.

Feedback of channel state information (CSI) in wireless systems is essential in order to exploit multi-user diversity and achieve the highest possible performance. When each spatially distributed user in the wireless system is assumed to have i.i.d. scalar CSI values, the optimal fixed-rate and entropy-constrained point density functions are established in the high-resolution regime for the quantization of the CSI feedback to a centralized scheduler under the mean square error (MSE) criterion. The spatially distributed nature of the users leads to a distributed functional scalar quantization approach for the optimal high resolution point densities of the CSI feedback. Under a mild absolute moment criterion, it is shown that with a greedy scheduling algorithm at the centralized scheduler, the optimal fixed-rate point density for each user corresponds to a point density associated with the maximal order statistic distribution. This result is generalized to monotonic functions of arbitrary order statistics. Optimal point densities under entropy-constrained quantization for the CSI are established under mild conditions on the distribution function of the CSI metric.

This paper presents an analysis of the post fabrication performance for a Bragg-mirrors-based photonic allpass filter. The development subsequently leads to a first definition of the stochastic measures in a realistic system implementation. A Digital Signal Processing (DSP) approach to examining photonic integrated circuitry is explored in depth. The physical origins of error that manifests in an unbalanced phase error is mathematically described in detail. We describe the details of error analysis and demonstrate a stochastic measure for the expected behavior of first-order allpass filter as well as high complexity systems. Our findings show that the frequency performance under error can be characterized by a convolution of the ideal response with a probability function based kernel.

In this paper, we explore the problem of separating and localizing multiple maneuvering speakers in a reverberant environment. Moreover, the speakers can become silent intermittently, making the problem more challenging especially when the speaker is moving while silent. A two stage method is proposed. In the first stage a multiple model particle filter (MMPF) is used to track the mixing matrices by “glimpsing” or listening in the silence gaps as the set of active speakers change with time. In the second stage, a secondary microphone array is utilized to localize and track the speakers by intersecting their directions of arrival (DOA) obtained from estimates of the tracked mixing matrices. Computer simulations using the proposed method are conducted showing improved performance when compared with an online independent component analysis (ICA)-based algorithm.

The performance of a multiuser MIMO broadcast system depends highly on how the users being served are selected from the pool of users requesting service. Though dirty paper coding can obtain the optimal capacity of multiuser MIMO Broadcast systems, the computational complexity is high. We examine this problem in the context of systems using Zero Forcing Beamforming (ZFBF) which have low complexity and reasonable throughput. The channels of the selected users may not be orthogonal to each other resulting in harmful interference and reduction in the performance of the system. This work identifies and thoroughly investigates the main factors affecting the performance and proposes some mitigating schemes to fully exploit multiuser diversity. In particular, we develop improved user selection schemes designed to maximize the sum rate that offer a range of complexity to performance tradeoff. Simulation results show that the selection schemes developed compare favorably with existing techniques.

This paper presents an approach to designing narrowband digital filters that are realizable using optical allpass building blocks. We describe a top-down design method by explicitly examining the derivation of an Infinite Impulse Response (IIR) architecture. Our result demonstrates a design that can achieve a 0.0025π passband edge while providing 60dB stopband attenuation. The design is aimed to reduce filter pole magnitudes, providing tolerance for waveguide losses and fabrication errors. The narrowband filter is based on the foundation of latticed allpass sections, which makes it naturally realizable using basic photonic components. Furthermore, analysis is performed on delay length variations that can result from the fabrication process.

Iterative reweighted algorithms, as a class of algorithms for sparse signal recovery, have been found to have better performance than their non-reweighted counterparts. However, for solving the problem of multiple measurement vectors (MMVs), all the existing reweighted algorithms do not account for temporal correlations among source vectors and thus their performance degrades significantly in the presence of the correlations. In this work we propose an iterative reweighted sparse Bayesian learning (SBL) algorithm exploiting the temporal correlations, and motivated by it, we propose a strategy to improve existing reweighted ℓ2algorithms for the MMV problem, i.e. replacing their row norms with Mahalanobis distance measure. Simulations show that the proposed reweighted SBL algorithm has superior performance, and the proposed improvement strategy is effective for existing reweighted ℓ2algorithms.

We develop the MultiPass Lasso (MPL) algorithm for sparse signal recovery. MPL applies the Lasso algorithm in a novel, sequential manner and has the following important attributes. First, MPL improves the estimation of the support of the sparse signal by combining high quality estimates of its partial supports which are sequentially recovered via the Lasso algorithm in each iteration/pass. Second, the algorithm is capable of exploiting the dynamic range in the nonzero magnitudes. Preliminary theoretic analysis shows the potential performance improvement enabled by MPL over Lasso. In addition, we propose the Reweighted MultiPass Lasso algorithm which substitutes Lasso with MPL in each iteration of Reweighted ℓ1Minimization. Experimental results favorably support the advantages of the proposed algorithms in both reconstruction accuracy and computational efficiency, thereby supporting the potential of the MultiPass framework for algorithmic development.

We consider joint scheduling and diversity to enhance the benefits of multiuser diversity in a multiuser OFDMA scheduling system. The OFDMA spectrum is assumed to consist of $N_{RB}$ resource blocks and the reduced feedback scheme consists of each user feeding back only the best- $N_{FB}$ signal to noise ratios. Assuming maximum SNR scheduling, we develop a framework to analyze the performance of both adaptive and non-adaptive rate schemes for a general value of $N_{FB}$. Based on this framework, we provide closed-form expressions for the sum rate and the bit error probability when transmit antenna selection is employed at the transmitter and maximal ratio combining is utilized at the receivers.

In this paper, the problem of adaptive bit and power allocation for coded OFDM systems is considered. With emphasis on low feedback overhead and low complexity, we present a simple adaptive allocation technique, where data are transmitted only using the strongest subcarriers with uniform power and uniform constellation size. Due to the reduced number of used subcarriers, a fraction of coded bits are not transmitted, while keeping the information bit rate unchanged. The problem of bit and power allocation to minimize packet error rate (PER) reduces to determine the constellation size and the number of unused subcarriers. The optimal solution provides a good tradeoff among average SNR, coding gain, and demapping loss. To solve the problem, we introduce an effective SNR maximization approach. Numerical results show that the proposed scheme significantly improves PER performance.

Current communication schemes rely on instantaneous adaptation, which increases the system complexity and is memoryless. We propose a scheme for adapting system parameters based on a priori knowledge of the channel statistics to reduce feedback requirements. As a vital parameter in OFDM system, cyclic prefix (CP) length depends mainly on two channel characteristics: the channel gains and the root mean square (RMS) delay spread. Previous research assume given RMS delay spread and utilize power delay profile (PDP) for CP optimization. Since PDP indeed acts as a priori statistics of channel gains, we compare its effectiveness against a feedback scheme based on the instantaneous channel. We further simplify CP optimization by employing the prior statistics of the RMS delay spread. We derive a closed form approximation scheme for computation simplicity assuming RMS delay spread follows a lognormal distribution. We consider ergodic capacity and measure the capacity loss w.r.t. an instantaneous scheme. The observed 1% - 4% capacity loss through simulation validates our proposed scheme. We also present an online method to learn the prior statistics of RMS delay spread from feedback whenever available.

In this paper, we consider the problem of exact support recovery of sparse signals via noisy linear measurements. The main focus is finding the sufficient and necessary condition on the number of measurements for support recovery to be reliable. By drawing an analogy between the problem of support recovery and the problem of channel coding over the Gaussian multiple-access channel (MAC), and exploiting mathematical tools developed for the latter problem, we obtain an information-theoretic framework for analyzing the performance limits of support recovery. Specifically, when the number of nonzero entries of the sparse signal is held fixed, the exact asymptotics on the number of measurements sufficient and necessary for support recovery is characterized. In addition, we show that the proposed methodology can deal with a variety of models of sparse signal recovery, hence demonstrating its potential as an effective analytical tool.

Many practical methods for finding maximally sparse coefficient expansions involve solving a regression problem using a particular class of concave penalty functions. From a Bayesian perspective, this process is equivalent to maximum a posteriori (MAP) estimation using a sparsity-inducing prior distribution (Type I estimation). Using variational techniques, this distribution can always be conveniently expressed as a maximization over scaled Gaussian distributions modulated by a set of latent variables. Alternative Bayesian algorithms, which operate in latent variable space leveraging this variational representation, lead to sparse estimators reflecting posterior information beyond the mode (Type II estimation). Currently, it is unclear how the underlying cost functions of Type I and Type II relate, nor what relevant theoretical properties exist, especially with regard to Type II. Herein a common set of auxiliary functions is used to conveniently express both Type I and Type II cost functions in either coefficient or latent variable space facilitating direct comparisons. In coefficient space, the analysis reveals that Type II is exactly equivalent to performing standard MAP estimation using a particular class of dictionary- and noise-dependent, nonfactorial coefficient priors. One prior (at least) from this class maintains several desirable advantages over all possible Type I methods and utilizes a novel, nonconvex approximation to thel0norm with most, and in certain quantifiable conditions all, local minima smoothed away. Importantly, the global minimum is always left unaltered unlike standardl1-norm relaxations. This ensures that any appropriate descent method is guaranteed to locate the maximally sparse solution.

Existing work on limited feedback for cooperative multicell beamforming quantizes the desired and interfering channel state information (CSI) using separate codebooks. In this paper, it is shown that comparatively higher sum-rates can be obtained by jointly quantizing the desired and interfering CSI using a single codebook. A selection criterion is developed for random vector quantization (RVQ) to show that joint quantization with RVQ yields higher sum-rates than those obtained using separate codebooks. The generalized Lloyd algorithm is then used to generate codebooks using the codeword design strategy proposed in this paper. Simulations are used to show that the proposed joint quantization approaches perform almost as well as the full CSI case.

In this work, we look at single user and multiuser Multiple-Input Multiple-Output (MIMO) beamforming networks with Channel Distribution Information (CDI). CDI does not need to be updated every time the channel changes, but only when the statistics of the channel change. Thus, feedback in the network is significantly reduced when compared to CSI schemes. With statistical information, we can only guarantee quality of service for a certain probability of outage in the network. The optimal beamformers and an optimal power control algorithm to minimize the power cost function in the network are presented for the well-known Kronecker model.

In this paper, we develop algorithms for robust linear regression by leveraging the connection between the problems of robust regression and sparse signal recovery. We explicitly model the measurement noise as a combination of two terms; the first term accounts for regular measurement noise modeled as zero mean Gaussian noise, and the second term captures the impact of outliers. The fact that the latter outlier component could indeed be a sparse vector provides the opportunity to leverage sparse signal reconstruction methods to solve the problem of robust regression. Maximum a posteriori (MAP) based and empirical Bayesian inference based algorithms are developed for this purpose. Experimental studies on simulated and real data sets are presented to demonstrate the effectiveness of the proposed algorithms.

In this paper, we explore the problem of separating convolutedly mixed signals in the overcomplete (degenerate) case of having more sources than sensors. We exploit a common form of nonstationarity, especially present in speech, wherein the signals have silence periods intermittently, hence varying the set of active sources with time. A novel approach is proposed that takes advantage of different combinations of silence gaps in the source signals at each time period. This enables the algorithm to “glimpse” or listen in the gaps, hence compensating for the global degeneracy by allowing it to learn the mixing matrices at periods where it is locally less degenerate. Experiments using simulated and real room recordings were carried out yielding good separation results.

In this paper we analyze the performance of random beamforming schemes in a multi-user Gaussian broadcast channel. Each user will have N > 1 receive antennas allowing optimal combining to be performed. To notify the transmitter of its current channel state, each user feeds back their SINR after LMMSE combining. Two feedback schemes are analyzed. The first scheme allows each user to feedback the post-processed SINR for each of the random transmit beams. To analyze this scheme, the distribution of the post-processed SINR is found. The second scheme attempts to limit feedback by allowing each user to feedback only the largest observed post-processed SINR and the index of the associated transmit beam. Using the Fréchet bounds, the throughput of the reduce feedback scheme is bounded and shown to have the same asymptotic scaling properties as the first scheme. Empirically, it is observed that as the number of users in the system increases, the reduced feedback scheme approaches the throughput of the scheme without thresholding.

Sparse signal recovery algorithms utilizing multiple measurement vectors (MMVs) are known to have better performance compared to the single measurement vector case. However, current work rarely consider the case when sources have temporal correlation, a likely situation in practice. In this work we examine methods to account for temporal correlation and its impact on performance. We model sources as AR processes, and then incorporate such information into the framework of sparse Bayesian learning for sparse signal recovery. Experiments demonstrate the superiority of the proposed algorithms. They also show that the performance of existing algorithms are limited by temporal correlation, and that if such correlation can be fully exploited, as in our proposed algorithms, the limitation can be overcome.

We consider a convolutionally coded orthogonal frequency-division multiple access (OFDMA) based cognitive radio system where each user achieves perfect synchronization for its own signal, while different users are asynchronized due to random timing offsets. The presence of asynchronous secondary users introduces potential multiple access interference (MAI) to each primary user's receiver. The strength of the MAI is determined by the secondary users' transmission powers, and their distances and frequency separations to the primary user. Expressions for the primary user's pairwise error probability and average probability of error are derived, and an error floor prediction method is presented. Finally, the trade-off between the MAI to the primary user and the secondary users' performances are investigated through numerical examples and simulations.

In Multi-User MIMO networks, receivers decode multiple concurrent signals using Successive Interference Cancellation (SIC). With SIC a weak target signal can be deciphered in the presence of stronger interfering signals. However, this is only feasible if each strong interfering signal satisfies a signal-to-noise-plus-interference ratio (SINR) requirement. This necessitates the appropriate selection of a subset of links that can be concurrently active in each receiver's neighborhood; in other words, a sub-topology consisting of links that can be simultaneously active in the network is to be formed. If the selected sub-topologies are of small size, the delay between the transmission opportunities on a link increases. Thus, care should be taken to form a limited number of sub-topologies. We find that the problem of constructing the minimum number of sub-topologies such that SIC decoding is successful with a desired probability threshold, is NP-hard. Given this, we propose MUSIC, a framework that greedily forms and activates sub-topologies, in a way that favors successful SIC decoding with a high probability. MUSIC also ensures that the number of selected sub-topologies is kept small. We provide both a centralized and a distributed version of our framework. We prove that our centralized version approximates the optimal solution for the considered problem. We also perform extensive simulations to demonstrate that (i) MUSIC forms a small number of sub-topologies that enable efficient SIC operations; the number of sub-topologies formed is at most 17% larger than the optimum number of topologies, discovered through exhaustive search (in small networks). (ii) MUSIC outperforms approaches that simply consider the number of antennas as a measure for determining the links that can be simultaneously active. Specifically, MUSIC provides throughput improvements of up to 4 times, as compared to such an approach, in various topological settings. The improvements can be directly attributable to a significantly higher probability of correct SIC based decoding with MUSIC.

We study the tradeoffs between the number of measurements, the signal sparsity level, and the measurement noise level for exact support recovery of sparse signals via random noisy measurements. By drawing analogy between exact support recovery and communication over the Gaussian multiple access channel, and exploiting mathematical tools developed for the latter problem, we derive sharp asymptotic sufficient and necessary conditions for exact support recovery. Specifically, when the number of nonzero entries is held fixed, the exact asymptotics on the number of measurements for support recovery is developed. When the number of nonzero entries increases in certain manners, we obtain sufficient conditions tighter than existing results. The proposed information theoretic framework for analyzing the performance of support recovery is further demonstrated to be capable of dealing with a variety of sparse signal recovery models.

Microphones and cameras have been extensively used to observe and detect human activity and to facilitate natural modes of interaction between humans and intelligent systems. Human brain processes the audio and video modalities, extracting complementary and robust information from them. Intelligent systems with audiovisual sensors should be capable of achieving similar goals. The audiovisual information fusion strategy is a key component in designing such systems. In this paper, we exclusively survey the fusion techniques used in various audiovisual information fusion tasks. The fusion strategy used tends to depend mainly on the model, probabilistic or otherwise, used in the particular task to process sensory information to obtain higher level semantic information. The models themselves are task oriented. In this paper, we describe the fusion strategies and the corresponding models used in audiovisual tasks such as speech recognition, tracking, biometrics, affective state recognition, and meeting scene analysis. We also review the challenges and existing solutions and also unresolved or partially resolved issues in these fields. Specifically, we discuss established and upcoming work in hierarchical fusion strategies and cross-modal learning techniques, identifying these as critical areas of research in the future development of intelligent systems.

Independent vector analysis (IVA) is a method for separating convolutedly mixed signals that significantly reduces the occurrence of the well-known permutation problem in frequency domain blind source separation (BSS). In this paper, we develop a novel IVA-based unifying framework for overcomplete/complete/undercomplete convolutive noisy BSS. We show that in order for the sources to be separable in the frequency domain, they must have a temporal dynamic structure. We exploit a common form of dynamics, especially present in speech, wherein the signals have silence periods intermittently, hence varying the set of active sources with time. This feature is extremely useful in dealing with overcomplete situations. An approach using hidden Markov models (HMMs) is proposed that takes advantage of different combinations of silence gaps of the source signals at each time period. This enables the algorithm to “glimpse” or listen in the gaps, hence compensating for the global degeneracy by allowing it to learn the mixing matrices at periods where it is locally less degenerate. The same glimpsing strategy can be employed to the complete/undercomplete case as well. Moreover, additive noise is considered in our model. Real and simulated experiments were carried out for overcomplete convoluted mixtures of speech signals yielding improved separation results compared to a sparsity-based robust time-frequency masking method. Signal-to-disturbance ratio (SDR) and machine intelligibility of a speech recognizer was used to evaluate their performances. Experiments were also conducted for the classical complete setting using the proposed algorithm and compared with standard IVA showing that the results compare favorably.

This paper proposes a two microphone-based source localization technique for multiple speech sources utilizing speech specific properties and novel clustering algorithms. Voiced speech is sparse in the frequency domain and can be represented by sinusoidal tracks via sinusoidal modeling which provides high local signal-to-noise ratio (SNR). By utilizing the inter-channel phase differences (IPDs) between the dual channels on the sinusoidal tracks, the source localization of the mixed multiple speech sources is turned into a clustering problem on the IPD versus frequency plot. The generalized mixture decomposition algorithm (GMDA) is used to cluster the groups of points corresponding to multiple sources and thus estimate the direction of arrival (DOA) of the sources. Experiments illustrate the proposed GMDA algorithm with the Laplacian noise model can estimate the number of sources accurately and exhibits smaller DOA estimation error than the baseline histogram based DOA estimation algorithm in various scenarios including reverberant and additive white noise environments. Experiments suggest that appropriate power thresholding can be a simple and good approximation to the sinusoidal modeling, for the purpose of selecting time-frequency points with high local SNR, with slight loss in performance.

Independent vector analysis (IVA) is a method for separating convolutedly mixed signals that avoids the well-known permutation problem in frequency domain blind source separation (BSS). In this paper, we exploit the nonstationarity of signals, a common feature, for BSS. One common type of nonstationarity, especially in speech, is that the signal can have silence periods intermittently, hence varying the set of active sources with time. To deal with such situations, we propose a novel state-based IVA algorithm. Moreover, we consider additive noise in our model. Computer simulations are conducted to compare the proposed method with the standard IVA and the results compare favorably.

Reverberant environments pose a challenge to speech acquisition from distant microphones. Approaches using microphone arrays have met with limited success. Recent research using audio-visual sensors for tasks such as speaker localization has shown improvement over traditional audio-only approaches. Using computer vision techniques we can estimate the orientation of the speaker's head in addition to the location of the speaker. In this paper we study the utility of using the head pose information for effective beamforming and clean speech acquisition from distant microphones. The improvements in speech recognition accuracy relative to that of a close talking microphone are presented and the results provide sufficient motivation for incorporating head pose information in beamforming techniques.

A two microphone direction of arrival (DOA) estimation technique for multiple speech sources is developed which exploits speech specific properties, namely sparsity in time-frequency (spectrum) domain. For robustness, we exploit the sparsity in the frequency domain by focusing on the spectral content concentrated in sinusoidal tracks obtained through sinusoidal modeling. When multiple speeches are mixed in the two microphone system, the inter-channel phase differences (IPD) between the dual channels on those sinusoidal tracks will be dominated by the spatial information of the most powerful source at that specific time-frequency point because of the spectrum sparsity and masking effects. Thereby, the source localization problem is turned into a clustering problem on the IPD versus frequency plot, and the generalized mixture decomposition algorithm (GMDA) is used to cluster the groups of points corresponding to multiple sources. The DOA of each source is derived from the parameters of each cluster. Experimental results conducted show the scheme to be very effective.

In this paper, we propose two approaches for combining geometric information with ICA algorithm to solve permutation problem under the scenario where a rough information about the direction of the desired source is known. The first approach is a new blind extraction algorithm with a soft quadratic geometric constraint. The desired source is guaranteed to be conveyed to the output with little distortion by the quadratic constraint and the negentropy maximization criterion is used to ensure that the other sources get suppressed at the output. The second approach employs a quadratic geometric test as a post-processing step to pickup the desired source after ICA processing. An advantage of the proposed two approaches is that they do not require accurate knowledge of the number of sources in the mixtures to recover the desired source, in contrast, other geometric ICA approaches usually fail if the number of sources is not known accurately.

This paper proposes a method to improve the robustness of the carrier sense system for opportunistic MAC (OMAC). This carrier sense method is specific for MIMO and the new version here illustrated is robust to noise and interference. Network simulations prove that this modified MIMO carrier sense correctly estimates whether a new communication would harm other ongoing transmissions for a wide range of system parameters.

In this paper, an adaptive opportunistic routing scheme for multi-hop wireless ad-hoc networks is proposed. The proposed scheme utilizes a reinforcement learning framework to achieve the optimal performance even in the absence of reliable knowledge about channel statistics and network model. This scheme is shown to be optimal with respect to an expected average per packet cost criterion. The proposed routing scheme jointly addresses the issues of learning and routing in an opportunistic context, where the network structure is characterized by the transmission success probabilities. In particular, this learning framework leads to a stochastic routing scheme which optimally ldquoexploresrdquo and ldquoexploitsrdquo the opportunities in the network.

In this paper, we analyze the performance of transmit beamforming on multiple-antenna Rayleigh fading channels with imperfect channel feedback. We characterize the feedback imperfections in terms of noisy channel estimation, feedback delay, and finite-rate channel quantization. We develop a general framework, that is valid for an arbitrary two-dimensional linear modulation, to capture the aforementioned imperfections and derive the symbol and bit error probability expressions for both M-PSK and M-ary rectangular QAM constellations with Gray code mapping. We show that the proposed analytical formulation is valid for a frequency-domain duplexing system with/without finite-rate channel quantization and a time-domain duplexing system. We validate the accuracy of the analysis through simulations, and assess the relative effects of channel estimation inaccuracy, feedback delay, and finite-rate quantization on the symbol and bit error performances for various constellations.

This paper investigates the applicability of multiple-input-multiple-output (MIMO) technology to broadband fixed wireless access (BFWA) systems operating in the 10-66 GHz frequency range. In order to employ the MIMO principle at these frequencies, the spatial channel benefits that may arise from the rainfall spatial inhomogeneity are more relevant since multipath is insignificant. Therefore, a special MIMO/BFWA channel may be implemented if every subscriber is equipped with multiple antennas and communicates with multiple base stations. The exact relationship between conventional MIMO and the proposed 10-66 GHz MIMO/BFWA channels is established. Then, emphasis is put on two different topics from the field of MIMO applications: (i) capacity enhancement for spatial multiplexed MIMO/BFWA systems; and (ii) interference reduction for MIMO/BFWA diversity systems employing receive antenna selection. More specifically, in the first case, a communication-oriented single-user capacity analysis of a 2 times 2 MIMO/BFWA spatial multiplexing system is presented, the relevant optimal power allocation policy is explored and useful analytical expressions are derived for the outage capacity achieved in the asymptotically low and high SNR regions. The effect of feedback on the capacity is investigated and quantified through Monte Carlo simulations. In the second case, a 2 times 2 MIMO/BFWA diversity system with receive selection combining is considered and its efficiency to mitigate intrasystem/intersystem cochannel interference over the downstream channel is studied from a propagation point of view. A general analytical prediction model for the interference reduction obtained by such a 2times2 MIMO/BFWA diversity system is presented along with a numerical validation.

In this paper we describe the design concepts and prototype implementation of a situation aware ubiquitous computing system using multiple modalities such as National Marine Electronics Association (NMEA) data from Global Positioning System (GPS) receivers, text, speech, environmental audio, and handwriting inputs. While most mobile and communication devices know where and who they are, by accessing context information primarily in the form of location, time stamps, and user identity, the concept of sharing of this information in a reliable and intelligent fashion is crucial in many scenarios. A framework which takes the concept of context aware computing to the level of situation aware computing by intelligent information exchange between context aware devices is designed and implemented in this work. Four sensual modes of contextual information like text, speech, environmental audio, and handwriting are augmented to conventional contextual information sources like location from GPS, user identity based on IP addresses (IPA), and time stamps. Each device derives its context not necessarily using the same criteria or parameters but by employing selective fusion and fission of multiple modalities. The processing of each individual modality takes place at the client device followed by the summarization of context as a text file. Exchange of dynamic context information between devices is enabled in real time to create multimodal situation aware devices. A central repository of all user context profiles is also created to enable self-learning devices in the future. Based on the results of simulated situations and real field deployments it is shown that the use of multiple modalities like speech, environmental audio, and handwriting inputs along with conventional modalities can create devices with enhanced situational awareness.

Tracking humans in an indoor environment is an essential part of surveillance systems. Vision based and microphone array based trackers have been extensively researched in the past. Audio-visual tracking frameworks have also been developed. In this paper we consider human tracking to be a specific instance of a more general problem of information fusion in multimodal systems. Dynamic Bayesian networks have been the modeling technique of choice to build such information fusion schemes. The complexity and non-Gaussianity of distributions of the dynamic Bayesian networks for such multimodal systems have led to the use of particle filters as an approximate inference technique. In this paper we present an alternative approach to the information fusion problem. The iterative decoding algorithm is based on the theory of turbo codes and factor graphs used in communication systems. We modify and adapt the iterative decoding algorithm to do probabilistic inference for the problem of tracking humans in an indoor space, using multiple cameras and microphone arrays.

In this paper, we propose a novel asynchronous media access control (MAC) protocol, opportunistic MAC (OMAC), for multiple input multiple output (MIMO) ad-hoc networks. The proposed solution is based on closed loop minimal feedback antenna selection diversity scheme and optimum receive combining. The use of antenna selection diversity contributes to a reduction in the feedback information and the effective interference produced. To utilize the spatial degrees of freedom offered by MIMO, we propose the use of a novel rank based metric to obtain interference information as well as to enable multiple simultaneous transmissions and to make MAC decisions. The rank of the interference matrix, (RI) is used as a metric. We present the performance of the proposed solution from the throughput perspective for a single hop ad-hoc wireless network. Through analysis and simulation, we found that the proposed protocol significantly outperforms 802.11 MIMO and it obtained as high spatial degree of freedom utilization as 85%.

We address the problem of analyzing the effect of channel estimation errors on the average block error probability (BLEP) of transmit beamforming multiple input single output (MISO) system in a slowly varying Rayleigh fading wireless channel. For this purpose, we develop an accurate model of estimation errors in a block fading context and derive an analytical expression quantifying the impact of channel estimation errors on feedback MISO systems. In particular, the block fading model implies that the channel estimate and its error component are fixed for the entire block and an appropriate performance criteria is the average BLEP, a more complex metric to analyze. The derived closed form analytical expression for the average BLEP is validated by simulations.

In this paper, we present robust semi-blind (SB) algorithms for the estimation of beamforming vectors for multiple-input multiple-output wireless communication. The transmitted symbol block is assumed to comprise of a known sequence of training (pilot) symbols followed by information bearing blind (unknown) data symbols. Analytical expressions are derived for the robust SB estimators of the MIMO receive and transmit beamforming vectors. These robust SB estimators employ a preliminary estimate obtained from the pilot symbol sequence and leverage the second-order statistical information from the blind data symbols. We employ the theory of Lagrangian duality to derive the robust estimate of the receive beamforming vector by maximizing an inner product, while constraining the channel estimate to lie in a confidence sphere centered at the initial pilot estimate. Two different schemes are then proposed for computing the robust estimate of the MIMO transmit beamforming vector. Simulation results presented in the end illustrate the superior performance of the robust SB estimators.

This paper presents techniques relevant to the online training of Gaussian mixture vector quantizer (GMVQ) systems. techniques for learning from quantized data are considered, which enables online training configurations wherein the training is carried out remotely from the encoder. Next, methods for recursive training are presented, which eliminate the need to store large databases of example data, and also enable adaptive operation of the GMVQ system. These techniques are demonstrated on the problem of wideband speech spectrum quantization, and the performance losses due to the use of quantized training data are experimentally quantified as a function of the bit rate.

In this paper, we examine the problem of overcomplete representations and provide new insights into the problem of stable recovery of sparse solutions in noisy environments. We establish an important connection between the inverse problem that arises in overcomplete representations and wireless communication models in network information theory. We show that the stable recovery of a sparse solution with a single measurement vector (SMV) can be viewed as decoding competing users simultaneously transmitting messages through a Multiple Access Channel (MAC) at the same rate. With multiple measurement vectors (MMV), we relate the inverse problem to the wireless communication scenario with a Multiple-Input Multiple-Output (MIMO) channel. In each case, based on the connection established between the two domains, we leverage channel capacity results with outage analysis to shed light on the fundamental limits of any algorithm to stably recover sparse solutions in the presence of noise. Our results explicitly indicate the conditions on the key model parameters, e.g. degree of overcompleteness, degree of sparsity, and the signal-to-noise ratio, to guarantee the existence of asymptotically stable reconstruction of the sparse source.

We derive an asymptotic Newton algorithm for quasi-maximum likelihood estimation of the ICA mixture model, using the ordinary gradient and Hessian. The probabilistic mixture framework yields an algorithm that can accommodate non-stationary environments and arbitrary source densities. We prove asymptotic stability when the source models match the true sources. An example application to EEC segmentation is given.

The fusion of information from heterogenous sensors is crucial to the effectiveness of a multimodal system. Noise affect the sensors of different modalities independently. A good fusion scheme should be able to use local estimates of the reliability of each modality to weight the decisions. This paper presents an iterative decoding based information fusion scheme motivated by the theory of turbo codes. This fusion framework is developed in the context of hidden Markov models. We present the mathematical framework of the fusion scheme. We then apply this algorithm to an audio-visual speech recognition task on the GRID audio-visual speech corpus and present the results.

For delay-tolerant data applications in a wireless system, using average throughput as a quality of service (QoS) measure results in more efficient resources allocation strategies compared to using instantaneous signal to interference and noise ratio (SINR). By adapting both power and rate in a multiuser multi-antenna system based on the channel conditions, the overall power consumption can be greatly reduced. In this paper, we establish a general framework for optimal power and rate allocation when there is an average throughput constraint for each user. A key feature of the framework is that it allows consideration and evaluation of a wide range of practical receiver structures. The structural results show that the range of optimal rate and power allocation policy can be specified by a set of optimization problems over vectors, one for each fading state. The rate constraints are taken care by a rate weight vector μ, which can be tracked by a stochastic approximation algorithm. The performances of different receiver structures are compared numerically.

In this paper, we examine the performance limits of the Orthogonal Matching Pursuit (OMP) algorithm, which has proven to be effective in solving for sparse solutions to inverse problem arising in overcomplete representations. To identify these limits, we exploit the connection between sparse solution problem and multiple access channel (MAC) in wireless communication domain. The forward selective nature of OMP helps it to be recognized as a successive interference cancellation (SIC) scheme that decodes non-zero entries one at a time in a specific order. We leverage this SIC decoding order and utilize the criterion for successful decoding to develop the information-theoretic performance limitation for OMP, which involves factors such as dictionary dimension, signal-to-noise-ratio, and importantly, the relative behavior of the non-zeros entries. Supported by computer simulations, our proposed criterion is demonstrated to be asymptotically effective in explaining the behavior of OMP.

Recent research has demonstrated that by utilizing channel state information at the transmitter, the physical layer can be optimized to provide higher link capacity and throughput, more efficiently share the channel with multiple users, increase range by exploiting diversity due to spatial and frequency selectivity, and simplify multi-user receivers through known interference cancellation. Unfortunately, acquiring channel state information at the transmitter is difficult. In most systems, the only opportunity for the transmitter to learn about the channel is through a feedback control channel. Because feedback information is control overhead, the rate of the feedback channel is limited. This motivates the study of limited feedback techniques where only partial or quantized information from the receiver is conveyed back to the transmitter.

It is now well known that employing channel adaptive signaling in wireless communication systems can yield large improvements in almost any performance metric. Unfortunately, many kinds of channel adaptive techniques have been deemed impractical in the past because of the problem of obtaining channel knowledge at the transmitter. The transmitter in many systems (such as those using frequency division duplexing) can not leverage techniques such as training to obtain channel state information. Over the last few years, research has repeatedly shown that allowing the receiver to send a small number of information bits about the channel conditions to the transmitter can allow near optimal channel adaptation. These practical systems, which are commonly referred to as limited or finite-rate feedback systems, supply benefits nearly identical to unrealizable perfect transmitter channel knowledge systems when they are judiciously designed. In this tutorial, we provide a broad look at the field of limited feedback wireless communications. We review work in systems using various combinations of single antenna, multiple antenna, narrowband, broadband, single-user, and multiuser technology. We also provide a synopsis of the role of limited feedback in the standardization of next generation wireless systems.

In this paper, the problem of finite-rate feedback for spatially and temporally correlated Rayleigh fading multiple input single output (MISO) channels with estimation errors at the receiver and feedback delay is addressed. A model that captures estimation errors, feedback delay, and finite-rate quantization of the channel is developed. A novel codebook design algorithm that minimizes the loss in ergodic capacity is proposed. Simulation results show that the new codebook designed under the consideration of estimation errors and feedback delay outperforms the codebook designed assuming ideal conditions. Analysis for the loss in ergodic capacity for spatially i.i.d channels with channel estimation errors and delay (EED) is presented and validated through simulations.

We present a robust approach to modeling voiced speech using a family of minimum variance distortionless response (MVDR) spectral estimates. The method exploits the fact that for a fixed model order, for a sinusoidal signal in noise, the MVDR estimate at the sinusoidal frequency is approximately related to the sinusoidal and noise power in a simple linear manner with the coefficients being dependent on the model order. Modeling voiced speech as a sum of harmonic signals, we then use the aforementioned relationship along with a least squares approach to combine a family of MVDR estimates (MVDR estimates of different orders) and develop a robust approach for modeling voiced speech. Experimental results of spectral estimation of sinusoids, synthetic vowels, and actual speech signals at SNR of 0 dB and 5 dB using this approach indicate an increased resolution in the estimated MVDR spectra. The MFCC computed from the MVDR spectra using this approach are also used for speaker identification experiments on the TEMIT database at various SNR. The results indicate a reasonable improvement in recognition performance when compared to the MFCC and the fixed order MVDR-MFCC.

In this paper, utilizing high resolution quantization theory, we analyze the loss in average symbol error probability (SEP) for finite rate feedback MISO systems with rectangular M-QAM constellation. Assuming perfect channel estimation, no-feedback delay and error-less feedback, for spatially i.i.d and correlated channels we derive analytical expressions for loss in average SEP due to finite-rate channel quantization. We then consider the high-SNR regime and show that the loss associated with correlated case is related to the loss associated with the i.i.d case by a scaling constant given by the determinant of the correlation matrix. We also present simulation results in support of the analytical expressions.

This paper considers the high-rate performance of channel optimized source coding for noisy discrete symmetric channels with random index assignment. Specifically, with mean squared error (MSE) as the performance metric, an upper bound on the asymptotic (i.e., high-rate) distortion is derived by assuming a general structure on the codebook. This structure enables extension of the analysis of the channel optimized source quantizer to one with a singular point density: for channels with small errors, the point density that minimizes the upper bound is continuous, while as the error rate increases, the point density becomes singular. The extent of the singularity is also characterized. The accuracy of the expressions obtained are verified through Monte Carlo simulations.

A variety of Bayesian methods have recently been introduced for finding sparse representations from overcomplete dictionaries of candidate features. These methods often capitalize on latent structure inherent in sparse distributions to perform standard MAP estimation, variational Bayes, approximation using convex duality, or evidence maximization. Despite their reliance on sparsity-inducing priors however, these approaches may or may not actually lead to sparse representations in practice, and so it is a challenging task to determine which algorithm and sparse prior is appropriate. Rather than justifying prior selections and modelling assumptions based on the credibility of the full Bayesian model as is commonly done, this paper bases evaluations on the actual cost functions that emerge from each method. Two minimal conditions are postulated that ideally any sparse learning objective should satisfy. Out of all possible cost functions that can be obtained from the methods described above using (virtually) any sparse prior, a unique function is derived that satisfies these conditions. Both sparse Bayesian learning (SBL) and basis pursuit (BP) are special cases. Later, all methods are shown to be performing MAP estimation using potentially non-factorable implicit priors, which suggests new sparse learning cost functions.

We analyze the performance of transmit beamforming on multiple-antenna Rayleigh fading channels with imperfect channel feedback. The feedback imperfections are characterized by noisy channel estimation, feedback delay, and finite rate channel quantization. We develop a general framework, valid for an arbitrary two-dimensional linear modulation, that captures the aforementioned imperfections, and derive the average symbol and bit error probability expressions for M- PSK constellation. We show that the proposed analytical formulation is valid for frequency-domain duplexing system with/without finite rate channel quantization and time-domain duplexing system. The simulation results show accurate agreement with the analytical expressions.

MAESTRO, a novel transmission scheme using multiple synchronized transmitters, is introduced in this paper. A non-zero (but known) symbol timing offset is introduced between the signals transmitted from the different transmitters to improve the performance of MIMO systems. This leads to a reduction in interference and it is shown that an advanced receiver can utilize this information to extract significant performance gains. This transmission scheme may be used in conjunction with different kinds of receivers. In particular, comparisons are made to the joint detection MMSE receivers, to the MIMO VBLAST MMSE receiver as well as to the MIMO VBLAST MMSE receiver with ordered successive interference cancellation. It is shown that under certain conditions the MAESTRO system could achieve close to 6dB of performance gain in comparison to an equivalent system with no timing offset.

This paper describes a robust feature extraction technique for continuous speech recognition. Central to the technique is the minimum variance distortionless response (MVDR) method of spectrum estimation. We consider incorporating perceptual information in two ways: 1) after the MVDR power spectrum is computed and 2) directly during the MVDR spectrum estimation. We show that incorporating perceptual information directly into the spectrum estimation improves both robustness and computational efficiency significantly. We analyze the class separability and speaker variability properties of the features using a Fisher linear discriminant measure and show that these features provide better class separability and better suppression of speaker-dependent information than the widely used mel frequency cepstral coefficient (MFCC) features. We evaluate the technique on four different tasks: an in-car speech recognition task, the Aurora-2 matched task, the Wall Street Journal (WSJ) task, and the Switchboard task. The new feature extraction technique gives lower word-error-rates than the MFCC and perceptual linear prediction (PLP) feature extraction techniques in most cases. Statistical significance tests reveal that the improvement is most significant in high noise conditions. The technique thus provides improved robustness to noise without sacrificing performance in clean conditions

This paper examines the design of recursive vector quantization systems built around Gaussian mixture vector quantizers. The problem of designing such systems for minimum high-rate distortion, under input-weighted squared error, is discussed. It is shown that, in high dimensions, the design problem becomes equivalent to a weighted maximum likelihood problem. A variety of recursive coding schemes, based on hidden Markov models are presented. The proposed systems are applied to the problem of wideband speech line spectral frequency (LSF) quantization under the log spectral distortion (LSD) measure. By combining recursive quantization and random coding techniques, the systems are able to attain transparent quality at rates as low as 36 bits per frame

This paper examines the problem of designing fixed-rate transform coders for sources whose distributions are unknown and presumably non-Gaussian, under input-weighted squared error distortion measures. As a component of this system, a flexible scalar compander based on Gaussian mixtures is proposed. The high-rate analysis of transform coders is reviewed, and extended to the case of input-weighted squared error. An algorithm is developed to set the parameters of the system using a data-driven technique that automatically balances the source statistics, distortion measure, and structure of the transform coder to minimize the high-rate distortion. The implementation of Gaussian mixture companders is explored, resulting in a flexible, low-complexity scalar quantizer. Additionally, modifications to the system for operation at moderate rates, using unstructured scalar quantizers, are presented. The operation of the system for the problem of wideband speech line spectral frequencies (LSF) quantization with log spectral distortion is illustrated, and shown to provide good performance with very low complexity

For any multiple-input multiple-output (MIMO) network with linear beamformers, it is shown that there exists a dual network that attains the same signal-to-interference-plus-noise ratio (SINR) performance. This network duality concept is a generalization of the virtual uplink concept investigated by Rashid-Farrokhi . We first develop the dual relation using a generic duality theory in linear programming. This approach naturally leads to the construction of a dual network, and provides machinery for handling a generalized cost function. We then consider the joint MIMO beamforming and power control problem with individual SINR constraints. We apply the network duality to this problem and propose a high-performance iterative algorithm which, compared with past centralized approaches, has improved convergence behavior. Finally, we propose a simpler distributed version of the algorithm tailored for a cellular downlink. This algorithm can be readily implemented in a distributed fashion, since it does not require information exchange between base stations. The algorithm achieves performance close to that of centralized ones, and outperforms other decentralized approaches available in the literature

In this paper, we demonstrate a driver intent inference system that is based on lane positional information, vehicle parameters, and driver head motion. We present robust computer vision methods for identifying and tracking freeway lanes and driver head motion. These algorithms are then applied and evaluated on real-world data that are collected in a modular intelligent vehicle test bed. Analysis of the data for lane change intent is performed using a sparse Bayesian learning methodology. Finally, the system as a whole is evaluated using a novel metric and real-world data of vehicle parameters, lane position, and driver head motion.

In this paper, we propose two efficient low-complexity quantization methods for multiple-input multiple-output (MIMO) systems with finite-rate feedback based on proper parameterization of the information to be fed back followed by quantization in the new parameter domain. For a MIMO channel which has multiple orthonormal vectors as channel spatial information, we exploit the geometrical structure of orthonormality while quantizing the spatial information matrix. The parameterization is of two types: one is in terms of a set of unit-norm vectors with different lengths, and the other is in terms of a minimal number of scalar parameters. These parameters are shown to be independent for the i.i.d. flat-fading Rayleigh channel, facilitating efficient quantization. In the first scheme, each of the unit-norm vectors is independently quantized with a finite number of bits using an optimal vector quantization (VQ) technique. Bit allocation is needed between the vectors, and the optimum bit allocation depends on the operating SNR of the system. In the second scheme, the scalar parameters are quantized. In slowly time-varying channels, the scalar parameters are also found to be smoothly changing over time, leading to the development of a simple quantization and feedback method using adaptive delta modulation. The results show that the proposed feedback scheme has a channel tracking feature and achieves a capacity very close to perfect feedback with a reasonable feedback rate

Summary form only given. This paper discusses the design of fixed-rate Gaussian mixture vector quantizers (GMVQs) under input-weighted squared error distortion measures. The goal is to select the system parameters so as to minimize the expected high-rate distortion. GMVQ systems produce low complexity by operating M Gaussian codebooks in parallel (typically with low-complexity structures) and then choosing amongst their outputs with an M-point vector quantizer. Thus, the total codebook is the union of the component Gaussian codebooks, and the total encoder regions are optimal provided the component encoders are optimal with respect to their individual codebooks

In this paper, new results are derived for the high-rate performance of source coding for symmetric error channels (i.e., a channel where all index errors are equally likely) for a large class of distortion measures. Expressions are derived for the expected distortion including the effect of channel errors as the number of quantization levels N gets large. It is shown that the distortion can asymptotically be approximated as the sum of the source quantization distortion and the channel error induced distortion. In addition, the expressions obtained can be used to glean key insights on the relative amounts of source and channel coding necessary to attain a balanced system, i.e., one where neither the distortion caused by the source quantization nor the distortion caused by channel errors dominate the performance. Optimization of the codebook for minimizing the expected distortion is also considered, and theoretical expressions are derived for the optimal point density.

This paper considers the development of a general framework for the analysis of transmit beamforming methods in multiple antenna systems with finite-rate feedback. Inspired by the results of classical high resolution quantization theory, the problem of a finite-rate quantized communication system is formulated as a general fixed-rate vector quantization problem with side information available at the encoder (or the quantizer) but unavailable at the decoder. The framework of the quantization problem is sufficiently general to include quantization schemes with general non-mean square distortion functions, and constrained source vectors. The result of the asymptotic distortion analysis of the proposed general quantization problem is presented, which extends the vector version of Bennett's integral. Specifically, tight lower and upper bounds on the average asymptotic distortion are proposed. The proposed general methodology provides a powerful analytical tool to study a wide range of finite-rate feedback systems. To illustrate the utility of the framework, the analysis of a finite-rate feedback MISO beamforming system over i.i.d. Rayleigh flat fading channels is derived. Numerical and simulation results are presented to further confirm the accuracy of the analytical results.

This paper examines the problem of designing fixed-rate transform coders for sources with arbitrary distributions, under input-weighted squared error distortion measures. As a component of this system, a flexible scalar compander using Gaussian mixtures is proposed. An algorithm is developed to set the parameters of the system using a data-driven technique that automatically balances the source statistics, distortion measure, and structure of the transform coder to minimize the high-rate distortion. The implementation of Gaussian mixture companders is explored, resulting in a flexible, low-complexity scalar quantizer. The operation of this system for the problem of wideband speech spectrum quantization with log spectral distortion is illustrated, and shown to provide good performance with very low, rate-independent complexity

In this paper, the sensitivity of the high-rate performance of conventional source coding to symmetric channel errors (i.e., a channel where all index errors are equally likely) with arbitrary distortion measures is analyzed. It is shown that, in general, the overall distortion due to source quantization and channel errors cannot be expressed as the sum of the distortion due to the finite bit representation of the source and the distortion due to channel errors. An exception to this is when the distortion is measured as the mean-squared error. The binary symmetric channel with random index assignment is a special case of the analysis, and as the number of code-points gets large, the performance approaches a nonzero constant. Finally, the framework is applied to the wideband speech spectrum quantization problem, where it correctly predicts the channel error rate permissible for operation at a particular distortion level

In this paper, we develop a robust wideband adaptive beamforming algorithm for microphone array based speech recognition. We develop a quadratic constraint based approach to deal with the uncertainty in the look direction. In order to address the ill-conditioning associated with the constraint matrix, diagonal loading (DL) is employed. The advantage of adding DL to the constraint matrix is that the constraint matrix is only determined by the geometry of the array thereby allowing the DL level to be chosen offline. We also develop an iterative algorithm (and corresponding adaptive algorithm) to solve for the robust beamformer coefficients. The developed algorithm is applied to the problem of beamforming using microphone arrays for speech recognition and shown to be superior to existing algorithms.

We consider in this paper the analysis of transmit beamforming methods in multiple antenna systems with finite-rate feedback of the channel state information. We focus our attention on providing capacity analysis of a quantized MISO system over correlated fading channels with sub-optimal and mismatched channel quantizers. Two types of mismatched quantizers are investigated, which include: 1) quantizers designed with simple suboptimal criterion, and 2) quantizers whose codebooks are designed with a mismatched channel covariance matrix. We approach this problem from a source coding perspective by formulating the quantized MISO system as a general vector quantization problem with encoder side information, constrained quantization space and non-mean-squared distortion function. By utilizing the high-resolution distortion analysis of the generalized quantizer, we obtain tight lower bounds of the capacity loss of a quantized MISO system with both optimal and mismatched channel quantizers. Theoretical as well as empirical results reveal significant performance degradation of the mismatched quantizers when compared to the optimal channel quantizers. This elucidates the importance of choosing proper codebook design criterion and using correct source statistical distributions.

In this paper, we consider multiple-input, single-output (MISO) systems with per-antenna constrained beamforming at the transmitter, also known as equal gain transmission (EGT). The i.i.d. Rayleigh flat fading channel is assumed known to the receiver, and is quantized and sent to the transmitter through through a noisy finite-rate feedback channel. We model the noisy feedback channel as a symmetric error channel (i.e.,a channel where all index errors are equally likely), and analyze the effect of feedback channel errors on the performance of quantized EGT systems. It is found that the asymptotic performance (as the number of quantized codepoints gets large) of EGT with quantized feedback via a noisy channel depends on the channel behavior as the number of quantized points increase. It turns out that the binary symmetric channel (BSC) with random index assignment is a special case of the analysis, and the asymptotic performance approaches that of random beamforming. The accuracy of the expressions obtained are further verified through Monte Carlo simulations.

We consider in this paper the analysis of transmit beamforming methods in multiple antenna systems over correlated fading channels and with finite rate feedback of the channel state information. The problem is formulated as a general vector quantization problem with encoder side information, constrained quantization space and non-mean-square distortion function. By utilizing the high-resolution distortion analysis of the generalized quantizer, which is applicable to a wide range of scenarios, we obtain a tight lower bound on the capacity loss of the finite rate quantized MISO system over correlated fading channels. The lower bound of the capacity loss of correlated MISO channels is a generalization of existing results available for i.i.d. channels. The bound, in addition to providing insight into the exact nature of dependence of the quantization loss on the channel correlation matrix, indicates that the loss is less than that of the i.i.d. channels but with the same exponential decaying factor w.r.t. the feedback rate. The generality of the framework is further demonstrated by considering its application to the analysis of suboptimal mismatched channel quantizers, i.e. quantizers designed with an incorrect channel covariance matrix, and comparing it to systems with optimal quantizers. Finally, numerical and simulation results of the finite rate quantized MISO beamforming system with codebook designed by the Lloyd algorithm are presented that confirm the accuracy of the obtained analytical results.

The ill-posed nature of the MEG/EEG source localization problem requires the incorporation of prior assumptions when choosing an appropriate solution out of an infinite set of candidates. Bayesian methods are useful in this capacity because they allow these assumptions to be explicitly quantified. Recently, a number of empirical Bayesian approaches have been proposed that attempt a form of model selection by using the data to guide the search for an appropriate prior. While seemingly quite different in many respects, we apply a unifying framework based on automatic relevance determination (ARD) that elucidates various attributes of these methods and suggests directions for improvement. We also derive theoretical properties of this methodology related to convergence, local minima, and localization bias and explore connections with established algorithms.

In this paper, we show how the Gaussian mixture modeling framework used to develop efficient source encoding schemes can be further exploited to model source statistics during channel decoding in an iterative framework to develop an effective joint source-channel decoding scheme. The joint probability density function (PDF) of successive source frames is modeled as a Gaussian mixture model (GMM). Based on previous work, the marginal source statistics provided by the GMM is used at the encoder to design a low-complexity memoryless source encoding scheme. The source encoding scheme has the specific advantage of providing good estimates to the probability of occurrence of a given source code-point based on the GMM. The proposed iterative decoding procedure works with any channel code whose decoder can implement the soft-output Viterbi algorithm that uses a priori information (APRI-SOVA) or the BCJR algorithm to provide extrinsic information on each source encoded bit. The source decoder uses the GMM model and the channel decoder output to provide a priori information back to the channel decoder. Decoding is done in an iterative manner by trading extrinsic information between the source and channel decoders. Experimental results showing improved decoding performance are provided in the application of speech spectrum parameter compression and communication.

In this paper, we use the Gaussian mixture model (GMM) based multidimensional companding quantization framework to develop two important quantization schemes. In the first scheme, the scalar quantization in the companding framework is replaced by more efficient lattice vector quantization. Low-complexity lattice pruning and quantization schemes are provided for the$E_8$Gossett lattice. At moderate to high bit rates, the proposed scheme recovers much of the space-filling loss due to the product vector quantizers (PVQ) employed in earlier work, and thereby, provides improved performance with a marginal increase in complexity. In the second scheme, we generalize the compression framework to accommodate recursive coding. In this approach, the joint probability density function (PDF) of the parameter vectors of successive source frames is modeled using a GMM. The conditional density of the parameter vector of the current source frame based on the quantized values of the parameter vector of the previous source frames is used to generate a new codebook for every current source frame. We demonstrate the efficacy of the proposed schemes in the application of speech spectrum quantization. The proposed scheme is shown to provide superior performance with moderate increase in complexity when compared with conventional one-step linear prediction based compression schemes for both narrow-band and wide-band speech.

This paper investigates quantization methods for feeding back the channel information through a low-rate feedback channel in the context of multiple-input single-output (MISO) systems. We propose a new quantizer design criterion for capacity maximization and develop the corresponding iterative vector quantization (VQ) design algorithm. The criterion is based on maximizing the mean-squared weighted inner product (MSwIP) between the optimum and the quantized beamforming vector. The performance of systems with quantized beamforming is analyzed for the independent fading case. This requires finding the density of the squared inner product between the optimum and the quantized beamforming vector, which is obtained by considering a simple approximation of the quantization cell. The approximate density function is used to lower-bound the capacity loss due to quantization, the outage probability, and the bit error probability. The resulting expressions provide insight into the dependence of the performance of transmit beamforming MISO systems on the number of transmit antennas and feedback rate. Computer simulations support the analytical results and indicate that the lower bounds are quite tight

In this paper, we analyze the performance of the uplink of multi-cellular MIMO systems in flat Rayleigh fading. There is co-channel interference from users within the same cell as well as from other cell users. I The channel model includes lognormal shadowing and path loss along with power control, resulting in a statistical model for user powers. Consistent with practical scenarios, the co-channel interference is categorized into two groups: intracell interference from users within the same cell as the desired user and intercell interference from outer cell users. We derive a compact, easily computable closed form outage probability expression in the form of finite sums. This expression allows for simpler and faster analysis of various MIMO configurations. It has been shown that using antennas on the receiver side results in better performance, since transmit diversity does not combat interference from same cell users.

In this work, we consider quantization of the beamforming vector for finite rate feedback-based communication in flat-fading multi input single output (MISO) systems with a per-antenna power constraint. Using the capacity loss with respect to perfect channel feedback as performance metric, we derive a new criterion, the mean-square weighted inner product (MSwIP) criterion, for designing the quantized beamforming vectors. We develop an iterative algorithm (based on the Lloyd algorithm) to generate the beamforming vector codebook that locally optimizes the design criterion. For the case of i.i.d. flat fading channels, we analyze the performance of the proposed algorithm in terms of capacity loss and the outage probability. We show that the capacity loss (in terms of bits) of a MISO system with quantized beamforming under a per-antenna power constraint is roughly half that obtained under a total power constraint. Simulation results verify the accuracy of the analytical expressions

We consider an LDPC-coded MIMO system composed of M transmit and N receive antennas operating in a flat fading environment. The channel state information is assumed to be unavailable both to the transmitter and the receiver. A soft iterative receiver structure is developed which consists of three main blocks, a soft MIMO detector and two LDPC component soft decoders. Without forming any specific channel estimate, we propose several soft MIMO detectors at the component level that offer an effective tradeoff between complexity and performance. At the structural level, the LDPC-coded MIMO receiver is constructed in a unconventional manner where the soft MIMO detector and LDPC variable node decoder form one super soft-decoding unit, and the LDPC check node decoder forms the other component of the iterative decoding scheme. By exploiting the proposed receiver structure, tractable extrinsic information transfer functions of the component soft decoders are obtained, which further lead to a simple and efficient LDPC code degree profile optimization algorithm with proven global optimality and guaranteed convergence from any initialization. Finally, numerical and simulation results are provided to confirm the advantages of the proposed design approach for the coded system.

We investigate semi-blind channel estimation for multiple input multiple output (MIMO) quasi-static flat fading channels when maximum ratio transmission (MRT) is employed. We propose a closed-form semi-blind solution (CFSB) for estimating the optimum transmit and receive beamforming vectors of the channel matrix. Employing matrix perturbation theory, we develop expressions for the mean squared error (MSE) in the beamforming vector and average received SNR of both the semi-blind and the conventional least squares estimation (CLSE) schemes. It is found that the proposed estimation technique outperforms CLSE for a wide range of training lengths and training SNRs.

We consider the problem of interference suppression in a multipath DS-CDMA downlink using a general RAKE receiver structure with M fingers and arbitrary finger delays. Utilizing asymptotically optimal sampling design theory, a new design approach that optimizes the receiver performance over the finger delays is proposed. It provides a set of delays which are asymptotically optimal for large M, and sheds light on the behavior of various finger placements' performance, as the number of fingers increases, and on the effect of the smoothness of the underlying chip pulse. Numerical results show that for certain Rayleigh fading channels and several chip pulses, the proposed finger placement is superior to the chip periodically-spaced finger placement.

Practical MIMO channels often exhibit structure in both space and time, i.e. a spatio-temporal structure. The potential of exploiting this structure in training based schemes is studied using a common ray-based channel model that captures parts of the structure observed in measurements. A lower bound, the Cramer-Rao lower bound (CRB), on the channel estimation error and a lower bound on the capacity are used to study the potential gain in exploiting channel structure. It is found that the training based capacity may be substantially increased since a more parsimonious channel model with fewer parameters to estimate can be used. Numerical evaluations indicate that the capacity grows with the number of antennas, similar to the case of a known channel if the structure is exploited. If it is not exploited, the training-based capacity reaches a maximum after which it decreases with the number of antennas. Furthermore, the temporal structure can be used to interpolate or predict the channel between training instants and it is found that prediction can improve performance for training based schemes.

We consider an LDPC-coded MIMO system operating in a fading environment, where channel state information is assumed to be unavailable both to the transmitter and the receiver. We propose at the component level a modified EM-based MIMO detector, which completely removes positive feedback between input and output extrinsic information and provide much better performance as compared to the regular EM-based detector that has strong correlations. At the structural level, motivated by the turbo iterative decoding strategy, a new LDPC-coded MIMO receiver is constructed in a novel manner where the soft MIMO detector and LDPC variable node decoder form one super soft decoding unit and the LDPC check node decoder the other component of the iterative decoding scheme. Based on the analysis of the extrinsic information transfer characteristic of the component soft decoders, a simple and efficient LDPC code optimization approach is also provided. Numerical and simulation results are provided to further confirm the optimality of the proposed design approach for the coded MIMO system.

This paper investigates the problem of transmit beamforming in MIMO spatial multiplexing (SM) systems with a finite-rate feedback channel. Assuming a fixed number of spatial channels and equal power allocation, we propose a new design criterion for designing the codebook of beamforming matrices that is based on minimizing the capacity loss resulting from the limited rate in the feedback channel. Using the criterion, we develop an iterative design algorithm that converges to an optimum codebook. Under the i.i.d, channel and high SNR assumption, the effect on channel capacity of the finite-bit representation of beamforming matrix is analyzed. Central to this analysis is the complex multivariate beta distribution and tractable approximations to the Voronoi regions associated with the code points. Furthermore, to compensate for the degradation due to the equal power allocation assumption, we propose a multi-mode SM transmission strategy wherein the number of data streams is determined based on the average SNR. This approach is shown to allow for effective utilization of the feedback bits.

For any multiple-input and multiple-output (MIMO) network with linear beamformers, there exists a dual network that attains the same signal to interference plus noise ratio (SINR) performance. This network duality is a generalization of the virtual uplink concept investigated in R. Rashid-Farrokhi et al. (1998) in the context of cellular networks. In this paper, we develop network duality which is applicable to arbitrary multi-user MIMO networks with a generalized cost function. More importantly, we provide an optimization theoretic perspective of network duality which naturally leads to the construction of a dual network. We then consider the joint MIMO beamforming and power control problem with individual SINR constraints. We apply the network duality to this problem and propose a high performance algorithm which compared to past approaches has improved convergence behavior.

We consider criteria for variational representations of non-Gaussian latent variables, and derive variational EM algorithms in general form. We establish a general equivalence among convex bounding methods, evidence based methods, and ensemble learning/Variational Bayes methods, which has previously been demonstrated only for particular cases.

Given a redundant dictionary of basis vectors (or atoms), our goal is to find maximally sparse representations of signals. Previously, we have argued that a sparse Bayesian learning (SBL) framework is particularly well-suited for this task, showing that it has far fewer local minima than other Bayesian-inspired strategies. In this paper, we provide further evi- dence for this claim by proving a restricted equivalence condition, based on the distribution of the nonzero generating model weights, whereby the SBL solution will equal the maximally sparse representation. We also prove that if these nonzero weights are drawn from an approximate Jef- freys prior, then with probability approaching one, our equivalence con- dition is satisfied. Finally, we motivate the worst-case scenario for SBL and demonstrate that it is still better than the most widely used sparse rep- resentation algorithms. These include Basis Pursuit (BP), which is based on a convex relaxation of the ℓ0 (quasi)-norm, and Orthogonal Match- ing Pursuit (OMP), a simple greedy strategy that iteratively selects basis vectors most aligned with the current residual.

The statistical gain differences between two common spatial combining algorithms: optimum combining (OC) and maximal ratio combining (MRC) are analyzed using a gain ratio method. Using the receive carrier-to-interference plus noise ratio (CINR), the gain ratio ClNR/sub OC/ /CINR/sub MRC/ is evaluated in a flat Rayleigh fading communications system with multiple interferers. Exact analytical solutions are derived for the probability density function (PDF) and the average gain ratio with one interferer. When more than one interferer is present, the PDF of the gain ratio is illustrated using Monte Carlo simulations and its mean value is shown in basic integral form. An upper bound to the gain ratio is derived providing a simple means to determine when OC will exhibit significant gains over MRC.

In this paper, we analyze the uplink outage capacity improvements in multicellular code-division multiple-access systems using receive antenna diversity and fast power control schemes that can track multipath fading. The channel is modeled as slowly varying Rayleigh fading with lognormal shadowing and path loss. Closed-form expressions are obtained for the outage capacity of the system by approximating the total intracell and intercell interference distributions. We show that by controlling the power level of a user at the output of the diversity combiner and imposing the condition that a user does not transmit when in a deep fade, the average intercell interference level of the system is significantly reduced. As a result of this reduction, we demonstrate both analytically and through simulations that the outage capacity of the system is improved more than linearly with increasing number of antenna elements.

We propose and study algorithms for constrained maximum-likelihood (ML) estimation of a unitary matrix in the context of semi-blind multi-input multi-output (MIMO) channel estimation. The flat-fading r/spl times/t MIMO channel matrix, H, for r/spl ges/t can be decomposed as the matrix product H = WQ/sup H/, where W is a whitening matrix and Q is a unitary rotation matrix. Exclusive estimation of Q from pilot symbols has been shown potentially to achieve a 3 dB or greater improvement in terms of channel estimation accuracy. We develop and present the OPML, IGML and ROML algorithms for the constrained estimation of the unitary matrix Q; they are appropriate for a variety of scenarios, e.g., orthogonal pilots, low complexity, etc. Simulation results are provided to demonstrate the efficacy of the algorithms.

In this paper, a low-complexity, high-quality recursive vector quantizer based on a generalized hidden Markov model of the source is presented. Capitalizing on recent developments in vector quantization based on Gaussian mixture models, we extend previous work on HMM-based quantizers to the case of continuous vector-valued sources, and also formulate a generalization of the standard HMM. This leads us to a family of parametric source models with very flexible modelling capabilities, with which are associated low-complexity recursive quantization structures. The performance of these schemes is demonstrated for the problem of wideband speech spectrum quantization, and shown to compare favorably to existing state-of-the-art schemes.

The problem of computing sparse solutions to linear inverse problems arises in a large number of signal processing application areas. We address the problem of finding sparse solutions to linear inverse problems when there are multiple measurement vectors (MMV) and the solutions are assumed to have a common, but unknown, sparsity profile. This is an important extension to the single measurement sparse solution problem that has been extensively studied in the past. Of particular interest are methods based on minimizing diversity measures. A measure appropriate for the multiple measurement problem is developed, and an algorithm is derived based on its minimization. The algorithm developed, M-FOCUSS, generalizes the focal underdetermined system solver (FOCUSS) algorithm developed for the single measurement case. The convergence of the algorithm is established and a simulation study is conducted to evaluate its effectiveness. The results clearly show the ability of M-FOCUSS to utilize multiple measurement vectors for accurate identification of the sparsity structure and sparse solution computation.

We analyze the performance of multi-cellular MIMO systems in Rayleigh fading. Consistent with practical scenarios, we assume two types of interference: intracell interference from users within the same cell as the desired user and intercell interference from users in other cells. We derive a compact closed form expression for the outage probability of such a system in the form of finite sums. The expression is easily computable and allows for a simpler and faster study of various MIMO configurations. An interesting outcome of the analysis is that using multiple antennas on the receiver side results in better performance since transmit diversity does not combat interference from same cell users.

Finding sparse representations of signals is an important problem in many application domains. Unfortunately, when the signal dictionary is overcomplete, finding the sparsest representation is NP-hard without some prior knowledge of the solution. However, suppose that we have access to such information. Is it possible to demonstrate any performance bounds in this restricted setting? We examine this question with respect to algorithms that minimize general /spl lscr//sub p/-norm-like diversity measures. Using randomized dictionaries, we analyze performance probabilistically under two conditions. First, when 0/spl les/p/spl les/1, we quantify (almost surely) the number and quality of every local minimum. Next, for the p=1 case, we extend the deterministic results of D.L. Donoho and M. Elad (see Proc. Nat. Acad. Sci., vol.100, no.5, 2003) by deriving explicit confidence intervals for a theoretical equivalence bound, under which the minimum /spl lscr//sub 1/-norm solution is guaranteed to equal the maximally sparse solution. These results elucidate our previous empirical studies applying /spl lscr//sub p/ measures to basis selection tasks.

We show how the Gaussian mixture modelling framework used to develop efficient source encoding schemes can be further exploited to model source statistics during channel decoding in an iterative framework to develop an effective joint source-channel decoding scheme. The joint probability density function (PDF) of successive source frames is modelled as a Gaussian mixture model (GMM). Based on previous work, the marginal source statistics provided by the GMM is used at the encoder to design a low-complexity memoryless source encoding scheme. The source encoding scheme has the specific advantage of providing good estimates to the probability of occurrence of a given source code-point based on the GMM. The proposed iterative decoding procedure works with any channel code whose decoder can implement the soft-output Viterbi algorithm that uses a priori information (APRI-SOVA) to provide extrinsic information on each source encoded bit. The source decoder uses the GMM model and the channel decoder output to provide a priori information back to the channel decoder. Decoding is done in an iterative manner by trading extrinsic information between the source and channel decoders. Experimental results showing improved decoding performance are provided in the application of speech spectrum parameter compression and communication.

We compare the performance of two multiple antenna systems to be used in quality of service (QoS) supported indoor wireless networks. While a conventional array antenna system (AAS) has collocated, closely spaced antenna elements, a distributed antenna system (DAS) has largely spaced antennas over the entire area of radio coverage. To support multimedia applications requiring high bandwidth and on time delivery, we propose a set of highly spectrum efficient radio resource management algorithms. We focus on the optimization of downlink since many kinds of Internet traffic show the downlink dominance in their traffic asymmetry. To maximize the downlink throughput, we present a new transmit beamforming algorithm which can be equally applied to both DAS and AAS. The beamforming algorithm is integrated with a link scheduling algorithm that exploits the space division multiplexing (SDM) capability of multiple antenna systems to meet the QoS requirements of all terminals. Numerical examples conducted for a line of sight (LOS) environment demonstrate that a network with DAS outperforms one with AAS in terms of signal coverage and provides 40-160% higher capacity.

We consider a mobile wireless communication system composed of M transmit and N receive antennas operating in a fading environment. Assuming channel state information is unavailable to the transmitter and the receiver, a capacity upper bound of the unknown MIMO channel under the assumption of restricted input distributions is provided. By analyzing the proposed capacity upper bounds, we reinforce the advantages of using an orthogonal pilot structure which minimizes the mean square estimation error, in that it also maximizes the capacity upper bounds. Interestingly, the capacity upper bound is shown to be a monotonically decreasing function with respect to the number of pilot symbols, T/sub /spl tau//. Numerical evaluations of the capacity upper bound further demonstrate that the capacity gain is insignificant when T/sub /spl tau// decreases below M, suggesting an optimum training duration of M time slots.

Finding the sparsest, or minimum ℓ0-norm, representation of a signal given an overcomplete dictionary of basis vectors is an important prob- lem in many application domains. Unfortunately, the required optimiza- tion problem is often intractable because there is a combinatorial increase in the number of local minima as the number of candidate basis vectors increases. This deficiency has prompted most researchers to instead min- imize surrogate measures, such as the ℓ1-norm, that lead to more tractable computational methods. The downside of this procedure is that we have now introduced a mismatch between our ultimate goal and our objective function. In this paper, we demonstrate a sparse Bayesian learning-based method of minimizing the ℓ0-norm while reducing the number of trou- blesome local minima. Moreover, we derive necessary conditions for local minima to occur via this approach and empirically demonstrate that there are typically many fewer for general problems of interest.

We investigate the quantization of multiple antenna channel to feed back through a low-rate feedback channel. Specifically, for multiple-input single-output (MISO) systems, we propose a new design criterion and the corresponding design algorithm for quantization of the random beamforming vector. For multiple-input multiple-output (MIMO) channels, which have multiple orthonormal vectors as channel spatial information for quantization, a matrix factorization method is proposed which provides a way to exploit the geometrical structure of orthonormality while quantizing the spatial information matrix. Results show that the quantization bit allocation over multiple spatial channels has a critical effect on the performance, and that the optimum bit allocation depends on the operating transmit power of the system.

The capacity of a multiple-input multiple-output (MIMO) channel can be improved if the transmitter has a knowledge of the channel. In this paper, we propose an efficient and practical feedback method based on parameterization and quantization of channel parameters. The spatial information of the channel at the transmitter, which is represented as a matrix with orthonormal columns, has a geometrical structure. In parameterization, the geometrical structure is exploited to extract a set of parameters that has a one-to-one mapping to the original matrix. In slowly time-varying channels, the parameters are also found to he smoothly changing in time. We employ adaptive delta modulation to quantize and feed back each parameter. The results show that the proposed feedback scheme has a channel tracking feature and achieves a capacity very close to the perfect feedback case with a reasonable feedback rate.

In this paper, we analyze the performance of multi-cellular MIMO systems with co-channel interference from users within the same cell as well as from other cells. We initially assume that user power levels are deterministic. Then, we extend the channel model to include shadowing and path loss along with power control, resulting in a statistical model for user powers. We derive easily computable closed form outage probability expressions in the form of finite sums for both cases. It is found that using antennas on the receiver side results in better performance since transmit diversity does not combat interference from same cell users. It is also shown that the outage capacity increases more than linearly with increasing order of diversity.

An expression for the Cramer-Rao lower bound (CRB) on the covariance of unbiased estimators of a constrained complex parameter vector is derived. The application and usefulness of the result is demonstrated through its use in the context of a semiblind channel estimation problem.

In this letter, we examine the statistical distribution of the signal-to-interference-plus-noise ratio (SINR) of the optimum combining (OC) technique in receive-antenna diversity systems. We decompose the SINR into two components by projection onto the interference subspace for systems with more antennas than interferers. For the case when the number of interferers is larger than the number of antennas, upper and lower bounds on performance are provided.

We investigate transmission strategies for flat-fading multiple antenna channels with t transmit and r receive antennas, and with channel state information (CSI) partially known to the transmitter. We start with an assumption that the first n eigenvectors of H/sup /spl dagger//H, where 0/spl les/n/spl les/min(t,r) and H is the channel matrix in /spl Copf//sup r/spl times/t/, are available at the transmitter as partial spatial information of the channel. A beamforming method is proposed in which a beamforming matrix is determined from the n eigenvectors in some predefined way; as a result, the receiver also knows the beamforming matrix. With this beamforming scheme, we develop a new multiple antenna system concept that provides a mechanism to reduce the amount of channel feedback information. This paper focuses on deriving the channel capacity of the multiple antenna channels employing the proposed beamforming and feedback methods. An important task for achieving capacity is the solution of interesting optimization problems for the optimal power allocation over the transmit symbols. The results show that the proposed methods lead to systems wherein the amount of feedback information can be significantly reduced with a minor sacrifice of achievable transmission rate.

We provide a mathematical framework for developing adaptive filtering algorithms for exploiting/enforcing sparsity. The approach is based on minimizing a regularized mean squared error criterion with sparsity being promoted by the regularizing term which consists of a diversity measure. A steepest descent algorithm (SDA) is developed to minimize the regularized cost function. Then we extend the algorithm to the adaptive environment and develop a class of algorithms, which we term the pLMS algorithm class and which incudes important variants - pLLMS (leaky pLMS) and pNLMS (normalized pLMS). The framework is quite general and encompasses a broad range of adaptive algorithms with the pNLMS having similarity with the proportionate normalized least-mean-squares (PNLMS) algorithm. Computer simulations have been conducted using the echo canceller application as an example of a sparse environment. The simulations clearly show the ability of the developed algorithms to exploit the inherent sparsity structure, thereby outperforming conventional algorithms like the NLMS algorithm in this application.

A joint source-channel decoding scheme that improves the performance of conventional channel decoders over erasure channels by exploiting the cross-correlation between successive speech frames is presented. Speech spectrum parameters are quantized using the scheme presented in Subramaniam and Rao (2001). The joint probability density function (PDF) of the spectrum parameters of successive speech frames is modelled using a Gaussian mixture model (GMM). This model is then used to process the channel decoder output over erasure channels. The performance of two decoding strategies, namely, maximum likelihood decoding (ML) and minimum mean squared error decoding (MMSE) is shown to provide significantly better performance than prediction based schemes.

In this paper, we analyze the effects of channel correlation on the performance of maximal ratio combining (MRC) in receive antenna diversity systems. In the analysis, the channel is modelled as flat Rayleigh fading, slowly varying and an arbitrary number of cochannel interferers are assumed to be present. In an interference limited scenario, an exact closed form expression for signal to interference ratio (SIR) distribution is obtained for a dual antenna system. The degradation in the outage probability as a function of the correlation severity is investigated.

Sparse Bayesian learning and specifically relevance vector machines have received much attention as a means of achieving parsimonious representations of signals in the context of regression and classification. We provide a simplified derivation of this paradigm from a Bayesian evidence perspective and apply it to the problem of basis selection from overcomplete dictionaries. Furthermore, we prove that the stable fixed points of the resulting algorithm are necessarily sparse, providing a solid theoretical justification for adapting the methodology to basis selection tasks. We then include simulation studies comparing sparse Bayesian learning with basis pursuit and the more recent FOCUSS class of basis selection algorithms, empirically demonstrating superior performance in terms of average sparsity and success rate of recovering generative bases.

A closed-loop transmit diversity system is evaluated, taking several major feedback non-idealities into account both separately and in combination, contrasting most previous work in the field. The focus of the study is on the trade-off between quantization errors and feedback periodicity (i.e., a trade-off data). Different number of transmit antennas, transmission rates and receiver velocities are investigated. In addition, we also study the impact of feedback delay. Some main results are as follows: for each receiver velocity and feedback channel rate, there exists an optimal choice of quantization resolution, and hence an optimal choice of feedback period. Furthermore, there is an optimum choice of the number of transmit antennas to employ for a given degree of Doppler and a given feedback rate. Finally, the bit error performance for a fixed feedback rate and a given receiver velocity is practically independent of the transmission rate.

We consider flat-fading multiple antenna channels with t transmit and r receive antennas, which is modeled by an r /spl times/ t complex matrix H. The first n eigenvectors of H/sup /spl dagger//H, where 0/spl les/n/spl les/min(t,r) are assumed to be available at the transmitter as partial spatial information of channel. A transmission method was proposed which enables better use of the multiple-input multiple output (MIMO) channels. By using the transmission scheme, the MIMO channel can be decomposed into two parts: n parallel channels and a new small coupled MIMO channel. One reasonable coding strategy is to employ conventional time-domain only code for each of the parallel channels and a space-time code for the small MIMO channel. This paper focuses on deriving the channel capacity of the multiple antenna channels employing such a coding strategy. The results show that the proposed methods lead to systems wherein the amount of feedback information can be significantly reduced with a minor sacrifice of achievable transmission rate.

In this paper, we analyze the outage capacity improvements in multi-cellular code division multiple access (CDMA) systems using receive antenna diversity and fast power control schemes that can track multipath fading. The channel is modelled as slowly varying Rayleigh fading with lognormal shadowing and path loss. Closed form expressions are obtained for the outage capacity of the system by approximating the total intercell and intercell interference distributions. We show that by controlling the power level of a user at the output of the diversity combiner and imposing the condition that the user denies transmission when in deep fade, the average intercell interference level of the system is significantly reduced. As a result of this reduction, we demonstrate both analytically and through simulations that the outage capacity of the system is improved more than the linearity with increasing number of antenna elements.

Recently, relevance vector machines (RVM) have been fashioned from a sparse Bayesian learning (SBL) framework to perform supervised learn- ing using a weight prior that encourages sparsity of representation. The methodology incorporates an additional set of hyperparameters govern- ing the prior, one for each weight, and then adopts a specific approxi- mation to the full marginalization over all weights and hyperparameters. Despite its empirical success however, no rigorous motivation for this particular approximation is currently available. To address this issue, we demonstrate that SBL can be recast as the application of a rigorous vari- ational approximation to the full model by expressing the prior in a dual form. This formulation obviates the necessity of assuming any hyperpri- ors and leads to natural, intuitive explanations of why sparsity is achieved in practice.

Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).

Room reverberation is typically the main obstacle for designing robust microphone-based source localization systems. The purpose of the paper is to analyze the achievable performance of acoustical source localization methods when room reverberation is present. To facilitate the analysis, we apply well known results from room acoustics to develop a simple but useful statistical model for the room transfer function. The properties of the statistical model are found to correlate well with results from real data measurements. The room transfer function model is further applied to analyze the statistical properties of some existing methods for source localization. In this respect we consider especially the asymptotic error variance and the probability of an anomalous estimate. A noteworthy outcome of the analysis is that the so-called PHAT time-delay estimator is shown to be optimal among a class of cross-correlation based time-delay estimators. To verify our results on the error variance and the outlier probability we apply the image method for simulation of the room transfer function.

A computationally efficient, high quality, vector quantization scheme based on a parametric probability density function (PDF) is proposed. In this scheme, the observations are modeled as i.i.d realizations of a multivariate Gaussian mixture density. The mixture model parameters are efficiently estimated using the expectation maximization (EM) algorithm. A low complexity quantization scheme using transform coding and bit allocation techniques which allows for easy mapping from observation to quantized value is developed for both fixed rate and variable rate systems. An attractive feature of this method is that source encoding using the resultant codebook involves very few searches and its computational complexity is minimal and independent of the rate of the system. Furthermore, the proposed scheme is bit scalable and can switch seamlessly between a memoryless quantizer and a quantizer with memory. The usefulness of the approach is demonstrated for speech coding where Gaussian mixture models are used to model speech line spectral frequencies. The performance of the memoryless quantizer is 1-3 bits better than conventional quantization schemes.

Maximal ratio transmission is a transmit beamforming technique where multiple antennas are used both for transmission and reception. It can effectively increase the diversity order without modifying the receiver. In this letter, an expression for the probability of error of maximal ratio transmission is derived for the special case of two receive antennas and multiple transmit antennas.

An iterative receiver with soft-decision feedback is derived by using the expectation-maximization algorithm for maximum a posteriori estimate of fast Rayleigh flat fading channels. Simulation results indicate that in a fast fading environment, the derived receiver can perform better than an iterative receiver with hard-decision feedback.

In this paper, we examine the impact of channel estimation errors on Maximum Ratio Combining (MRC) in receive diversity systems, in the presence of co-channel interferers. The channel is modelled as flat Rayleigh fading, slowly varying and spatially independent. It is assumed that the spatial combiner weights are imperfect estimates of the desired user's fading coefficients and complex Gaussian distributed. Closed form expressions for signal to interference plus noise ratio (SINR) distribution and outage probability is obtained. Using these expressions, the effect of channel estimation quality on performance is investigated.

The main obstacle for designing robust microphone-based source localization systems is the effects of room reverberation. Utilizing results from statistical room acoustics, we analyze the performance of GCC-based methods for time-delay estimation. An interesting outcome of the analysis is that the so-called PHAT time-delay estimator is shown to be optimal among a class of cross-correlation based time-delay estimators.

A low complexity vector quantization scheme for the recursive coding of speech spectrum parameters is proposed. In this scheme, the joint probability density function (pdf) of the spectrum parameters of successive speech frames is modelled using a Gaussian mixture model whose parameters are estimated using the Expectation Maximization (EM) algorithm. The conditional density of the spectrum parameters of the current speech frame based on the quantized values of the spectrum parameters of previous speech frames is used to generate a new codebook for every current speech frame. An efficient quantization scheme using transform coding and bit allocation techniques which allows easy and computationally efficient mapping from observation to quantized value is developed. Transparent quality speech for the first order fixed rate recursive coding case is achieved at 20 bits per frame.

In this paper, we develop a Kalman channel predictor based receiver for DS-CDMA systems. In particular, we develop and analyze the performance of a minimum-output-energy (MOE) receiver with short spreading codes, when fast Rayleigh flat fading channels are predicted by the Kalman filter. For comparison, we also derive the performance of a matched filter receiver with long spreading codes. It is shown by analysis and simulation that in a fast fading environment, MOE with short spreading codes is better than matched filter with long spreading codes.

A sparse representation of a signal, i.e., a representation using a small number of vectors chosen from a dictionary of vectors, is highly desirable in many applications. Here, we extend the backward elimination sparse representation algorithm presented by Reeves (see ibid., vol.6, p.266-68, Oct. 1999) to allow for an overcomplete dictionary and develop recursions for its implementation. In the overcomplete case, the representation error cannot be used as a general criterion for elimination of a dictionary vector and other criteria must be considered. Simulation on a test-case dictionary shows that the performance of the proposed algorithm can improve upon that of forward selection methods.

Channels with a sparse impulse response arise in a number of communication applications. Exploiting the sparsity of the channel, we show how an estimate of the channel may be obtained using a matching pursuit (MP) algorithm. This estimate is compared to thresholded variants of the least squares (LS) channel estimate. Among these sparse channel estimates, the MP estimate is computationally much simpler to implement and a shorter training sequence is required to form an accurate channel estimate leading to greater information throughput.

Matching pursuit (MP) uses a greedy search to construct a subset of vectors, from a larger set, which best represents a signal of interest. We extend this search for the best subset by keeping the K vectors which maximize the selection criterion at each iteration. This is termed the MP:K algorithm and represents a suboptimal search through the tree of all possible subsets where each node is limited to having K children. As a more suboptimal search, we can use the M-L search to select a subset of dictionary vectors, leading to the MP:M-L algorithm. We compare the computation and storage requirements for three variants of the MP algorithm using these searches. Through simulations, the significantly improved performance obtained using the MP:K and MP:M-L algorithms is demonstrated. We conclude that the modified matching pursuit (MMP) algorithm offers the best compromise between performance and complexity using these search techniques.

Describes a robust feature extraction method for continuous speech recognition. Central to the method is the minimum variance distortionless response (MVDR) method of spectrum estimation and a feature trajectory smoothing technique for reducing the variance in the feature vectors. The above method, when evaluated on continuous speech recognition tasks in a stationary and moving car, gave an average relative improvement in WER of greater than 30%.

We examine the statistical properties of the output signal to interference plus noise ratio (SINR) of a spatial combiner where the spatial weights used are either the Maximal Ratio Combiner (MRC) weights or the Optimal Combining (Wiener Hopf) weights. The channels are modeled as slow flat Rayleigh fading channels and multiple interferers are assumed present. In particular, the modified F-distribution is introduced to provide an exact characterization of a MRC receiver and to bound the performance of the OC receiver. Simulation results are provided to support the analytical results and to provide insight.

A computationally efficient, high quality, vector quantization scheme based on a parametric probability density function (PDF) is proposed. In this scheme, speech line spectral frequencies (LSF) are modeled as i.i.d realizations of a multivariate Gaussian mixture density. The mixture model parameters are efficiently estimated using the expectation maximization (EM) algorithm. An efficient quantization scheme using transform coding and bit allocation techniques which allows for easy and computationally efficient mapping from observation to quantized value is developed for both fixed rate and variable rate systems. An attractive feature of this method is that source encoding using the resultant codebook involves very few searches and its computational complexity is minimal and independent of the rate of the system. Furthermore, the proposed scheme is bit scalable and can switch between memoryless and quantizer with memory seamlessly. The performance of the memoryless quantizer is 2-3 bits better than conventional quantization schemes.

We develop an approach for using an antenna array for tracking fast Rayleigh flat fading channels and suppressing cochannel interference. The fast flat fading process is assumed to be a general autoregressive (AR) process in order to characterize temporal variation of channels and evaluate its effect on the receiver structure and performance. The optimal array receiver structure that minimizes the probability of error for BPSK signals is derived, which includes a Kalman filter to predict the fading channels. A simple integral expression for the probability of error is also derived for the optimal receiver. In particular, we analyze the case with identical shaping filters. An irreducible probability of error is shown to exist due to the prediction error of multiple channels. Another interesting observation from the study is that the diversity gain with m antenna elements in the presence of k interferences is usually greater than (m-k), even in the presence of channel prediction error. Simulations are carried out to verify the theoretical analysis.

In this paper, equalization of channels with large delay spread but with few nonzero taps, such as those encountered in HDTV, are considered. Exploiting the sparse nature of the channel impulse response, we propose a decision feedback equalizer (DFE) which is based on a channel estimate obtained via a matching pursuit (MP) algorithm rather than the standard least squares (LS) channel estimate. It is shown that equalizers so designed result in a large performance gain. It is also shown that while truncation of the LS channel estimate to give a sparse channel estimate leads to some performance improvement, it falls well short of the MP-based DFE. In addition, an adaptive scheme which is intermediate in complexity between the LMS and RLS is presented for updating the sparse channel estimate in a time-varying environment. This adaptive algorithm outperforms the LMS.

We present all-pole models based upon the minimum variance distortionless response (MVDR) spectrum for spectral modeling of speech. The MVDR method, which is popular in array processing, provides all-pole spectra that are robust for modeling both voiced and unvoiced speech. Although linear prediction (LP) is a popular method for obtaining all-pole model parameters, LP spectral envelopes overestimate and overemphasize the medium and high pitch voiced speech spectral powers, thereby featuring unwanted sharp contours, and do not improve in spectral envelope modeling performance as the filter order is increased. In contrast, the MVDR all-pole spectrum which can be easily obtained from the LP coefficients, features improved spectral envelope modeling as the filter order is increased. In particular, the high order MVDR spectrum models voiced speech spectra very well, particularly at the perceptually important harmonics, and features a smooth contoured envelope. Furthermore, the MVDR spectrum can be based upon either conventional time domain correlation estimates or upon spectral samples, a task that is common in frequency domain speech coding. In particular, the MVDR spectrum of sufficient order provides an all-pole envelope that models a set of spectral samples exactly. In addition, the MVDR all-pole spectrum is also suitable for modeling unvoiced speech spectra.

Sparse solutions to the linear inverse problem Ax=Y and the determination of an environmentally adapted overcomplete dictionary (the columns of A) depend upon the choice of a "regularizing function" d(x) in several previously proposed procedures. We discuss the interpretation of d(x) within a Bayesian framework, and the desirable properties that "good" (i.e., sparsity ensuring) regularizing functions, d(x) might have. These properties are: Schur-concavity (d(x) is consistent with majorization); concavity (d(x) has sparse minima); parameterizability (d(x) is drawn from a large, parameterizable class); and factorizability of the gradient of d(x) in a certain manner. The last property (which naturally leads one to consider separable regularizing functions) allows d(x) to be efficiently minimized subject to Ax=Y using an affine scaling transformation (AST)-like algorithm "adapted" to the choice of d(x). A Bayesian framework allows the algorithm to be interpreted as an independent component analysis (ICA) procedure.

We present several analytical properties of minimum variance distortionless response (MVDR) based all-pole models that demonstrate the advantages and usefulness of these models for speech spectral coding. In particular, we show that a sufficient order MVDR all-pole model provides a spectral envelope that fits a set of spectral samples exactly with a parameterization convenient for quantization purposes. In addition, we show that MVDR all-pole filters provide a monotonically decreasing spectral distortion with increasing filter order. Furthermore, we show that the MVDR all-pole filter possesses the flexibility to be obtained from correlations based upon either spectral samples or conventional time-domain correlations. Finally, exploiting the insight gained from MVDR modeling, we introduce a novel class of constrained all-pole models for efficient spectral coding. In this approach, a subset of the line spectral frequency (LSF) parameters associated with the all-pole model are judiciously fixed, leading to a simpler model parameterization.

This paper focuses on on-line learning procedures for locally recurrent neural networks with emphasis on multilayer perceptron (MLP) with infinite impulse response (IIR) synapses and its variations which include generalized output and activation feedback multilayer networks (MLN's). We propose a new gradient-based procedure called recursive backpropagation (RBP) whose on-line version, causal recursive backpropagation (CRBP), presents some advantages with respect to the other on-line training methods. The new CRBP algorithm includes as particular cases backpropagation (BP), temporal backpropagation (TBP), backpropagation for sequences (BPS), Back-Tsoi algorithm among others, thereby providing a unifying view on gradient calculation techniques for recurrent networks with local feedback. The only learning method that has been proposed for locally recurrent networks with no architectural restriction is the one by Back and Tsoi. The proposed algorithm has better stability and higher speed of convergence with respect to the Back-Tsoi algorithm, which is supported by the theoretical development and confirmed by simulations. The computational complexity of the CRBP is comparable with that of the Back-Tsoi algorithm, e.g., less that a factor of 1.5 for usual architectures and parameter settings. The superior performance of the new algorithm, however, easily justifies this small increase in computational burden. In addition, the general paradigms of truncated BPTT and RTRL are applied to networks with local feedback and compared with the new CRBP method. The simulations show that CRBP exhibits similar performances and the detailed analysis of complexity reveals that CRBP is much simpler and easier to implement, e.g., CRBP is local in space and in time while RTRL is not local in space.

A general framework based on majorization, Schur-concavity, and concavity is given that facilitates the analysis of algorithm performance and clarifies the relationships between existing proposed diversity measures useful for best basis selection. Admissible sparsity measures are given by the Schur-concave functions, which are the class of functions consistent with the partial ordering on vectors known as majorization. Concave functions form an important subclass of the Schur-concave functions which attain their minima at sparse solutions to the basis selection problem. Based on a particular functional factorization of the gradient, we give a general affine scaling optimization algorithm that converges to a sparse solution for measures chosen from within this subclass.

In this paper, we propose some new modeling techniques that provide a more synergistic approach to multistage time-domain speech compression. In particular, we propose a new error criterion for determining all-pole filters, and a unique method for jointly coding the pulse information in excitation vectors. The new error criterion for determining all-pole filters is based upon minimizing the sum of the residual signal's absolute values raised to a power less than one. It is shown to be a desirable cost function for yielding residual signals that are more sparse, and consequently better suited for multistage compression than linear prediction residuals. Statistical reasons supporting the new criterion are also provided. Furthermore, exploiting the properties of, and the relationship between, the linear prediction and minimum variance spectra, we propose a novel parameter set for jointly coding the excitation vector's pulse position, sign, and gain information.

An overview is given of the role of the sparseness constraint in signal processing problems. It is shown that this is a fundamental problem deserving of attention. This is illustrated by describing several applications where sparseness of solution is desired. Lastly, a review is given of the algorithms that are currently available for computing sparse solutions.

In this paper we propose the MVDR method, which is based upon the minimum variance distortionless response (MVDR) spectrum estimation method, for modeling voiced speech. Developed to overcome some of the shortcomings of linear prediction models, the MVDR method provides better models for medium and high pitch voiced speech. The MVDR model is an all-pole model whose spectrum is easily obtained from a modest non-iterative computation involving the linear prediction coefficients thereby retaining some of the computational attractiveness of LPC methods. With the proper choice of filter order, which is dependent on the number of harmonics, the MVDR spectrum models the formants and spectral powers of voiced speech exactly. An efficient reduced model order MVDR method is developed to further enhance its applicability. An extension of the reduced order MVDR method for recovering the correct amplitudes of the harmonics of voiced speech is also presented.

This paper introduces noncausal all-pole models that are capable of efficiently capturing both the magnitude and phase information of voiced speech. It is shown that noncausal all-pole filter models are better able to match both magnitude and phase information and are particularly appropriate for voiced speech due to the nature of the glottal excitation. By modeling speech in the frequency domain, the standard difficulties that occur when using noncausal all-pole filters are avoided. Several algorithms for determining the model parameters based on frequency-domain information and the masking effects of the ear are described. Our work suggests that high-quality voiced speech can be produced using a 14th-order noncausal all-pole model.

This paper addresses the design and development of a software environment for experimenting with filter banks, wavelets, and wavelet packets. An object-oriented class library, called SurfWave, is developed that provides a straightforward interface for experimenting and applying wavelet theory. SurfWave uses the power of C++ for efficiency and ease-of-use by isolating the user from distracting software and hardware peculiarities. It can be easily modified to provide new functionality, i.e., support for new cost functions, optimization algorithms, filter bank topologies, etc. SurfWave is available on several platforms (Unix, PC/Windows, Macintosh) and languages (C++, Matlab, CPX), presenting a uniform "point-of-departure" to the widest cross-section of wavelet enthusiasts.

This paper presents affine scaling transformation based methods for finding low complexity sparse solutions to optimization problems. The methods achieve sparse solutions in a more general context, and generalize our earlier work on FOCUSS developed to deal with the underdetermined linear inverse problem. The key result is a theorem which shows a simple condition that a sequence has to satisfy for it to converge to a sparse limiting solution. Three approaches to incorporate this condition into optimization problems are presented. These consist of either imposing the condition as an additional optimization constraint, or suitably modifying the cost function, or using a combination of the two. The benefits of the methodology when applied to the linear inverse problem are twofold. Firstly, it allows for the treatment of the overdetermined problem in addition to the underdetermined problem, and secondly it enables establishing sufficient conditions under which regularized versions of FOCUSS are assured of convergence to sparse solutions.

In this paper a general class of fast learning algorithms for feedforward neural networks is introduced and described. The approach exploits the separability of each layer into linear and nonlinear blocks and consists of two steps. The first step is the descent of the error functional in the space of the outputs of the linear blocks (descent in the neuron space), which can be performed using any preferred optimization strategy. In the second step, each linear block is optimized separately by using a least squares (LS) criterion. To demonstrate the effectiveness of the new approach, a detailed treatment of a gradient descent in the neuron space is conducted. The main properties of this approach are the higher speed of convergence with respect to methods that employ an ordinary gradient descent in the weight space backpropagation (BP), better numerical conditioning, and lower computational cost compared to techniques based on the Hessian matrix. The numerical stability is assured by the use of robust LS linear system solvers, operating directly on the input data of each layer. Experimental results obtained in three problems are described, which confirm the effectiveness of the new method.

This paper presents a class of quadratically weighted distortion measures which provide optimal performance for the high rate vector quantization (VQ) of linear predictive coding (LPC) parameters. It is shown that the quantization distortion of a high rate VQ converges to a quadratically weighted measure, where the quadratic weighting matrix is a "sensitivity" matrix, which is a generalization of the scalar sensitivity concept to the vector case. The sensitivity matrix is the second order term of the Taylor series expansion of the original distortion measure. Closed form expressions and computationally efficient methods for computing the sensitivity matrices of the different LPC parameterizations are given, which involve no numerical integration and can be implemented in real-time on modern DSP chips. In the general case, the "sum of sensitivity weighted scalar errors" is not equivalent to the original distortion measure. However the sensitivity matrix of the line spectral pair (LSP) frequencies is exactly diagonal, demonstrating that for LSPs only a "sum of sensitivity weighted scalar errors" will result in optimal performance.

The performance of text dependent, short utterance speaker verification systems degrades significantly with channel and background artifacts. The authors investigate maximum likelihood and adaptive techniques to compensate for a stationary channel and noise. Maximum likelihood channel and noise compensation was introduced by Cox and Bridle (1989), and has been shown to be effective in many other speech applications. For adaptive estimation, a Bussgang like algorithm is developed which is more suitable for real-time implementation. These techniques are evaluated on a speaker verification system that uses the nearest neighbor metric. The results show that for telephone speech with channel differences, channel compensation can provide substantial performance improvement. For un-cooperative speakers, background compensation resulted in a 35% improvement.

The paper presents a theoretical analysis of high-rate vector quantization (VQ) systems that use suboptimal, mismatched distortion measures, and describes the application of the analysis to the problem of quantizing the linear predictive coding (LPC) parameters in speech coding systems. First, it is shown that in many high-rate VQ systems the quantization distortion approaches a simple quadratically weighted error measure, where the weighting matrix is a "sensitivity matrix" that is an extension of the concept of the scalar sensitivity. The approximate performance of VQ systems that train and quantize using mismatched distortion measures is derived, and is used to construct better distortion measures. Second, these results are used to determine the performance of LPC vector quantizers, as measured by the log spectral distortion (LSD) measure, which have been trained using other error measures, such as mean-squared (MSE) or weighted mean-squared error (WMSE) measures of LEPC parameters, reflection coefficients and transforms thereof, and line spectral pair (LSP) frequencies. Computationally efficient algorithms for computing the sensitivity matrices of these parameters are described. In particular, it is shown that the sensitivity matrix for the LSP frequencies is diagonal, implying that a WMSE measured LSP frequencies converges to the LSD measure in high-rate VQ systems. Experimental results to support the theoretical performance estimates are provided.>

Mixed-phase AR models are introduced for encoding the magnitudes and phases of the harmonics of voiced speech. Motivation for the use of the mixed-phase AR models is given and several cost functions are introduced, forming the basis for algorithms which estimate the model parameters. An efficient algorithm based on a quasi-linear least squares approach is presented, and a more sophisticated algorithm based on the perceptual masking properties of the ear is described. When the algorithms are used to model voiced speech signals using a 14th order mixed-phase model, high quality speech can be produced.>

Various methods for mapping signal processing algorithms into systolic arrays have been developed in the past few years. In this paper, efficient scheduling techniques are developed for the partitioning problem, i.e. problems with size that do not match the array size. In particular, scheduling for the locally parallel-globally sequential (LPGS) technique and the locally sequential-globally parallel (LSGP) technique are developed. The scheduling procedure developed exploits the fact that after LPGS and LSGP partitioning, the locality constraints become modified allowing for more flexibility. The new structure allows the authors to develop a flexible scheduling order for LPGS that is useful in evaluating a trade-off between execution time and size of partitioning buffers. The benefits of the scheduling techniques are illustrated with the help of matrix multiplication and least-squares examples.>

This paper is a tutorial on linear, state space, model-based methods for certain nonlinear estimation problems commonly encountered in signal and data analysis. A prototypical problem that is studied is that of estimating the frequencies of multiple, superimposed sinusoids from a short record of noise-corrupted data. The approach expounded however, is applicable to a vast range of nonlinear signal analysis problems and applications in direction finding and damped sinusoid retrieval are dealt with in some detail. The benefits that result from using a state space description of the signal are highlighted in this paper. It is shown that state space models provide an elegant tool for exposing the structure present in the problem. The approach also allows for robust parameterization of the model with respect to finite precision errors. The robustness of the parameter set is complemented by the availability of numerically robust tools to estimate the parameters. The resulting algorithms are compatible with multiprocessor implementations.>

A systematic approach for the analysis of roundoff noise in floating point digital filters is presented. The analysis is based on a high level model developed to deal with the errors in the inner product operation. The model consists of an efficient procedure to keep track of the addition scheme is used in the inner product, and to compute the statistics of the errors. The tractability of the analysis is demonstrated by deriving general expressions for the roundoff noise of FIR (finite impulse response) and IIR (infinite impulse response) filters. Also, the connection between coefficient sensitivity analysis and roundoff noise analysis is discussed.>

The effect of spatial smoothing on direction of arrival (DOA) estimates obtained using weighted eigen-based state space methods/ESPRIT is analyzed. Expressions for the asymptotic mean-squared error in the estimates of the signal zeros and the DOA estimates, along with some general properties of the estimates are derived. It is shown that proper choice of the subarray length can improve the performance of the method significantly. Based on the asymptotic expressions, an optimum weighting matrix minimizing the mean-squared error in one of the DOA is derived.>

The use of adaptive filters for the enhancement of images is studied. In particular, the enhancement of images where the region of interest has a small spatial extent compared to the noise is considered. A two stage approach for enhancing the desired signal is presented. At each stage, a two dimensional adaptive filter employing the least mean square (LMS) algorithm is used. By properly choosing the adaptation step size, in the first stage the colored noise is whitened, and in the second stage the desired signal is recovered from the white noise. Procedures for selecting the step size are discussed. Computer simulations to support the results are also presented.>

The properties of the LMS (least mean square) FIR (finite impulse response) ALE (adaptive line enhancer) when the input consists of p real sinusoids is examined. Unlike the general input case, it is shown that the convergence and sensitivity properties of this ALE are fairly good. The Wiener-Hopf solution is shown to depend only on the 2p dominant eigenvalues of the covariance matrix. This in turn is used to show that the covergence of the LMS algorithm for the zero initial condition depends only on the dominant 2p eigenvalues. It is also shown that when a filter of length L, where L>>2p, is used, the parameter sensitivity is fairly low. The low parameter sensitivity is traced to the minimum-norm characterization of the solution in the noise-free case. The minimum-norm criterion also turns out to be useful in selecting an appropriate value for the decorrelation delay delta .>

The effect of using a spatially smoothed forward-backward covariance matrix on the performance of noise subspace-based methods like MUSIC and the minimum-norm method for the direction-of-arrival (DOA) estimation problem is analyzed. In particular, asymptotic results for the mean-squared error in the estimates of the signal zeros and the DOA are derived. An important outcome of this analysis is that for MUSIC the error in the signal zeros is shown to exhibit a different trend compared to the error in the DOA estimates, leading to difficulty in interpreting the spatial spectrum. For instance, when smoothing is used, the peaks in the spatial spectrum become sharper, giving the impression of higher resolution, whereas in reality the estimates of the DOA may, in fact, have deteriorated compared with the ones obtained using minimal or no smoothing. With regard to the minimum-norm method, the errors in the signal zeros exhibit the same trend as the DOA estimates causing no such problem. The relative comparison of the methods shows that proper spatial smoothing enables the performance of the minimum-norm method to be made comparable to MUSIC.>

The convergence properties of an adaptive lattice filter using a stochastic gradient algorithm are investigated using differential equations. The mean of the PARCOR coefficients of the adaptive lattice filter is obtained by analyzing an associated ordinary differential equation (ODE). An efficient way to compute the statistics required for the solution of the ODE is presented. An expression for the variance of the PARCOR coefficients is derived from the stochastic differential equation (SDE) associated with the normalized error process. Simulation results are given to support the theoretical results.>

The authors analyze the performance of the minimum-norm method for estimating the direction of arrival (DOA) of plane waves in white noise in the case of a linear equispaced sensor array. They examine the perturbation in the roots of the polynomial formed in the intermediate step of the minimum-norm method. In particular, asymptotic results for the mean-squared error in the estimates of the direction of arrival are derived. Simple closed-form expressions are obtained for the one- and two-source case to get further insight. Computer simulation results that substantiate the analysis are provided and compared to those obtained for Root-MUSIC. It is shown that the relative performance of the two methods is directly dependent on the ratio of their parameter sensitivities.>

The authors suggest a modification of the Toeplitz approximation method for estimating frequencies of multiple sinusoids from covariance measurements. The method constructs a state-feedback matrix following a low-rank approximation of the Toeplitz covariance matrix via singular-value decomposition. Ideally, the eigenvalues of this state-feedback matrix will be on the unit circle in the complex plane, and the angles that they make with the real axis will be equal to the unknown sinusoid frequencies. The modification proposed here exploits this prior knowledge of the modulus of the eigenvalues and guarantees that even in the presence of noise the eigenvalues of the estimated state-feedback matrix will lie on the unit circle.>

The author examines the performance of singular value decomposition (SVD) based methods for estimating the frequencies of multiple sinusoids. The concept of angle between subspaces is used to derive a criterion for determining when SVD-based procedures fail. The signal-to-noise ratio (SNR) at which a method breaks down is termed the threshold SNR of the method. It is shown that existing SVD based methods have a higher threshold SNR than predicted by this criterion. A method that utilizes the singular vectors and directly minimizes the angle between subspaces is developed. The method is shown to have better performance at low SNRs. The procedure lowers the threshold SNR thereby extending the range of SNR for which SVD can be used.>

In this paper, we examine the issue of parameter sensitivity in spectrum estimation. In particular, state space models are considered and robust coordinate systems for spectrum estimation are identified. For the sinusoid problem it is shown that the ideal parameter set involves estimating a unitary matrix and simple procedures to estimate such a matrix are identified. For the damped sinusoid and the general ARMA spectrum estimation problem it is shown that the balanced coordinates are robust and reliable.

In this paper we study the problem of adaptively estimating the Autoregressive Moving Average (ARMA) parameters of a time varying ARMA process using a constant step size Gauss-Newton Algorithm. Using weak convergence theory and the concept of prescaling, it is shown that the "mean" behavior can be described by an ordinary differential equation (ODE). Computer simulations are provided to substantiate the analysis.