Spiking neural networks (SNN) have recently emerged as alternatives to traditional neural networks, owing to its energy efficiency benefits and capacity to capture biological neuronal mechanisms. However, the classic backpropagation algorithm for training traditional networks has been notoriously difficult to apply to SNN due to the hard-thresholding and discontinuities at spike times. Therefore, a large majority of prior work believes exact gradients for SNN w.r.t. their weights do not exist and has focused on approximation methods to produce surrogate gradients. In this paper, (1) by applying the implicit function theorem to SNN at the discrete spike times, we prove that, albeit being non-differentiable in time, SNNs have well-defined gradients w.r.t. their weights, and (2) we propose a novel training algorithm, called forward propagation (FP), that computes exact gradients for SNN. FP exploits the causality structure between the spikes and allows us to parallelize computation forward in time. It can be used with other algorithms that simulate the forward pass, and it also provides insights on why other related algorithms such as Hebbian learning and also recently-proposed surrogate gradient methods may perform well.

In this paper, we study the problem of user activity detection and large-scale fading coefficient estimation in a random access wireless uplink with a massive MIMO base station with a large number M of antennas and a large number of wireless single-antenna devices (users). We consider a block fading channel model where the M-dimensional channel vector of each user remains constant over a coherence block containing L signal dimensions in time-frequency. In the considered setting, the number of potential users Ktotis much larger than L but at each time slot only Katotof them are active. Previous results, based on compressed sensing, require that Ka≤ L, which is a bottleneck in massive deployment scenarios. In this work, we show that such limitation can be overcome when the number of base station antennas M is sufficiently large. More specifically, we prove that with a coherence block of dimension L and a number of antennas M such that Ka/M = o(1), one can identify Ka= O(L2/log2(Ktot/ Ka)) active users, which is much larger than the previously known bounds. We also provide two algorithms. One is based on Non-Negative Least-Squares, for which the above scaling result can be rigorously proved. The other consists of a low-complexity iterative componentwise minimization of the likelihood function of the underlying problem. While for this algorithm a rigorous proof cannot be given, we analyze a constrained version of the Maximum Likelihood (ML) problem (a combinatorial optimization with exponential complexity) and find the same fundamental scaling law for the number of identifiable users. Therefore, we conjecture that the low-complexity (approximated) ML algorithm also achieves the same scaling law and we demonstrate its performance by simulation. We also compare the discussed methods with the (Bayesian) MMV-AMP algorithm, recently proposed for the same setting, and show superior performance and better numerical stability. Finally, we use the discussed approximated ML algorithm as the inner decoder in a concatenated coding scheme for unsourced random access, a grant-free uncoordinated multiple access scheme where all users make use of the same codebook, and the receiver must produce the list of transmitted messages, irrespectively of the identity of the transmitters. We show that reliable communication is possible at any Eb/N0provided that a sufficiently large number of base station antennas is used, and that a sum spectral efficiency in the order ofO(Llog(L)) is achievable.

We study the problem of maximum-likelihood (ML) estimation of an approximate common eigenstructure, i.e. an approximate common eigenvectors set (CES), for an ensemble of covariance matrices given a collection of their associated i.i. d vector realizations. This problem has a direct application in multi-user MIMO communications, where the base station (BS) has access to instantaneous user channel vectors through pilot transmission and attempts to perform joint multi-user Downlink (DL) precoding. It is widely accepted that an efficient implementation of this task hinges upon an appropriate design of a set of common “virtual beams” that captures the common eigenstructure among the user channel covariances. In this paper, we propose a novel method for obtaining this common eigenstructure by casting it as an ML estimation problem. We prove that in the special case where the covariances are jointly diagonalizable, the global optimal solution of the proposed ML problem coincides with the common eigenstructure. Then we propose a projected gradient descent (PGD) method to solve the ML optimization problem over the manifold of unitary matrices and prove its convergence to a stationary point. Through exhaustive simulations, we illustrate that in the case of jointly diagonalizable covariances, our proposed method converges to the exact CES. Also, in the general case where the covariances are not jointly diagonalizable, it yields a solution that approximately diagonalizes all covariances. Besides, the empirical results show that our proposed method outperforms the well-known joint approximate diagonalization of eigenmatrices (JADE) method in the literature.

In spatial channel models used in multi-antenna wireless communications, the propagation from a single-antenna transmitter (e.g. a user) to an M-antenna receiver (e.g. a Base Station) occurs through scattering clusters located in the far field of the receiving array. The angular power spread function (APSF) of the corresponding M-dim channel vector describes the angular density of the received signal power at the array. In many applications, such as channel sounding and Uplink Downlink covariance transformation in FDD systems, estimating the APSF is required either implicitly or explicitly. However, the existing literature on the subject has mainly focused on channel covariance estimation from a set of noisy pilot observations. It is also assumed that the APSF consists only of discrete components corresponding to Line-of-Sight (LoS) paths and specular scattering. It turns out that while covariance estimation is a well-posed problem, APSF estimation is a much harder task and is in general ill-posed. The reason is that the propagation environment can also include diffuse scattering elements, resulting in continuous APSF components. Therefore, the APSF is a function belonging to the infinite-dimensional space of nonnegative measures over the angle domain. In this paper, we show that under a geometrically-consistent, group-sparse structure on the APSF, which is prevalent in massive MIMO channels, one is able to estimate the APSF properly. We propose an algorithm based on deep neural networks (DNNs) that learns this structure and yields precise APSF estimates, even when the number of available pilot observations is relatively small. We empirically show that our proposed method outperforms the state-of-the-art method in various performance metrics.

Obtaining channel covariance knowledge is of great importance in various Multiple-Input Multiple-Output MIMO communication applications, including channel estimation and user grouping. Considering recently proposed massive MIMO systems, covariance estimation proves to be challenging due to the large number of antennas (M >> 1) employed in the base station. In this case, the number of pilot transmissions N becomes comparable to the number of antennas and standard estimators, such as the sample covariance, yield a poor estimate of the true covariance and are hence undesirable. In this paper, we propose a Maximum-Likelihood (ML) massive MIMO covariance estimator, based on a parametric representation of the channel angular spread function (ASF). The parametric representation emerges from super-resolving discrete ASF components plus approximating its continuous components using carefully chosen limited-support density function. We maximize the likelihood function using a Concave-Convex procedure, which is initialized via a non-negative least-squares optimization problem. Our simulation results show that the proposed method outperforms the state of the art in various estimation quality metrics.

We propose a novel method for massive multiple-input multiple-output (massive MIMO) in frequency division duplexing (FDD) systems. Due to the large frequency separation between uplink (UL) and downlink (DL) in FDD systems, channel reciprocity does not hold. Hence, in order to provide DL channel state information to the base station (BS), closed-loop DL channel probing, and channel state information (CSI) feedback is needed. In massive MIMO, this typically incurs a large training overhead. For example, in a typical configuration with M ≃200 BS antennas and fading coherence block of T ≃ 200 symbols, the resulting rate penalty factor due to the DL training overhead, given by max{0, 1 - M/T }, is close to 0. To reduce this overhead, we build upon the well-known fact that the angular scattering function of the user channels is invariant over frequency intervals whose size is small with respect to the carrier frequency (as in current FDD cellular standards). This allows us to estimate the users' DL channel covariance matrix from UL pilots without additional overhead. Based on this covariance information, we propose a novel sparsifying precoder in order to maximize the rank of the effective sparsified channel matrix subject to the condition that each effective user channel has sparsity not larger than some desired DL pilot dimension Tdl, resulting in the DL training overhead factor max{0, 1 - Tdl/T } and CSI feedback cost of Tdl pilot measurements. The optimization of the sparsifying precoder is formulated as a mixed integer linear program, that can be efficiently solved. Extensive simulation results demonstrate the superiority of the proposed approach with respect to the concurrent state-of-the-art schemes based on compressed sensing or UL/DL dictionary learning.

Communication at millimeter wave (mm-wave) bands is expected to become a key ingredient of the next generation (5G) wireless networks. Effective mm-wave communications require fast and reliable methods for beamforming at both the user equipment (UE) and the base station sides, in order to achieve a sufficiently large signal-to-noise ratio after beamforming. We refer to the problem of finding a pair of strongly coupled narrow beams at the transmitter and receiver as the beam alignment problem. In this paper, we propose an efficient BA scheme for single-carrier mm-wave communications. In the proposed scheme, the BS periodically probes the channel in the downlink via a pre-specified pseudo-random beamforming codebook and pseudo-random spreading codes, letting each UE estimate the angle-of-arrival/angle-of-departure (AoA-AoD) pair of the multipath channel for which the energy transfer is maximum. We leverage the sparse nature of mm-wave channels in the AoA-AoD domain to formulate the BA problem as the estimation of a sparse non-negative vector. Based on the recently developed non-negative least squares technique, we efficiently find the strongest AoA-AoD pair connecting each UE to the BS. We evaluate the performance of the proposed scheme under a realistic channel model, where the propagation channel consists of a few multipath components each having different delays, AoAs-AoDs, and Doppler shifts. The channel model parameters are consistent with the experimental channel measurements. The simulation results indicate that the proposed method is highly robust to fast channel variations caused by the large Doppler spread between the multipath components. Furthermore, we also show that after achieving BA, the beamformed channel is essentially frequency-flat, such that single-carrier communication needs no equalization in the time domain.

In this paper, we propose a novel method for efficient implementation of a massive Multiple-Input Multiple- Output (massive MIMO) system with Frequency Division Duplexing (FDD) operation. Our main objective is to reduce the large overhead incurred by Downlink (DL) common training and Uplink (UL) feedback needed to obtain channel state information (CSI) at the base station. Our proposed scheme relies on the fact that the underlying angular distribution of a channel vector, also known as the angular scattering function, is a frequency-invariant entity yielding a ULDL reciprocity and has a limited angular support. We estimate this support from UL CSI and interpolate it to obtain the corresponding angular support of the DL channel. Finally we exploit the estimated support of the DL channel of all the users to design an efficient channel probing and feedback scheme that maximizes the total spectral efficiency of the system. Our method is different from the existing compressed-sensing (CS) based techniques in the literature. Using support information helps reduce the feedback overhead from O(s logM) in CS techniques to O(s) in our proposed method, with s andM being sparsity order of the channel vectors and the number of base station antennas, respectively. Furthermore, in order to control the channel sparsity and therefore the DL common training and UL feedback overhead, we introduce the novel concept of active channel sparsification. In brief, when the fixed pilot dimension is less than the required amount for reliable channel estimation, we introduce a pre-beamforming matrix that artificially reduces the effective channel dimension of each user to be not larger than the DL pilot dimension, while maximizing both the number of served users and the number of probed angles. We provide numerical experiments to assess the performance of our method and compare it with the state-of-the-art CS technique.

Millimeter-Wave (mmWave) communication has come into the spotlight as an enabling approach for next generation wireless networks. Communication at mmWave is however challenging due to the large path loss and limited power. This implies that antenna arrays with large directional gain are required both at the Base Station (BS) and the user sides. Finding the strongest narrow beam pair connecting the BS and the user is referred to as Beam Alignment (BA). In this paper, we propose an efficient BA scheme for multi-user systems via estimating the second order statistics of the channel. In the proposed scheme, the BS probes the channel in the downlink letting each user estimate its own channel, where all the users within the BS coverage are trained simultaneously. We formulate the channel estimation at the user side as a Compressed Sensing (CS) of a non- negative sparse vector and use the recently developed Non- Negative Least Squares (NNLS) technique to solve it efficiently. We evaluate our method via numerical simulations and compare it with other competitive algorithms. It has been verified that the proposed approach incurs less training overhead, exhibits higher efficiency in multi-user scenarios, and is highly robust to fast time- varying channels.

The typical approach for recovery of spatially correlated signals is regularized least squares with a coupled regularization term. In the Bayesian framework, this algorithm is seen as a maximum-a-posterior estimator whose postulated prior is proportional to the regularization term. In this paper, we study distributed sensing networks in which a set of spatially correlated signals are measured individually at separate terminals, but recovered jointly via a generic maximum-a-posterior estimator. Using the replica method, it is shown that the setting exhibits the decoupling property. For the case with jointly sparse signals, we invoke Bayesian inference and propose the “multi-dimensional soft thresholding” algorithm which is posed as a linear programming. Our investigations depict that the proposed algorithm outperforms the conventional l2,1-norm regularized least squares scheme while enjoying a feasible computational complexity.

In this paper, we study the problem of activity detection (AD) in a massive MIMO setup, where the Base Station (BS) has M ≫ 1 antennas. We consider a block fading channel model where the M-dim channel vector of each user remains almost constant over a coherence block (CB) containing Dc signal dimensions. We study a setting in which the number of potential users Kcassigned to a specific CB is much larger than the dimension of the CB Dc(Kc≫ Dc) but at each time slot only Ac≪ Kcof them are active. Most of the previous results, based on compressed sensing, require that Ac≤ Dc, which is a bottleneck in massive deployment scenarios such as Internet-of-Things (IoT) and Device-to-Device (D2D) communication. In this paper, we show that one can overcome this fundamental limitation when the number of BS antennas M is sufficiently large. More specifically, we derive a scaling law on the parameters (M, Dc, Kc, Ac) and also Signal-to-Noise Ratio (SNR) under which our proposed AD scheme succeeds. Our analysis indicates that with a CB of dimension Dc, and a sufficient number of BS antennas M with Ac/M=o(1), one can identify the activity of Ac=O(Dc2/log2((Kc)/(Ac))) active users, which is much larger than the previous bound Ac=O(Dc) obtained via traditional compressed sensing techniques. In particular, in our proposed scheme one needs to pay only a poly-logarithmic penalty O(log2((Kc)/(Ac))) for increasing the number of potential users Kc, which makes it ideally suited for AD in IoT setups. We propose low-complexity algorithms for AD and provide numerical simulations to illustrate our results.

In this paper, we study the problem of multiband (frequency-variant) covariance interpolation with a particular emphasis towards massive MIMO applications. In massive MIMO, the communication between each Base Station (BS) with M ≫ 1 antennas and each single-antenna user occurs through a collection of scatterers in the environment, where the channel vector of each user at BS antennas consists in a weighted linear combination of the array responses of the scatterers, where each scatterer has its own angle of arrival (AoA) and complex channel gain. The array response at a given AoA depends on the wavelength of the incoming planar wave and is naturally frequency dependent. While in typical wireless communication applications the signal bandwidth is narrow enough, such that the channel second-order statistics (notably, the channel covariance matrix) can be considered frequency independent, in many other applications such as Frequency Division Duplexing (FDD) the uplink (UL) and the downlink (DL) channels are separated by a large frequency interval, such that the dependence of the channel covariance on frequency cannot be ignored. In this paper, we show that although this dependence is generally negligible for a small number of antennas M, it results in a considerable distortion of the covariance matrix when M → ∞. Moreover, we prove that this frequency-dependent distortion can be fully compensated by a suitable covariance interpolation in frequency. We analyze the covariance interpolation problem mathematically and prove its stability under a very mild reciprocity condition on the angular power spread function (PSF) of the users. We also investigate the validity of our results using numerical simulations.

We study the problem of solving a linear sensing system when the observations are unlabeled. Specifically we seek a solution to a linear system of equations y = Ax when the order of the observations in the vector y is unknown. Focusing on the setting in which A is a random matrix with i.i.d. entries, we show that if the sensing matrix A admits an oversampling ratio of 2 or higher, then, with probability 1, it is possible to recover x exactly without the knowledge of the order of the observations in y. Furthermore, if x is of dimension K, then any 2K entries of y are sufficient to recover x. This result implies the existence of deterministic unlabeled sensing matrices with an oversampling factor of 2 that admit perfect reconstruction. The result is universal in that conditioned on the realization of matrix A, recovery is guaranteed for all possible choices of x. While the proof is constructive, it uses a combinatorial algorithm which is not practical, leaving the question of complexity open. We also analyze a noisy version of the problem and show that local stability is guaranteed by the solution. In particular, for every x, the recovery error tends to zero as the signal-to-noise ratio tends to infinity. The question of universal stability is unclear. In addition, we obtain a converse of the result in the noiseless case: If the number of observations in y is less than 2K, then with probability 1, universal recovery fails, i.e., with probability 1, there exist distinct choices of x which lead to the same unordered list of observations in y. We also present extensions of the result of the noiseless case to special cases with non-i.i.d. entries in A, and to a different setting in which the labels of a portion of the observations y are known. In terms of applications, the unlabeled sensing problem is related to data association problems encountered in different domains including robotics where it is appears in a method called “simultaneous localization and mapping”, multi-target tracking applications, and in sampling signals in the presence of jitter.

Massive MIMO is a variant of multi-user MIMO in which the number of antennas at the base station (BS) M is very large and typically much larger than the number of served users (data streams) K. Recent research has widely investigated the system-level advantages of the massive MIMO, and in particular, the beneficial effect of increasing the number of antennas M. These benefits, however, come at the cost of a dramatic increase in hardware and computational complexity. This is partly due to the fact that the BS needs to compute precoding/receiving vectors in order to coherently transmit/detect data to/from each user, where the resulting complexity grows proportionally to the number of antennas M and the number of served users K. Recently, different algorithms based on tools from asymptotic random matrix theory and/or approximated message passing have been proposed to reduce such complexity. The underlying assumption in all these techniques, however, is that the exact statistics (covariance matrix) of the channel vectors of the users is a priori known. This is far from being realistic, especially taking into account that, in the high-dim regime of M ≫ 1, estimating the channel covariance matrices of the users is also challenging in terms of both computation and storage requirements. In this paper, we propose a novel technique for computing the precoder/detector in a massive MIMO system. Our method is based on the randomized Kaczmarz algorithm and does not require a priori knowledge of the statistics of users' channel vectors. We analyze the performance of our proposed algorithm theoretically and compare its performance with that of other techniques based on random matrix theory and approximate message passing via numerical simulations. Our results indicate that our proposed technique is computationally very competitive and yields quite a comparable performance while it does not require the knowledge of the statistics of users' channel vectors.

Millimeter-wave (mm-wave) frequency bands provide an opportunity for much wider channel bandwidth compared with the traditional sub-6-GHz band. Communication at mm-waves is, however, quite challenging due to the severe propagation pathloss incurred by conventional isotropic antennas. To cope with this problem, directional beamforming both at the base station (BS) side and at the user equipment (UE) side is necessary in order to establish a strong path conveying enough signal power. Finding such beamforming directions is referred to as beam alignment (BA). This paper presents a new scheme for efficient BA. Our scheme finds a strong propagation path identified by an angle-of-arrival (AoA) and angle-of-departure (AoD) pair, by exploring the AoA-AoD domain through pseudo-random multi-finger beam patterns and constructing an estimate of the resulting second-order statistics (namely, the average received power for each pseudo-random beam configuration). The resulting under-determined system of equations is efficiently solved using non-negative constrained least-squares, yielding naturally a sparse non-negative vector solution whose maximum component identifies the optimal path. As a result, our scheme is highly robust to variations of the channel time dynamics compared with alternative concurrent approaches based on the estimation of the instantaneous channel coefficients, rather than of their second-order statistics. In the proposed scheme, the BS probes the channel in the downlink and trains simultaneously an arbitrarily large number of UEs. Thus, “beam refinement,” with multiple interactive rounds of downlink/uplink transmissions, is not needed. This results in a scalable BA protocol, where the protocol overhead is virtually independent of the number of UEs, since all the UEs run the BA procedure at the same time. Extensive simulation results illustrate that our approach is superior to the state-of-the-art BA schemes proposed in the literature in terms of training overhead in multi-user scenarios and robustness to variations in the channel dynamics.

Massive MIMO is a variant of multiuser MIMO, in which the number of antennas M at the base-station is very large and generally much larger than the number of spatially multiplexed data streams to the users. It turns out that by increasing the number of antennas M at the base-station and as a result increasing the spatial resolution of the array, although the received signal from each user tends to be very high-dim, it lies on a low-dim subspace due to the limited angular spread of the user. This low-dim subspace structure can be exploited to improve estimation of the channel state during the training period. For example, channel vectors of the users can be estimated by sampling only a small subset rather than the whole number of antenna elements, which reduces the number of required RF chains and A/D converters at receiver front end. Moreover, the subspace information can be used to group the users based on the similarity of their subspaces in order to serve them more efficiently. Thus, it is apparent that estimating the signal subspace of the users from low-dim noisy sketches of their channel vectors plays a crucial role in massive MIMO. In this paper, we aim to design such a subspace estimation/tracking algorithm. Our proposed algorithm requires sampling only a small number of antennas in each training period, has a very low computational complexity, and is able to track the sharp transitions in the channel statistics very quickly.

In this paper, we study the recovery of a signal from a collection of unlabeled and possibly noisy measurements via a measurement matrix with random i.i.d. Gaussian components. We call the measurements unlabeled since their order is missing, namely, it is not known a priori which elements of the resulting measurements correspond to which row of the measurement matrix. We focus on the special case of ordered measurements, where only a subset of the measurements is kept and the order of the taken measurements is preserved. We identify a duality between this problem and the traditional Compressed Sensing, where we show that the unknown support (location of the nonzero elements) of a sparse signal in Compressed Sensing corresponds in a natural way to the unknown location of the measurements kept in unlabeled sensing. While in Compressed Sensing it is possible to recover a sparse signal from an under-determined set of linear equations (less equations than the dimension of the signal), successful recovery in unlabeled sensing requires taking more samples than the dimension of the signal. We develop a low-complexity alternating minimization algorithm to recover the target signal from the set of its unlabeled samples. We also study the behavior of the proposed algorithm for different signal dimensions and number of measurements empirically via numerical simulations. The results are a reminiscent of the phasetransition similar to that occurring in Compressed Sensing.

In this paper, we study the prediction of a circularly symmetric zero-mean stationary Gaussian process from a window of observations consisting of finitely many samples. This is a prevalent problem in a wide range of applications in communication theory and signal processing. Due to stationarity, when the autocorrelation function or equivalently the power spectral density (PSD) of the process is available, the Minimum Mean Squared Error (MMSE) predictor is readily obtained. In particular, it is given by a linear operator that depends on autocorrelation of the process as well as the noise power in the observed samples. The prediction becomes, however, quite challenging when the PSD of the process is unknown. In this paper, we propose a blind predictor that does not require the a priori knowledge of the PSD of the process and compare its performance with that of an MMSE predictor that has a full knowledge of the PSD. To design such a blind predictor, we use the random spectral representation of a stationary Gaussian process. We apply the well-known atomic-norm minimization technique to the observed samples to obtain a discrete quantization of the underlying random spectrum, which we use to predict the process. Our simulation results show that this estimator has a good performance comparable with that of the MMSE estimator.

In this paper, we show that the Hadamard matrix acts as an extractor over the reals of the Rényi Information Dimension (RID), in an analogous way to how it acts as an extractor of the discrete entropy over finite fields. More precisely, we prove that the RID of an i.i.d. sequence of mixture random variables polarizes to the extremal values of 0 and 1 (corresponding to discrete and continuous distributions) when transformed by a Hadamard matrix. Furthermore, we prove that the polarization pattern of the RID admits a closed form expression and follows exactly the Binary Erasure Channel (BEC) polarization pattern in the discrete setting. We discuss the applications of the RID polarization to Compressed Sensing of i.i.d. sources. In particular, we use the RID polarization to construct a family of deterministic ±1-valued sensing matrices for Compressed Sensing. We run numerical simulations to compare the performance of the resulting matrices with that of the random Gaussian and the random Hadamard matrices. The results indicate that the proposed matrices afford competitive performances, while being explicitly constructed.

We consider a massive MIMO system based on time division duplexing (TDD) and channel reciprocity, where the base stations (BSs) learn the channel vectors of their users via the pilots transmitted by the users in the uplink (UL). It is well-known that, in the limit of very large number of BS antennas, the system performance is limited by pilot contamination, due to the fact that the same set of orthogonal pilots is reused in multiple cells. In the regime of moderately large number of antennas, another source of degradation is channel interpolation because the pilot signal of each user probes only a limited number of orthogonal frequency division multiplexing (OFDM) subcarriers, and the channel must be interpolated over the other subcarriers, where no pilot symbol is transmitted. In this paper, we propose a low-complexity algorithm that uses the received UL wideband pilot snapshots in an observation window comprising several coherence blocks (CBs) to obtain an estimate of the angle-delay power spread function (PSF) of the received signal. This is generally given by the sum of the angle-delay PSF of the desired user and the angle-delay PSFs of the copilot users, i.e., the users re-using the same pilot dimensions in other cells/sectors. We propose supervised and unsupervised clustering algorithms to decompose the estimated PSF and isolate the part corresponding to the desired user only. We use this decomposition to obtain an estimate of the covariance matrix of the user wideband channel vector, which we exploit to decontaminate the desired user channel estimate by applying minimum mean squared error (MMSE) smoothing filter, i.e., the optimal channel interpolator in the MMSE sense. We also propose an effective low-complexity approximation/implementation of this smoothing filter. We use numerical simulations to assess the performance of our proposed method, and compare it with other recently proposed schemes that use the same idea of separability of users in the angle-delay domain.

In this paper, we study the capacity and degree-of-freedom (DoF) scaling for the continuous-time amplitude limited AWGN channels in radio frequency (RF) and intensity modulated optical communication (OC) channels. More precisely, we study how the capacity varies in terms of the OFDM block transmission time T, bandwidth W, amplitude A and the noise spectral density N0/2. We first find suitable discrete encoding spaces for both cases, and prove that they are convex sets that have a semi-definite programming (SDP) representation. Using tools from convex geometry, we find lower and upper bounds on the volume of these encoding sets, which we exploit to drive pretty sharp lower and upper bounds on the capacity. We also study a practical Tone-Reservation (TR) encoding algorithm and prove that its performance can be characterized by the statistical width of an appropriate convex set. Recently, it has been observed that in high-dimensional estimation problems under constraints such as those arisen in Compressed Sensing (CS) statistical width plays a crucial role. We discuss some of the implications of the resulting statistical width on the performance of the TR. We also provide numerical simulations to validate these observations.

We propose a novel algorithm to design an optimum array geometry for source localization inside an enclosure. We assume a square-law decay propagation model for the sound acquisition so that the additive noise on the measured source-microphone distances is proportional to the distances regardless of the noise distribution. We formulate the source localization as an instance of the “Generalized Trust Region Subproblem” (GTRS) whose solution gives the location of the source. We show that by suitable selection of the microphone locations, one can tremendously decrease the noise-sensitivity of the resulting solution. In particular, by minimizing the noise-sensitivity of the source location in terms of sensor positions, we find the optimal noise-robust array geometry for the enclosure. Simulation results are provided to show the efficiency of the proposed algorithm.

LDPC codes are typically decoded by running a synchronous message passing algorithm over the corresponding bipartite factor graph (made of variable and check nodes). More specifically, each synchronous round consists of 1) updating all variable nodes based on the information received from the check nodes in the previous round, and then 2) updating all the check nodes based on the information sent from variable nodes in the current round. However, in many applications, ranging from message passing in neural networks to hardware implementation of LDPC codes, assuming that all messages are sent and received at the same time is far from realistic. In this paper, we investigate the effect of asynchronous message passing on the decoding of LDPC codes over a Binary Erasure Channel (BEC). We effectively assume that there is a random delay assigned to each edge of the factor graph that models the random propagation delay of a message along the edge. As a result, the output messages of a check/variable node are also asynchronously updated upon arrival of a new message in its input. We show, for the first time for BEC, that the asymptotic performance of the asynchronous message passing is fully characterized by a fixed point integral equation that takes into account both the temporal and the spatial features of the factor graph. Our theoretical result is reminiscent of the fixed point equation in traditional BP decoding. Also, our simulation results show that asynchronous scheduling reduces decoding time compared to the traditional BP in certain cases in the finite block-length regime.

In this paper, we design a new iterative low-complexity algorithm for computing the Walsh-Hadamard transform (WHT) of an N dimensional signal with a K-sparse WHT. We suppose that N is a power of two and K = O(Nα), scales sublinearly in N for some α ∈ (0, 1). Assuming a random support model for the nonzero transform-domain components, our algorithm reconstructs the WHT of the signal with a sample complexity O(K log2(N/K)) and a computational complexity O(K log2(K) log2(N/K)). Moreover, the algorithm succeeds with a high probability approaching 1 for large dimension N. Our approach is mainly based on the subsampling (aliasing) property of the WHT, where by a carefully designed subsampling of the time-domain signal, a suitable aliasing pattern is induced in the transform domain. We treat the resulting aliasing patterns as parity-check constraints and represent them by a bipartite graph. We analyze the properties of the resulting bipartite graphs and borrow ideas from codes defined over sparse bipartite graphs to formulate the recovery of the nonzero spectral values as a peeling decoding algorithm for a specific sparse-graph code transmitted over a binary erasure channel. This enables us to use tools from coding theory (belief-propagation analysis) to characterize the asymptotic performance of our algorithm in the very sparse (α ∈ (0, 1/3]) and the less sparse (α ∈ (1/3, 1)) regime. Comprehensive simulation results are provided to assess the empirical performance of the proposed algorithm.

In this paper, the `Approximate Message Passing' (AMP) algorithm, initially developed for compressed sensing of signals under i.i.d. Gaussian measurement matrices, has been extended to a multi-terminal setting (MAMP algorithm). It has been shown that similar to its single-terminal counterpart, the behavior of MAMP algorithm is fully characterized by a `State Evolution' (SE) equation for large block-lengths. This equation is used to obtain the rate-distortion curve of a multi-terminal memoryless source. It is observed that by spatially coupling the measurement matrices, the rate-distortion curve of MAMP algorithm undergoes a phase transition, where the measurement rate region corresponding to a low-distortion (approximately zero distortion) regime is fully characterized by the joint and the conditional Rényi information dimension (RID) of the multi-terminal source. This measurement rate region is very similar to the rate region of the Slepian-Wolf distributed source coding problem where the RID plays a role similar to the discrete entropy. Simulations are done to investigate the empirical behavior of MAMP algorithm. It is observed that simulation results match very well with the predictions of SE equation for reasonably large block-lengths.

The entropy power inequality (EPI) yields lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(1) when X, X' are independent identically distributed (i.i.d.) with high entropy. This paper provides the inequality H(X + X') - H(X)≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+satisfying g(0) = 0. Extensions to nonidentically distributed random variables and to conditional entropies are also obtained.

This paper shows that the Rényi information dimension (RID) of an i.i.d. sequence of mixture random variables polarizes to the extremal values of 0 and 1 (fully discrete and continuous distributions) when transformed by an Hadamard matrix. This provides a natural counter-part over the reals of the entropy polarization phenomenon over finite fields. It is further shown that the polarization pattern of the RID is equivalent to the BEC polarization pattern, which admits a closed form expression. These results are used to construct universal and deterministic partial Hadamard matrices for analog to analog (A2A) compression of memoryless sources. In addition, a framework for the A2A compression of multi-terminal correlated sources is developed, providing a first counter-part of the Slepian-Wolf coding problem in the A2A setting.

The entropy power inequality (EPI) provides lower bounds on the differential entropy of the sum of two independent real-valued random variables in terms of the individual entropies. Versions of the EPI for discrete random variables have been obtained for special families of distributions with the differential entropy replaced by the discrete entropy, but no universal inequality is known (beyond trivial ones). More recently, the sumset theory for the entropy function yields a sharp inequality H(X + X') - H(X) ≥ 1/2 - o(l) when X,X' are i.i.d. with high entropy. This paper provides the inequality H(X + X') - H(X) ≥ g(H(X)), where X, X' are arbitrary i.i.d. integer-valued random variables and where g is a universal strictly positive function on R+satisfying g(0) = 0. Extensions to non identically distributed random variables and to conditional entropies are also obtained.

This paper investigates the construction of deterministic measurement matrices preserving the entropy of a random vector with a given probability distribution. In particular, it is shown that for a random vector with i.i.d. discrete components, this is achieved by selecting a subset of rows of a Hadamard matrix such that (i) the selection is deterministic (ii) the fraction of selected rows is vanishing. In contrast, it is shown that for a random vector with i.i.d. continuous components, no entropy preserving measurement matrix allows dimensionality reduction. These results are in agreement with the results of Wu-Verdu on almost lossless analog compression and provide a low-complexity measurement matrix. The proof technique is based on a polar code martingale argument and on a new entropy power inequality for integer-valued random variables.