We introduce a new approach to processing complex-valued data using DNNs consisting of parallel real-valued subnetworks with coupled outputs. Our proposed class of architectures, referred to as Steinmetz Neural Networks, incorporates multi-view learning to construct more interpretable representations in the latent space. Moreover, we present the Analytic Neural Network, which incorporates a consistency penalty that encourages analytic signal representations in the latent space of the Steinmetz neural network. This penalty enforces a deterministic and orthogonal relationship between the real and imaginary components. Using an information-theoretic construction, we demonstrate that the generalization gap upper bound posited by the analytic neural network is lower than that of the general class of Steinmetz neural networks. Our numerical experiments depict the improved performance and robustness to additive noise, afforded by our proposed networks on benchmark datasets and synthetic examples.

We consider an instance of a radar jamming countermeasure problem, where the radar and the jammer have uncertainty about the radar environment (for instance, about the noise variance and the radar cross section variance), and they have to account for these uncertainties with statistical priors. The radar-jammer interaction is modelled as a two-player zero-sum repeated Bayesian game (also called an incomplete information game). We discuss the computation of the optimal strategies and present numerical results that illustrate the optimal strategies in an example.

We present a new formula for computing arbitrary moments of unitarily invariant matrix distributions. The Schur-Weyl duality is used to decompose the expected value of tensor powers of the random matrices as a linear combination of projection operators onto unitary irreducible representations. The coefficients in this combination, which are labeled by Young diagrams, are expectations of products of determinants of the random matrices. It is demonstrated in a number of important cases, including matrix gamma and matrix beta distributions, that these coefficients can be simply computed from a knowledge of the normalization factors of the distributions.

We investigate the theory of affine groups in the context of designing radar waveforms that obey the desired wideband ambiguity function (WAF). The WAF is obtained by correlating the signal with its time-dilated, Doppler-shifted, and delayed replicas. We consider the WAF definition as a coefficient function of the unitary representation of the group a • x + b. This is essentially an algebraic problem applied to the radar waveform design. Prior works on this subject largely analyzed narrow-band ambiguity functions. Here, we show that when the underlying wideband signal of interest is a pulse or pulse train, a tight frame can be built to design that waveform. Specifically, we design the radar signals by minimizing the ratio of bounding constants of the frame in order to obtain lower sidelobes in the WAF. This minimization is performed by building a codebook based on difference sets in order to achieve the Welch bound. We show that the tight frame so obtained is connected with the wavelet transform that defines the WAF.

Occupancy grids encode for hot spots on a map that is represented by a two dimensional grid of disjoint cells. The problem is to recursively update the probability that each cell in the grid is occupied, based on a sequence of sensor measurements from a moving platform. In this paper, we provide a new Bayesian framework for generating these probabilities that does not assume statistical independence between the occupancy state of grid cells. This approach is made analytically tractable through the use of binary asymmetric channel models that capture the errors associated with observing the occupancy state of a grid cell. Binary-valued measurement vectors are the thresholded output of a sensor in a radar, sonar, or other sensory system. We compare the performance of the proposed framework to that of the classical formulation for occupancy grids. The results show that the proposed framework identifies occupancy grids with lower false alarm and miss detection rates, and requires fewer observations of the surrounding area, to generate an accurate estimate of occupancy probabilities when compared to conventional formulations.

In this paper, we analyze the impact of compressed sensing with random matrices on Fisher information and the CRB for estimating unknown parameters in the mean value function of a multivariate normal distribution. We consider the class of random compression matrices that satisfy a version of the Johnson-Lindenstrauss lemma, and we derive analytical lower and upper bounds on the CRB for estimating parameters from randomly compressed data. These bounds quantify the potential loss in CRB as a function of Fisher information of the non-compressed data. In our numerical examples, we consider a direction of arrival estimation problem and compare the actual loss in CRB with our bounds.

Each of a large number of nodes takes a measurement in sequence to decide between two hypotheses about the state of the world. Each node also has available the decisions of some of its immediate predecessors and uses these and its own measurement to make its decision. Each node broadcasts its decision through a binary symmetric channel, which randomly flips the decision. The question treated here is whether there exists a decision strategy consisting of a sequence of likelihood ratio tests such that the decisions approach the true hypothesis as the number of nodes increases. We show that if each node learns from bounded number of predecessors, then the decisions cannot converge to the underlying truth. We show that if each node learns from all predecessors then the decisions converge in probability to the underlying truth when the flipping probabilities are bounded away from 1/2. We also derive, in the case when the flipping probabilities tend to 1/2, a condition on the convergence rate of the flipping probabilities that is required for the decisions to converge to the true hypothesis in probability.

A Compressive Sensing (CS) based solution is proposed for centralized and distributed discovery of physical phenomena in large scale Wireless Sensor Networks (WSNs). WSNs monitoring environmental phenomena over large geographic areas collect measurements from a large number of distributed sensors. Compressive Sensing provides an effective means of discovery and reconstruction of functions with only a subset of samples. Traditional CS relies on uniformly distributed samples which limits practicality of CS based recovery. To enhance the flexibility of sampling and implementation, the proposed approach uses random walk based samples. Unlike uniform sampling, random walk based sampling enables individual nodes achieve phenomenon awareness, i.e., the physical distribution of the phenomenon. We also derive a theoretical upper bound for the reconstruction failure probability. Simulation results on the number of samples required and error show that random walk based sampling is comparable to uniform sampling but with superior energy efficiency. More importantly, the proposed scheme provides a practical solution for a range of applications where uniform sampling is less economical or even infeasible.

We present an approach to adaptive measurement selection in compressive sensing for estimating sparse signals. Given a fixed number of measurements, we consider the sequential selection of the rows of a compressive measurement matrix to maximize the mutual information between the measurements and the sparse signal's support. We formulate this problem as a partially observable Markov decision process (POMDP), which enables the application of principled reasoning for sequential measurement selection based on Bellman's optimality condition.

We study the detection error probability associated with a balanced binary relay tree, where the leaves of the tree correspond toNidentical and independent sensors. The root of the tree represents a fusion center that makes the overall detection decision. Each of the other nodes in the tree is a relay node that combines two binary messages to form a single output binary message. Only the leaves are sensors. In this way, the information from the sensors is aggregated into the fusion center via the relay nodes. In this context, we describe the evolution of the Type I and Type II error probabilities of the binary data as it propagates from the leaves toward the root. Tight upper and lower bounds for the total error probability at the fusion center as functions ofNare derived. These characterize how fast the total error probability converges to 0 with respect toN, even if the individual sensors have error probabilities that converge to 1/2.

Compressed sensing theory suggests that successful inversion of an image of the physical world from its modal parameters can be achieved at measurement dimensions far lower than the image dimension, provided that the image is sparse in an a priori known basis. The assumed basis for sparsity typically corresponds to a gridding of the parameter space, e.g., an DFT grid in spectrum analysis. However, in reality no physical field is sparse in the DFT basis or in an a priori known basis. No matter how finely we grid the parameter space the sources may not lie in the center of the grid cells and there is always mismatch between the assumed and the actual bases for sparsity. In this paper, we study the sensitivity of compressed sensing (basis pursuit to be exact) to mismatch between the assumed and the actual sparsity bases. Our mathematical analysis and numerical examples show that the performance of basis pursuit degrades considerably in the presence of basis mismatch.

A new approach based on graph wavelets for analyzing the spatial and temporal behavior of Internet traffic anomalies is presented. This approach is applied to Internet2 traffic measurements to evaluate the time duration and spatial spread (number of links affected) of anomalies. Based on the empirical results, a node model is proposed that captures the behavior of anomalies at individual network nodes. The model considers various aspects of anomalies, such as its origin, termination, propagation, duration and volume changes. The derivation of the model parameters requires only local node information, but the model is capable of producing network-wide anomalies whose behavior mimics network wide anomalies. Model is verified by using Internet2 traffic data. Since the proposed model can be specified using only a few parameters, it can be used in place of large anomaly traces with a great data reduction. As extensions, the model is applied over a path and an aggregated model that applies to a neighborhood in the network is also presented. A method to use the graph wavelet components found during the analysis to implement a real-time anomaly monitoring system is also discussed.

In this paper we will examine the effect of different parameters in the quality of real compressively sampled images in the compressed sensing framework. We will select a variety of different real images of different types and test the quality of the recovered images, the recovery time, and required resources when different measurement methods with different parameters are used or when different recovering methods are applied. Then we will propose an algorithm to reduce the noise in the recovered images and sharpen them simultaneously. The algorithm exploits a well-known bilateral filtering in order to increase the confidence in margins and edges, and then uses an adaptive unsharp mask method to sharpen the images. The adaptive unsharp mask method extends the ordinary unsharp mask method and uses machine learning square loss minimization and regression in order to learn the optimal unsharping parameters. We will argue why both bilateral filtering and unsharp mask methods should be used in the algorithm simultaneously. Finally, we will show the results of applying the algorithm on real images that are recovered using the compressed sensing method and we will interpret the experimental results.

We describe a method of constructing a sequence (pulse train) of phase-coded waveforms, for which the ambiguity function is free of range sidelobes along modest Doppler shifts. The constituent waveforms are Golay complementary waveforms which have ideal ambiguity along the zero Doppler axis but are sensitive to nonzero Doppler shifts. We extend this construction to multiple dimensions, in particular to radar polarimetry, where the two dimensions are realized by orthogonal polarizations. Here we determine a sequence of two-by-two Alamouti matrices where the entries involve Golay pairs and for which the range sidelobes associated with a matrix-valued ambiguity function vanish at modest Doppler shifts. The Prouhet–Thue–Morse sequence plays a key role in the construction of Doppler resilient sequences of Golay complementary waveforms.

In this paper, relation between multi-class linear and kernel Fisher discriminant analysis (FDA) and linear and kernel canonical correlation analysis (CCA) is established. It is shown that in a multi-class classification problem, the CCA between feature vectors (or a nonlinearly mapped version of them) as one-channel and the class label vectors as the second channel is equivalent to multi-class FDA. The multi-class Fisher distance is found to be decomposed into a sum of terms, each of which is determined by a canonical correlation. This result is extended to the kernel formulation without explicit computation of the nonlinear mappings. A simple example is presented to numerically verify the results.

A kernel-based formulation for decomposing nonlinear maps of two data channels into their canonical coordinates is derived. Each data channel is implicitly mapped to a high dimensional feature space defined by a nonlinear kernel. The canonical coordinates of the nonlinear maps are then found by transforming the kernel maps with the eigenvector matrices of a coupled asymmetric generalized eigenvalue problem. This generalized eigenvalue problem is constructed in the explicit space of kernel maps. The measures of linear dependence and coherence between the nonlinear maps of the channels are also presented. These measures may be determined in the kernel domain, without explicit computation of the nonlinear mappings. A numerical example is also presented.

A network structure for canonical coordinate decomposition is presented. The network consists of two single-layer linear subnetworks that together extract the canonical coordinates of two data channels. The connection weights of the networks are trained by a stochastic gradient descent learning algorithm. Each subnetwork features a hierarchical set of lateral connections among its outputs. The lateral connections perform a deflation process that subtracts the contribution of the already extracted coordinates from the input data subspace. This structure allows for adding new nodes for extracting additional canonical coordinates without the need for retraining the previous nodes. The performance of the network is evaluated on a synthesized data set.