This paper presents several approximate formulas for the Wiener filter (WF), the optimal linear filter minimizing the mean-squared error. Compared to the WF, our formulas do not directly involve inverting the observation covariance matrix. An important consequence is that our approximate filters do not suffer from the ill-conditioning of the covariance matrix and are numerically reliable to compute. In addition, we prove that the approximate formulas converge to the WF as certain approximate terms vanish. Finally, our performance-complexity tradeoff analysis with empirical data show that our filters are two orders of magnitude faster than the WF without compromising any accuracy.

This paper addresses the passive detection of a common signal in two multi-sensor arrays. For this problem, we derive a detector based on likelihood theory for the case of one-antenna transmitters, independent Gaussian noises with arbitrary spatial structure, Gaussian signals, and known channel subspaces. The detector uses a likelihood ratio where all but one of the unknown parameters are replaced by their maximum likelihood (ML) estimates. The ML estimation of the remaining parameter requires a numerical search, and it is therefore estimated using a sample-based estimator. The performance of the proposed detector is illustrated by means of Monte Carlo simulations and compared with that of the detector for unknown channels, showing the advantage of this knowledge.

We consider the problem of classifying 3-D objects undergoing rigid transformations. It has been shown that the rigid transformation universal manifold embedding (RTUME) provides a mapping from the orbit of observations on some object to a single low-dimensional linear subspace of Euclidean space. This linear subspace is invariant to the geometric transformations. In the classification problem the RTUME subspace extracted from an experimental observation is tested against a set of subspaces representing the different object manifolds, in search for the nearest class. We elaborate on the design problem of the RTUME operator in the case where the point cloud sampled from the object is sparse, noisy, and non-uniformly sampled. By introducing metric learning and negative-mining techniques into the framework of Grassmannian dimensionality reduction for universal manifold embedding, we improve classification performance for these challenging sampling conditions.

Occupancy grids encode for hot spots on a map that is represented by a two dimensional grid of disjoint cells. The problem is to recursively update the probability that each cell in the grid is occupied, based on a sequence of sensor measurements from a moving platform. In this paper, we provide a new Bayesian framework for generating these probabilities that does not assume statistical independence between the occupancy state of grid cells. This approach is made analytically tractable through the use of binary asymmetric channel models that capture the errors associated with observing the occupancy state of a grid cell. Binary-valued measurement vectors are the thresholded output of a sensor in a radar, sonar, or other sensory system. We compare the performance of the proposed framework to that of the classical formulation for occupancy grids. The results show that the proposed framework identifies occupancy grids with lower false alarm and miss detection rates, and requires fewer observations of the surrounding area, to generate an accurate estimate of occupancy probabilities when compared to conventional formulations.

Sensor fusion maybe used to improve detection performance in applications. The idea is to make decisions locally, and then transmit them to a global fusion centre where the global decision is made. For global decision making, Bayes or Neyman–Pearson reasoning determines the optimal use of the local decision variables. However, the determination of the local decision variables that minimise global error probability is intractable. In this study, the authors design local decisions that maximise the mutual information between a binary decision variable and the underlying binary state. This serves as a benchmark against which globally optimum solutions maybe compared. Then, they use the theory of large deviations (LDs) to determine a local decision rule that minimises asymptotic global error probability. The use of LD produces a one‐dimensional search on a receiver operating characteristic curve to equalise the error exponents for local false alarm and miss probabilities. Many interesting properties of the LD solution are proved. Numerical results illustrate the performance of the asymptotically optimum decision rule for finite collections of sensors.

This letter considers the problem of threshold selection for a correlation test among multiple (≥2) random vectors. The generalized likelihood ratio test (GLRT) for this problem uses a generalized Hadamard ratio to test for block diagonality in a composite covariance matrix. As the number of realizations used to estimate the composite covariance matrix grows large, the null distribution of the likelihood ratio statistic converges to a chi-squared distribution which can be used to prescribe thresholds needed to achieve a desired false alarm rate in high sample support situations. However, this asymptotic distribution can be slow to converge, making its use dubious in many practical scenarios. To address this problem, this letter uses saddlepoint approximations for the null distribution of the generalized Hadamard ratio. Simulations are provided to demonstrate the saddlepoint approximation's ability to achieve a desired false alarm probability, even in situations with low sample support.

We propose a new detector of primary users in cognitive radio networks. The main novelty of the proposed detector in comparison to most known detectors is that it is based on sound statistical principles for detecting cyclostationary signals. In particular, the proposed detector is (asymptotically) the locally most powerful invariant test, i.e. the best invariant detector for low signal-to-noise ratios. The derivation is based on two main ideas: the relationship between a scalar-valued cyclostationary signal and a vector-valued wide-sense stationary signal, and Wijsman's theorem. Moreover, using the spectral representation for the cyclostationary time series, the detector has an insightful interpretation, and implementation, as the broadband coherence between frequencies that are separated by multiples of the cycle frequency. Finally, simulations confirm that the proposed detector performs better than previous approaches.

This paper considers the problem of testing for the independence among multiple random vectors with each random vector representing a time series captured at one sensor. Implementing the Generalized Likelihood Ratio Test involves testing the null hypothesis that the composite covariance matrix of the channels is block-diagonal through the use of a generalized Hadamard ratio. These results are then extended to the problem of detecting the presence of correlated time series when several observers each employ an array of sensors. Assuming wide-sense stationary processes in both time and space, results on large block-Toeplitz matrices suggest the use of a broadband integral of a frequency-wavenumber dependent Hadamard ratio as an alternative test statistic.

We derive the generalized likelihood ratio test (GLRT) for detecting cyclostationarity in scalar-valued time series. The main idea behind our approach is Gladyshev's relationship, which states that when the scalar-valued cyclostationary signal is blocked at the known cycle period it produces a vector-valued wide-sense stationary process. This result amounts to saying that the covariance matrix of the vector obtained by stacking all observations of the time series is block-Toeplitz if the signal is cyclostationary, and Toeplitz if the signal is wide-sense stationary. The derivation of the GLRT requires the maximum likelihood estimates of Toeplitz and block-Toeplitz matrices. This can be managed asymptotically (for large number of samples) exploiting Szegö's theorem and its generalization for vector-valued processes. Simulation results show the good performance of the proposed GLRT.

In this paper, we examine adaptive compression policies, when the sequence of vector-valued measurements to be compressed is noisy and the compressed variables are themselves noisy. The optimization criterion is information gain. In the case of sequential scalar compressions, the unit-norm compression vectors that greedily maximize per-stage information gain are eigenvectors of an a priori error covariance matrix, and the greedy policy selects them according to eigenvalues of a posterior covariance matrix. These eigenvalues depend on all previous compressions and are computed recursively. A water-filling solution is given for the optimum compression policy that maximizes net information gain, under a constraint on the average norm of compression vectors. We provide sufficient conditions under which the greedy policy for maximizing stepwise information gain actually is optimal in the sense of maximizing the net information gain. In the case of scalar compressions, our examples and simulation results illustrate that the greedy policy can be quite close to optimal when the noise sequences are white.

Spectrum sensing is an important building block to realize the cognitive radio concept. In order to combat fading in the wireless environment, cooperation among the sensing users is usually employed. In this paper, we develop a closed-form optimal local decision threshold for cooperative spectrum sensing in cognitive radio systems via large deviation analysis. The resultant strategy is independent of the total number of cooperating users. We show that it is not only asymptotically optimal when the number of sensing users approaches infinity, but also can achieve the maximum diversity. Numerical results are provided to verify our analysis.

This paper considers the problem of testing for the independence among multiple (≥ 2) random vectors with each random vector representing a time series captured at one sensor. Implementing the Generalized Likelihood Ratio Test involves testing the null hypothesis that the composite covariance matrix of the channels is block-diagonal through the use of a generalized Hadamard ratio. Using the theory of linear prediction and its connection with Gram determinants, it is shown that this generalized Hadamard ratio can be written as a product of scalars which are independently drawn from a beta distribution under the null hypothesis. This result is useful from a Monte Carlo analysis standpoint in that it is much more computationally efficient to form a product of scalar beta random variables than it is to compute the determinant of complex Wishart matrices.

In this paper, we analyze the impact of compressed sensing with random matrices on Fisher information and the CRB for estimating unknown parameters in the mean value function of a multivariate normal distribution. We consider the class of random compression matrices that satisfy a version of the Johnson-Lindenstrauss lemma, and we derive analytical lower and upper bounds on the CRB for estimating parameters from randomly compressed data. These bounds quantify the potential loss in CRB as a function of Fisher information of the non-compressed data. In our numerical examples, we consider a direction of arrival estimation problem and compare the actual loss in CRB with our bounds.

This paper is motivated by the problem of integrating multiple sources of measurements. We consider two multiple-input-multiple-output (MIMO) channels, a primary channel and a secondary channel, with dependent input signals. The primary channel carries the signal of interest, and the secondary channel carries a signal that shares a joint distribution with the primary signal. The problem of particular interest is designing the secondary channel matrix, when the primary channel matrix is fixed. We formulate the problem as an optimization problem, in which the optimal secondary channel matrix maximizes an information-based criterion. An analytical solution is provided in a special case. Then an intrinsic search algorithm is proposed to approximate the optimal solutions in general cases. In particular, the intrinsic algorithm exploits the geometry of the unit sphere, a manifold embedded in Euclidean space.

In this paper, we study the existence of locally most powerful invariant tests (LMPIT) for the problem of testing the covariance structure of a set of Gaussian random vectors. The LMPIT is the optimal test for the case of close hypotheses, among those satisfying the invariances of the problem, and in practical scenarios can provide better performance than the typically used generalized likelihood ratio test (GLRT). The derivation of the LMPIT usually requires one to find the maximal invariant statistic for the detection problem and then derive its distribution under both hypotheses, which in general is a rather involved procedure. As an alternative, Wijsman's theorem provides the ratio of the maximal invariant densities without even finding an explicit expression for the maximal invariant. We first consider the problem of testing whether a set ofN-dimensional Gaussian random vectors are uncorrelated or not, and show that the LMPIT is given by the Frobenius norm of the sample coherence matrix. Second, we study the case in which the vectors under the null hypothesis are uncorrelated and identically distributed, that is, the sphericity test for Gaussian vectors, for which we show that the LMPIT is given by the Frobenius norm of a normalized version of the sample covariance matrix. Finally, some numerical examples illustrate the performance of the proposed tests, which provide better results than their GLRT counterparts.

This work addresses the problem of deciding whether a set of realizations of a vector-valued time series with unknown temporal correlation are spatially correlated or not. Specifically, the spatial correlation is induced by a colored source over a frequency-flat single-input multiple-output (SIMO) channel distorted by independent and identically distributed noises with temporal correlation. The generalized likelihood ratio test (GLRT) for this detection problem does not have a closed-form expression and we have to resort to numerical optimization techniques. In particular, we apply the successive convex approximations approach which relies on solving a series of convex problems that approximate the original (non-convex) one. The proposed solution resembles a power method for obtaining the dominant eigenvector of a matrix, which changes over iterations. Finally, the performance of the proposed detector is illustrated by means of computer simulations showing a great improvement over previously proposed detectors that do not fully exploit the temporal structure of the source.

This paper considers linear precoding for time-varying multiple input multiple-output (MIMO) channels. We show that linear minimum mean-squared error (LMMSE) equalization based on the conjugate gradient (CG) method can result in significantly reduced complexity compared with conventional approaches. This reduction is achieved by incorporating a condition number constraint into the precoder optimization framework, which leads to clustered eigen values of the measurement covariance matrix. The cost is a small increase in MSE compared to the optimal precoder.

Compressed sensing theory suggests that successful inversion of an image of the physical world from its modal parameters can be achieved at measurement dimensions far lower than the image dimension, provided that the image is sparse in an a priori known basis. The assumed basis for sparsity typically corresponds to a gridding of the parameter space, e.g., an DFT grid in spectrum analysis. However, in reality no physical field is sparse in the DFT basis or in an a priori known basis. No matter how finely we grid the parameter space the sources may not lie in the center of the grid cells and there is always mismatch between the assumed and the actual bases for sparsity. In this paper, we study the sensitivity of compressed sensing (basis pursuit to be exact) to mismatch between the assumed and the actual sparsity bases. Our mathematical analysis and numerical examples show that the performance of basis pursuit degrades considerably in the presence of basis mismatch.

One of the key problems in cognitive radio (CR) is the detection of primary activity in order to determine which parts of the spectrum are available for opportunistic access. This detection task is challenging, since the wireless environment often results in very low SNR conditions. Moreover, calibration errors and imperfect analog components at the CR spectral monitor result in uncertainties in the noise spectrum, making the problem more difficult. In this work, we present a new multiantenna detector which is based on the fact that the observation noise processes are spatially uncorrelated, whereas any primary signal present should result in spatial correlation. In particular, we derive the generalized likelihood ratio test (GLRT) for this problem, which is given by the quotient between the determinant of the sample covariance matrix and the determinant of its block-diagonal version. For stationary processes the GLRT tends asymptotically to the integral of the logarithm of the Hadamard ratio of the estimated power spectral density matrix. Additionally, we present an approximation of the frequency domain detector in the low SNR regime, which results in computational savings. The performance of the proposed detectors is evaluated by means of numerical simulations, showing important advantages over existing detectors.

Wireless sensor networks are considered in which sensors convey binary decisions over fading channels to a common fusion center. The fusion center first takes each received signal and makes an estimate of the transmitted bit. The average of the estimated bits is compared to a threshold to make a global decision. Exponential error bounds are derived that allow one to trade off signal-to-noise ratio versus the number of sensors to achieve desired average error levels. An attractive feature of the bounds is that they do not require exact knowledge of the wireless channel statistics; approximations are sufficient.

We provide a review of independent component analysis (ICA) for complex-valued improper and noncircular random sources. An improper random signal is correlated with its complex conjugate, and a noncircular random signal has a rotationally variant probability distribution. We present methods for ICA using second-order statistics, and higher-order statistics. For ICA based on second-order statistics, we emphasize the key role played by the circularity coefficients, which are the canonical correlations between the source and the complex conjugate. For ICA based on higher-order statistics, we show how to extend algorithms for real-valued ICA to the complex domain using Wirtinger calculus.

This is a paper on modulation theory that addresses joint analog precoder and equalizer design for multichannel data transmission over the frequency-selective additive Gaussian noise (AGN) channel. The design goal is to maximize mutual information rate, minimize the mean square error, or minimize the bit error rate subject to a transmit power constraint. We assume a continuous channel model with precoder transmissions for m subchannels that lie in an n-dimensional linear subspace of L2(R). m and n are design parameters. We first design the subspace according to the channel characteristics, and then design the precoders as functions in this subspace. After the design of the optimal precoder and equalizer, we explore the geometry of these designs. We show that all of these precoder and equalizer designs are, in fact, decompositions of a virtual two- channel problem into a system of canonical coordinates, wherein variables in the canonical message channel are correlated only pairwise with corresponding variables in the canonical measurement channel. This finding clarifies the geometry of precoder and equalizer designs and illustrates that they decompose the two-channel communication problem into what might be called the Shannon channel.

In this paper we propose decision schemes to distinguish between the H0hypothesis that range cells under test contain disturbance only (i.e., noise plus interference) and the H1hypothesis that they also contain signal components along a direction which is a priori unknown, but constrained to belong to a given subspace (H) of the observables. The disturbance is modeled in terms of complex normal noise vectors plus deterministic interference assumed to belong to a known subspace (J) of the observables. At the design stage we resort to either the plain generalized likelihood ratio test (GLRT) or the two-step GLRT-based design procedure. Moreover, we assume that a set of noise only (secondary) data is available. A preliminary performance analysis, conducted by resorting to simulated data, shows that the one-step GLRT performs better than the two-step GLRT-based design procedure

In this work, we present a class of low-complexity reduced-dimension adaptive beamformers constructed from expanding Krylov subspaces. We demonstrate how the data dimensionality reduction obtained from Krylov pre-processing decreases the sensitivity of reduced-rank adaptive beamforming techniques to incorrect model-order selection and lessens the computational complexity of systems involving large arrays with many elements. An important advantage of the proposed dimensionality reduction scheme is that it relieves reduced-rank methods from the stringent requirement on the precise model order determination.

A complex random vector is called improper if it is correlated with its complex conjugate. In this paper, we present a generalized likelihood ratio test (GLRT) for impropriety. This test is compelling because it displays the right invariances: The proposed GLR is invariant to linear transformations on the data, including rotation and scaling, just as propriety is preserved by linear transformations. Because canonical correlations make up a complete, or maximal, set of invariants for the Hermitian and complementary covariance matrices under linear transformations, the GLR can be shown to be a function of the squared canonical correlations between the data and its complex conjugate. This validates our intuition that the internal coordinate system should not matter for this hypothesis test

Cramer-Rao bounds have been previously generalized to the class of nonlinear estimation problems on manifolds. This new approach can be used to derive a broad class of quadratic error performance bounds. A generalized intrinsic score function on the manifold-valued parameter space is introduced that distinguishes one bound from another. The derivation itself is invariant to transformations of the parameter space and score space. The resulting generalized Weiss-Weinstein bounds are shown to be invariant to certain transformations of the score. Applications of this work include cases where ambiguities, low signal-to-noise, or low sample support limit the utility of Cramer-Rao bounds, and more general quadratic bounds on manifold-valued parameters must be considered

This paper addresses the trade-off between multiplexing gain and diversity gain for the frequency-selective channel. We derive this trade-off by considering the scaling law of the ergodic capacity, which determine the multiplexing gain, and the error probability, which determines the diversity gain at high SNR. It is proved that this trade-off only depends on the number of independent taps of the equivalent FIR channel filter. The error probability is bounded by the outage probability and the error probability without outage. The scaling law of the outage probability and the error probability without outage at high SNR is derived, as are approximations of the outage probability at both low and high SNR

A complex random vector is called improper if it is correlated with its complex conjugate. We present a hypothesis test for impropriety based on a generalized likelihood ratio (GLR). This GLR is invariant to linear transformations on the data, including rotation and scaling, because propriety is preserved by linear transformations. More specifically, we show that the GLR is a function of the squared canonical correlations between the data and their complex conjugate. These canonical correlations make up a complete, or maximal, set of invariants for the Hermitian and complementary covariance matrices under linear, but not widely linear, transformation

In this paper, we address the problem of detecting a signal whose associated spatial signature is subject to uncertainties, in the presence of subspace interference and broadband noise, and using multiple snapshots from an array of sensors. To account for steering vector uncertainties, we assume that the spatial signature of interest lies in a given linear subspacewhile its coordinates in this subspace are unknown. The generalized likelihood ratio test (GLRT) for the problem at hand is formulated. We show that the GLRT amounts to searching for the best direction in the subspaceafter projecting out the interferences. The distribution of the GRLT under both hypotheses is derived and numerical simulations illustrate its performance.

Realizing the large user capacity planned for ultra-wideband (UWB) systems motivates multiuser detection (MUD). However, it is impractical to implement conventional chip-rate MUD methods, because UWB signaling gives rise to high detection complexity and difficulty in capturing energy scattered by dense multipath. In this paper, we develop a reception model for UWB multiple access based on frame-rate sampled signals in lieu of chip-rate samples. This model enables low-complexity MUD, of which we examine a reduced-rank Wiener filter for blind symbol detection. We show that frame-rate UWB samples have a small number of distinct eigenvalues in the data covariance matrix, resulting in warp convergence of reduced-rank filtering. The proposed MUD method exhibits good performance at low complexity, even in the presence of strong frequency-selective multipath fading.

Based on the Cramer-Loeve spectral representation for a harmonizable random process, the Rihaczek distribution is a time- and frequency-shift covariant, bilinear time-frequency distribution. It can be expressed as a complex Hilbert space inner product between the time series and its infinitesimal stochastic Fourier generator. We show that we may attach an illuminating geometry to this inner product, wherein the cosine-squared of the angle between the time series and its infinitesimal stochastic Fourier generator is given by the Rihaczek distribution. We propose to construct estimators of the Rihaczek distribution using a factored kernel in Cohen's class of bilinear time-frequency distributions

The Rihaczek distribution for stochastic signals is a time- and frequency-shift covariant bilinear time-frequency distribution (TFD) based on the Crame/spl acute/r-Loe/spl grave/ve spectral representation for a harmonizable process. It is a complex Hilbert space inner product (or cross correlation) between the time series and its infinitesimal stochastic Fourier generator. To this inner product, we may attach an illuminating geometry, wherein the cosine squared of the angle between the time series and its infinitesimal stochastic Fourier generator is given by the Rihaczek distribution. The Rihaczek distribution also determines a time-varying Wiener filter for estimating a time series from its infinitesimal stochastic Fourier generator and measures the resulting error covariance. We propose a factored kernel to construct estimators of the Rihaczek distribution that are contained in Cohen's class of bilinear TFDs.

Nonstationary complex random signals are in general improper (not circularly symmetric), which means that their complementary covariance is nonzero. Since the Karhunen-Loeve (K-L) expansion in its known form is only valid for proper processes, we derive the improper version of this expansion. It produces two sets of eigenvalues and improper observable coordinates. We then use the K-L expansion to solve the problems of detection and estimation of improper complex random signals in additive white Gaussian noise. We derive a general result comparing the performance of conventional processing, which ignores complementary covariances, with processing that takes these into account. In particular, for the detection and estimation problems considered, we find that the performance gain, as measured by deflection and mean-squared error (MSE), respectively, can be as large as a factor of 2. In a communications example, we show how this finding generalizes the result that coherent processing enjoys a 3-dB gain over noncoherent processing.

For complex signals, n-th order moment functions can be defined in 2/sup n/ different ways, depending on the placement of complex conjugates. We demonstrate that, for stationary analytic signals, only a few of these different moments are actually required for a complete n-th order description. Which, and how many of them, depends on the signal's spectrum. We investigate properties of n-th order moments and spectra with different conjugation patterns and show how they provide different information about the signal.

A kernel-based formulation for decomposing nonlinear maps of two data channels into their canonical coordinates is derived. Each data channel is implicitly mapped to a high dimensional feature space defined by a nonlinear kernel. The canonical coordinates of the nonlinear maps are then found by transforming the kernel maps with the eigenvector matrices of a coupled asymmetric generalized eigenvalue problem. This generalized eigenvalue problem is constructed in the explicit space of kernel maps. The measures of linear dependence and coherence between the nonlinear maps of the channels are also presented. These measures may be determined in the kernel domain, without explicit computation of the nonlinear mappings. A numerical example is also presented.

The detection of constrained subspace signals in additive infinite-dimensional interference and noise is motivated by consideration of multipath-Doppler channels subject to interference from partially overlapping frequency bands of other sources. Since the interference lies in an infinite-dimensional subspace, the standard method of projecting onto the subspace containing the signal plus interference does not yield a finite-dimensional detection problem. However, an alternative approach may be used to extract the appropriate finite-dimensional problem. Moreover, an energy constraint may be imposed on the desired signal. The generalized likelihood-ratio receiver for this problem is obtained, and expressions for its average probability of error are given.

Non-stationary complex random signals are in general improper (not circularly symmetric), which means that their complementary covariance is non-zero. Since the Karhunen-Loeve expansion in its known form is only valid for proper processes, we derive the improper version of this expansion. It produces two sets of eigenvalues and an improper internal description. We use the Karhunen-Loeve expansion to solve the problem of detecting non-stationary improper complex random signals in additive white Gaussian noise. Using the deflection criterion we compare the performance of conventional processing, which ignores complementary covariances, with processing that takes these into account. The performance gain can be as great as a factor of 2.

A network structure for canonical coordinate decomposition is presented. The network consists of two single-layer linear subnetworks that together extract the canonical coordinates of two data channels. The connection weights of the networks are trained by a stochastic gradient descent learning algorithm. Each subnetwork features a hierarchical set of lateral connections among its outputs. The lateral connections perform a deflation process that subtracts the contribution of the already extracted coordinates from the input data subspace. This structure allows for adding new nodes for extracting additional canonical coordinates without the need for retraining the previous nodes. The performance of the network is evaluated on a synthesized data set.

We consider M-ary communication with K users over a space diversity channel, consisting of a single transmit antenna for each user and multiple receive antennas. We examine two different flat fading models, namely, phase coherent wavefront fading and noncoherent element-to-element fading. In the case of wavefront fading, the fade is constant across the face of the receive antenna and we can associate an angle of arrival to the signal. We present a variation of the MUSIC algorithm for estimating this parameter and use it to form a spatial beam. In the case of noncoherent element-to-element fading, the fading path to each sensor is different (although possibly correlated) and no angle of arrival can be exploited for conventional beamforming. For each channel model, we develop several detection strategies which assume various amounts of prior information about the fading. We then consider blind extensions of these detectors based on subspace tracking, which do not require a prior model for the interfering users' signals.

The Rihaczek distribution was originally proposed as a second-order complex valued time-frequency correlation function for energy signals. In this paper, we show that the generalization to higher orders and to stochastic processes arises naturally from considerations of the nonstationary moment functions. These so-called higher-order Rihaczek spectra (HORS) are shown to exhibit many interesting, important, and desirable properties as measures of the time and frequency behavior of stochastic processes. We show that the n-th order HORS is to be interpreted as the distribution of complex correlation between a stochastic process and an (n − 1) dimensional (multifrequency) random field. Under stationarity assumptions, we found that the HORS collapses to the conventional stationary moment spectrum. The frequency and time marginals of the n-th order are the n-th order instantaneous moment, and the n-th order stationary moment spectrum, respectively. Finally, we evaluated the n-th order HORS for the output process from a nonstationary channel excited by a white, stationary, and non-Gaussian source, and we found that all orders of the HORS contain useful information about the time-frequency behavior of the channel.

We consider the problem of minimum mean squared error (MMSE) estimation of complex random vectors in the improper case. Accounting for the information present in the complementary covariance requires the use of widely linear transformations. Based on these, we present the eigenanalysis of improper complex random vectors. This paves the way for a study of two different rank-reduced implementations of the complex Wiener Filter that make use of canonical coordinates: one that is optimum with respect to maximizing mutual information and one that minimizes mean squared error.

We develop the minimum mean-squared-error (MMSE) multiuser detector for nonorthogonal multipulse modulation over the noncoherent additive white Gaussian noise channel. We analyze the asymptotic performance of the detector and show that, unlike the case of linear modulation, the MMSE detector does not generally approach the generalized maximum-likelihood (GML) detection rule as the noise power vanishes. It does, however, approach a detector which nulls out the multiaccess interference. This detector is termed the multipulse decorrelating detector due to its similarity to the linear decorrelating detector. The probability of error for this detector is derived and used to find the asymptotic multiuser efficiencies of both the multipulse decorrelating detector and the MMSE detector. It is shown that for noncoherent binary signaling, in which the multipulse modulation is two-dimensional, the multipulse decorrelating detector is superior to the GML detector asymptotically. This result does not generalize to larger dimensionality signal sets.

The natural models of multi-access communication and modern radar and sonar systems involve infinite-dimensional waveform spaces. A common problem in these systems is the detection of subspace signals measured in the presence of subspace interference and broadband noise. By and large, the existing theory for such problems has been developed for finite-dimensional measurement spaces rather than the infinite-dimensional waveform spaces needed here. In this paper the log of the generalized likelihood ratio detector for the waveform problem is derived and is shown to have a certain chi-squared distribution, depending on the hypothesis.

We consider the detection of nonorthogonal multipulse signals on multiple-access fading channels. The generalized maximum-likelihood rule is employed to decode users whose complex fading gains are unknown. We develop geometrical interpretations for the resulting detectors and their corresponding asymptotic efficiencies. The generalized maximum-likelihood detection rule is then applied to find a matched subspace detector for the frequency-selective fading channel, under the assumption of a short coherence time (or long coherence time without the computational power to track the fading parameters). We propose blind implementations of these detectors for nonorthogonal multipulse signaling on both frequency-nonselective and frequency-selective multiple-access fading channels. These blind detectors extend the results of Wang and Poor (see ibid., vol.44, p.677-89, 1998) to multipulse modulation and fast frequency selective fading. For comparison, the minimum mean-squared error decision rules for these channels are derived and blind implementations of their corresponding detectors are developed.

Some quadratic time-frequency representations (TFRs) may be called time-varying spectrum estimators. They are derived from first principles, and they turn out to be time-varying multiwindow spectrum estimators. In special cases they are time-varying spectrograms that may be written as Fourier transforms of lag-windowed, time-varying correlation sequences or as spectrally smoothed time-varying periodograms. These are not ad-hoc variations on stationary ideas to accommodate time variation. Rather, they are the only variations one can obtain for time-varying spectrum analysis.

The Wiener filter is analyzed for stationary complex Gaussian signals from an information theoretic point of view. A dual-port analysis of the Wiener filter leads to a decomposition based on orthogonal projections and results in a new multistage method for implementing the Wiener filter using a nested chain of scalar Wiener filters. This new representation of the Wiener filter provides the capability to perform an information-theoretic analysis of previous, basis-dependent, reduced-rank Wiener filters. This analysis demonstrates that the cross-spectral metric is optimal in the sense that it maximizes mutual information between the observed and desired processes. A new reduced-rank Wiener filter is developed based on this new structure which evolves a basis using successive projections of the desired signal onto orthogonal, lower dimensional subspaces. The performance is evaluated using a comparative computer analysis model and it is demonstrated that the low-complexity multistage reduced-rank Wiener filter is capable of outperforming the more complex eigendecomposition-based methods.

In this paper, we revisit the problem of interrelations between large correlated data sets by considering cross correlations between a few linear combinations of the elements of each. This problem was studied by Hotelling (see Biometrika, vol.28, p.321-77) and Anderson (1958). We generalize the problem by studying linear transformations of the data sets, and applying our results to the case where one of the transformed data sets is noise corrupted. We derive best reduced-rank linear transformations and present asymptotic results. We conclude the paper by studying a causal filtering version of this problem and connecting it with the asymptotic case.

The Gabor transform (or the windowed Fourier transform) is a widely used tool in signal processing. We generalize the windowed Fourier transform to the variable-windowed Fourier transform. This generalization brings the Gabor transform and the wavelet transform under the same framework. Using frame theory we characterize frames and orthonormal bases for the variable windowed Fourier series (VWFS). These characterizations are formulated explicitly in terms of window functions. Therefore they can serve as guidelines for designing windows for the VWFS. We introduce the notion of "complete orthogonal support" and, with the help of this notion, we construct a class of orthonormal VWFS bases for L/sup 2/(R/sup +/).

Reduced rank linear predictive frequency and direction-of-arrival (DOA) estimation algorithms use the singular value decomposition (SVD) to produce a noise-cleaned linear prediction vector. These algorithms then root this vector to obtain a subset of roots, whose angles contain the desired frequency or DOA information. The roots closest to the unit circle are deemed to be the "signal roots". The rest of the roots are "extraneous". The extraneous roots are expensive to calculate. Further, a search must be done to discern the signal roots from the extraneous roots. Here, we present a reduced polynomial order linear prediction method that simplifies the rooting computation for applications where high-speed processing is critical.

How does one adaptively split a measurement subspace into signal and orthogonal subspaces of reduced rank so that detectors, estimators, and quantizers may be adaptively designed from experimental data? The authors provide some answers to this question by decomposing experimental correlations into their Wishart distributed Schur complements and showing how these distributions may be used to identify subspaces.

A general representation is presented for quadratic estimators of the far-field frequency-wavenumber spectrum. The quadratic estimators are constructed from linear transformations of the array data. These linear transformations contain the windows normally associated with multiple window spectral estimators and act as multidimensional baseband beamformers. It is the selection of these windows that is most important in the design of quadratic spectral estimators. A discussion is presented of the two dimensional periodogram and the two dimensional Thomson (1982) estimator as prototypical examples of quadratic estimators.>

The authors introduce a fast algorithm for computing the QR factors of a complex vertically circulant matrix C. The complexity of the algorithm is 5mn+2n/sup 2/+O(m) complex-fixed point operations or 3mn+n/sup 2/+O(m) complex floating-point operations, where m is the number of rows in C and n is the number of columns (it is supposed that m>

The authors introduce a fast algorithm for computing the QR factors of a complex column Vandermonde matrix V. The complexity of the algorithm is 5 mn+7 n/sup 2//2+O(m), where m is the number of rows in V and n is the number of columns (they assume that m>n). The matrices Q and R can be computed independently if desired. Such an algorithm allows an important saving when solving such systems in the least-squares sense, as for example when estimating the magnitude and phase of damped exponentials in the least-squares version of the Prony method.>

The authors pose a sequence of linear prediction problems. By solving this sequence of problems they are able to QR factor all of the data matrices usually associated with correlation, pre-windowed and post-windowed, and covariance methods of linear prediction. Their solutions cover the forward, backward, and forward-backward problems. The QR factor orthogonalizes the data matrix and solves the problem of Cholesky factoring the experimental correlation matrix and its inverse. This means they can use generalized Levinson algorithms to derive generalized QR algorithms, which are then used to derived generalized Schur algorithms. All three algorithms are true lattice algorithms that can be implemented either on a vector machine or on a multiline lattice, and all three algorithms generate generalized reflection coefficients that may be used for filtering or classification.>

The authors study the finite arithmetic properties of the Schur and the lattice recursions. They derive the error variance of the reflection coefficient at each stage of each algorithm and present experimental results which agree very closely with the analytical results. They also show that the lattice recursions have a significant accuracy advantage over the Schur recursions when the problem is ill-conditioned or, equivalently, when the absolute values of the reflection coefficients are close to one.>

We review the coupled vector recursions for the Levinson, Cybenko and LeRoux-Gueguen algorithms that Cholesky and QR factor Toeplitz matrices. We generalize the algorithms to include close-to-Toeplitz matrices, and show that these generalized algorithms may be imbedded in the same vector recursions. Then, depending on how the initial conditions are set, and how the reflection coefficients are computed, one gets either generalized Levinson recursions [1], generalized LeRoux-Gueguen recursions, or generalized Cybenko recursions. The Cybenko recursions produce an interesting way to compute the generalized reflection coefficients and also lead to generalized LeRoux-Gueguen recursions for all the cases of linear prediction.

The Levinson recursions, the Cybenko recursions and the Le Roux-Gueguen recursions provide alternative ways of computing reflection coefficients for stationary time series. In this paper we show that when a Toeplitz correlation matrix is a product of two Toeplitz data matrices, as in the correlation method of linear prediction, then the Levinson recursions may be used to derive the Cybenko recursions, and the Cybenko recursions may be used to derive the Le Roux-Gueguen recursions. We explore the close relation between QR and Cholesky algorithms in the Toeplitz case and we compare their respective numerical properties when run in finite precision arithmetic.

In spectral estimation spectra are usually derived from an AR or MA model fitted to the data. An implicit step common to these methods is the estimation of the correlation matrix. In this paper our approach consists in doing maximum likelihood identification of structured correlation matrix. We have used two different structures corresponding to Toeplitz matrices and to matrices with DFT representation. Both structures are related to time invariant series. We have studied the performances of the spectral estimates obtained from our correlation matrix. In particular we show mean square error versus SNR plots for the frequency estimation of two noisy sinusoids.

The communication problems of phase tracking [1-3], convolutional decoding [4,5], and dynamic time warping [6], can all be solved with a Dynamic Programming algorithm to find the shortest path through a graph. We present a modular systolic architecture for the specific problem of forming the maximum a posteriori (MAP) estimate of the sequence of visited states of an M-state Markov chain, using noisy observations of a memoryless function of the chain. The architecture is composed of three main sections: (1) a Metric Processor which pre-processes each input datum into M input metrics, (2) a Likelihood Processor which uses the input metrics and the apriori Markov model to update the likelihood measures of the M implicit survivor paths, and (3) a Path Processor which stores the array of back-pointers used to explicitly construct the single most likely survivor path. The likelihood processor is composed of a systolic ring of M simple processing elements which perform the O(M2) computations for each input in O(M) time. If the length N of the sequence is long, the pipe-lined path processor can implement sub-optimal periodic path truncation decoding, which includes the familiar fixed-lag and fixed-interval decoding.

This paper describes a coding strategy which stores digital images in terms of their contours. The data in the original raw image is reorganised into a set of 1-d trajectories and these are normalised in length and approximated by low rank models. It is the coefficients of these models which are retained. The second order statistical information is used to derive a vector codebook containing standard sets of coefficients, and each contour is allocated the codeword of the nearest standard vector. In this way a codebook of generating functions for standard shapes is obtained. The problems of non closing contours are addressed along with interpolation methods to produce a reconstructed image from a contoured image.

The classical result of Lloyd for the optimum scalar quantizer in the asymptotic case of fine quantization is derived from first principles. The derivation is offered as a simple alternative to Lloyd's original and elegant piece of analysis, and the result is used to derive the optimum compander. We then show why a compander that presents uniformly distributed random variables to the quantizer is not a good idea.

A large class of physical signals may be characterized by mode parameters (natural frequencies and damping coefficients) and initial conditions. The mode parameters, usually governed by well understood physical models, are often accurately known apriori. Conversely, very little is usually known about initial conditions. The problem of interest here is to detect signals with known modes, but unknown initial conditions, in additive Gaussian noise of unknown level. The signals may also be characterized as impulse responses of ARMA systems with unknown MA parts. We derive an F-statistic that is optimum (uniformly most powerful) in the class of all receivers that are invariant to certain tranformations of the data. We argue that the invariances are natural constraints. The statistic we derive provides constant false alarm rate performance.

A general framework for deriving and interpreting analysis and synthesis spectra of the autoregressive (AR) and moving average (MA) type is presented. Investigation of AR linear transformations of finite dimensional data records yields a set of intermediate MA techniques associated with approximation of the inverse correlation matrix R-1. The corresponding spectrum we call a parameterized maximum likelihood method (pMLM) spectrum. Investigation of MA linear transformations yields a set of intermediate MA techniques associated with approximation of the correlation matrix R. The corresponding spectrum we call a parameterized Bartlett spectrum (pBA). Simulations on synthetic AR, MA and ARMA data sets illustrate the techniques and lead to interesting remarks concerning the use of parameterizations of R and R-1to differentiate between data sets of AR and MA type.

In this paper we make the point that a wide variety of spectrum types admit to modal analysis wherein the modes are characterized by amplitudes, frequencies, and damping factors. The associated modal decomposition is appropriate for both continuous and discrete components of the spectrum. The domain of attraction for the decomposition includes ARMA sequences, harmonically- or nonharmonically-related sinusoids, damped sinusoids, white noise, and linear combinations of these. Numerical results are presented to illustrate the identification Of mode parameters and corresponding spectra from finite records of perfect and estimated covariance sequences. The results for sinusoids and sinusoids in white noise are interpreted in terms of inphase and quadrature effects attributable to the finite record length.

The problem of simultaneously estimating phase and decoding data symbols from baseband data is posed. The phase sequence is assumed to be a random sequence on the circle, and the symbols are assumed to be equally likely symbols transmitted over a perfectly equalized channel. A dynamic programming algorithm (Viterbi algorithm) is derived for decoding a maximum {\em a posteriori} (MAP) phase-symbol sequence on a finite dimensional phase-symbol trellis. A new and interesting principle of Optimality for simultaneously estimating phase and decoding phase-amplitude coded symbols leads to an efficient two-step decoding procedure for decoding phase-symbol sequences. Simulation results for binary,8-ary phase shift keyed (PSK), and 16-quadrature amplitude shift keyed (QASK) symbol sets transmitted over random walk and sinusoidal jitter channels are presented and compared with results one may obtain with a decision-directed algorithm or with the binary Viterbi algorithm introduced by Ungerboeck. When phase fluctuations are severe and when occasional large phase fluctuations exist, MAP phase-symbol sequence decoding on circles is superior to Ungerboeck's technique, which in turn is superior to decision-directed techniques.

In this paper we explore an alternative solution to the general nonlinear least squares approximation problem for digital filter design. We choose to approximate covariance sequences because they arise more naturally than unit pulse sequences when approximating spectra and when identifying ARMA sequences from random data. The design procedure resulting from this approach requires solution of a nonlinear algebraic equation in the form of a polynomial rootfinding problem, in addition to solving a system of linear equations and an Inverse Discrete Fourier Transform (IDFT).

The Szegö-Kolmogorov-Krein theorem is the natural basis for the spectral flatness measure commonly advocated in linear predictive speech processing and parametric spectrum analysis. From this theorem it follows that the logarithm of any normalized spectrum averages to zero. The normalization constant is the minimum prediction error for the underlying process. Jensen's theorem is introduced as a practical method of computing prediction error and spectral flatness for rational spectrum models. The resulting computation algorithm is closely related to Fejer's factorization of rational spectra. Thus our discussion generalizes Makhoul's discussion of the so-called Fejer method for computing prediction error in MA models. One property of any measure of spectral flatness that would seem essential is the following: the introduction of white noise should increase spectral flatness. This property is established by appealing to a classical information-theoretic theorem.

The probabilistic evolution of random walk on the circle is studied, and the results are used to derive a maximum {\em a posteriori} probability (MAP) sequence estimator for phase. The sequence estimator is a Viterbi tracker for tracking phase on a finite-dimensional grid in[-\pi,\pi). The algorithm is shown to provide a convenient method for obtaining fixed-lag phase estimates. Performance characteristics are presented and compared with several published nonlinear filtering algorithms.

Procedures are given for the systematic design of ARMA filters. All of the design algorithms are linear and of the Levinson type. Each design is initiated with a long AR approximation of an ideal spectrum. The long AR is used to generate consistent unit pulse and covariance sequences for use in the Levinson type algorithm of Mullis and Roberts. The latter algorithm allows one to approximate the unit pulse and covariance sequences and thereby obtain a low-order ARMA approximation. There is no need for the ARMA approximation to be proper.

A variety of likelihood ratios are derived for detecting Gauss-Markov and finite-state Markov sequences in additive Gaussian noise. The Bayesian recursions appropriate to related filtering problems are exploited, together with "known-form" likelihood ratios, to obtain the desired results. In the derivation of a discrete-time Gauss-Markov likelihood ratio, a "pure" causal estimator-correlator structure is sought and a "locally stable" state estimator is encountered that is of some interest in its own right. The likelihood ratio is "pure" in the sense that the locally stable estimator is used in precisely the same manner as the stored replica is used in known-form signal detection problems to form the likelihood ratio. Consequently, the likelihood ratio is devoid of the extra data-dependent term that arises whenever one uses least squares state estimators to form the likelihood ratio statistic. The locally stable estimator equalizes, within a constant related to the {\em a priori} and {\em a posteriori} filtering error covariances, the {\em a priori} and {\em a posteriori} filtering densities. Heuristically, the estimator is a compromise between the one-step predictor and the filtered estimator of a discrete-time Kalman filter. When the observation noise covariance is unknown, a generalization of the so-called unknown level problem, then a Wishart prior is assigned to the innovations covariance and an integral representation is obtained for the desired likelihood ratio. The representation suggests a parallel structure for approximating the likelihood ratio when the observation noise covariance is unknown. Finally, the likelihood ratio for detecting finite-state Markov sequences is derived to illustrate that in general no "pure" estimator-correlator structure can exist when the state-space is finite.

When discretizing continuous-time systems or signals, one is often interested in preserving a property termed covariance-invariance. In this paper a technique is outlined for synthesizing discrete-time systems and signals which are covariance-invariant with corresponding continuous-time systems and signals. Applications of the technique to process simulation, minimum mean-squared error estimation, and digital filter synthesis are outlined, with example designs presented for covariance-invariant Butterworth and Chebychev digital filters. Based on the frequency response of these designs it is argued that the method of covariance-invariance is superior to the methods of impulse-invariance and bilinear-z as a response matching design technique for the synthesis of digital filters. This superiority is especially apparent at sampling rates that are marginal with respect to filter critical frequencies.

Information measures and performance bounds are derived for frequency-domain linear array processors deployed in homogeneous Gaussian random fields.J-divergence, a measure of the (net) information rate of an array, is shown to be a useful measure of how effectively detection and estimation functions can be performed in optimum and conventional array processing structures. In a detection context,J- divergence becomes a detection index that can be interpreted in terms of array gain and output signal-to-noise ratio (SNR). Comparisons between the divergence of optimum and conventional processors indicate, for example, that optimum processing can provide on the order of a 13 dB gain over conventional processing when trying to detect a 20 dB signal in the presence of a 20 dB interference located within the Rayleigh limit of the array. In an estimation context, J-divergence can be used to derive "critical divergence" and Cramér-Rao bounds on resolution variance. These bounds indicate that approximately 25 dB output signal-to-noise ratio is required to obtain a 10:1 improvement over the classical Rayleigh resolution limit. The Rayleigh limit is argued to have significance only at output SNR's of approximately 10 dB. The argument is based on a new resolution limit termed the critical divergence limit. This limit is shown to give resolution limits approximately three times the Cramér-Rao bound, indicating that the latter bound is perhaps an optimistic resolution limit.

The detection of information-bearing Gaussian processes immersed in additive white Gaussian noise (WGN) is an important problem that arises in many signal processing applications. When the level of the WGN is unknown, classical approaches to the problem fail. In this paper a principle of invariance is used to derive a detector with performance that is invariant (or insensitive) to system gain, or equivalently channel attenuation. The detector structure can be realized and detection thresholds set without prior knowledge of the WGN level. When the observation interval is large the detector has the structure of a spectral estimator-correlator, the output of which is compared to an adaptive threshold. The invariance feature of the detector makes it a constant false alarm rate (CFAR) receiver; an ad hoc structure for suboptimal CFAR Gauss-Gauss detection is discussed as well.

The concept of invariance in hypothesis testing is brought to bear on the problem of detecting signals of known form and unknown energy in Gaussian noise of unknown level. The noise covariance function is assumed to beK(t,u) = \sigma^2 \pho(t,u)where\rho(t,u)is the known form of the covariance function and\sigma^2is the unknown level. Classical approaches to signal detection depend on the assumption thatK(t,u)is known completely. Then, a correlation-type receiver that is the uniformly most powerful (UMP) test ofH_o(signal absent) versusH_1(signal present) can be derived. When\sigma^2is unknown, there exists no UMP test. However, it is shown in this paper that there exists a test ofH_oversusH_1that is UMP-invariant for a very natural group of transformations on the space of observations. The derived test is found to be independent of knowledge about the noise level\sigma^2, since the derived test (receiver) contains an error-free estimate of\sigma^2. This utopian conclusion is reconciled by noting that the derived receiver can never be physically realized. It is shown that any physically realizable version of the receiver has at-distributed test statistic. This permits choice of operating receiver thresholds and evaluation of performance characteristics.