We consider the classical detection problem, of deciding whether a received signal consists of noise only or of a known signal of interest embedded in noise. The classical tool used for such problems is the Matched Filter (MF), which, under the assumption of Gaussian noise, provides the optimal decision rule via the Likelihood Ratio Test (LRT). However, when the noise is non-Gaussian, and, in particular, when the noise is skewed, the MF’s deviation from optimality may become significant. Moreover, often the full probability distribution of the noise is unknown, so the LRT cannot be applied. In this work we proposed a Modified MF (MOMAF), which is based on a "first-order modification" of the LRT, using the Gram-Charlier expansion in terms of the skewness parameter of the noise (assumed to be known). The MOMAF takes an implementation-friendly form, combining multipliers and two Linear, Time-Invariant filters, and reduces to the classical Matched Filter when the noise skewness is zero. We demonstrate the performance improvement of the MOMAF with various distributions of skewed noise in simulation.

We consider the blind separation of noisy mixtures of independent sources over a finite field. Namely, the source signals, the elements of the mixing matrix, the noise signals, the noisy output signals and the associated arithmetic operations all reside in a finite (Galois) field. The source signals are assumed to be mutually independent and temporally stationary with unknown probability distributions, and the goal is to estimate the unknown mixing matrix based on the observed (noise-contaminated) output signals only. Previous work on this problem only considered the noiseless case, and several separation approaches have been proposed. In this work we address the more challenging noisy case, where we assume that each of the observed mixture signals is contaminated by independent additive noise (over the field), reflected by occasional symbol errors. To this end, we propose a modification of the "Ascending Minimization of EntRopies for ICA" ("AMERICA") algorithm. The modified version (dubbed "AMERICANO" - "AMERICA" with NOise) accounts for the noise through mitigation of the empirical characteristic tensor of the observations. We demonstrate the loss of equivariance inflicted on AMERICA by the noise, as well as the resulting improvement by AMERICANO.

Anti-spoofing is the task of speech authentication. That is, identifying genuine human speech compared to spoofed speech. The main focus of this paper is to suggest new representations for genuine and spoofed speech, based on the probability mass function (PMF) estimation of the audio waveforms' amplitude. We introduce a new feature extraction method for speech audio signals: unlike traditional methods, our method is based on direct processing of time-domain audio samples. The PMF is utilized by designing a feature extractor based on different PMF distances and similarity measures. As an additional step, we used filterbank preprocessing, which significantly affects the discriminative characteristics of the features and facilitates convenient visualization of possible clustering of spoofing attacks. Furthermore, we use diffusion maps to reveal the underlying manifold on which the data lies. The suggested embeddings allow the use of simple linear separators to achieve 12.99% Equal Error Rate (EER) on ASVspoof2019 logical Access (LA) test set for female samples, and 12.09% for male samples. In addition, we present a convenient way to visualize the data, which helps to assess the efficiency of different spoofing techniques. Furthermore, we present reduced complexity embedding method by using compander quantization, which in some cases even improves the EER on the test set up to 3.00%. The experimental results show the potential of using multichannel PMF-based features for the anti-spoofing task, in addition to the benefits of using diffusion maps both as an analysis tool and as an embedding tool.

Probability mass function (PMF) estimation using a low-rank model for the PMF tensor has gained increased popularity in recent years. However, its performance evaluation relied mostly on empirical testing. In this work, we derive theoretical bounds on the attainable performance under this model assumption. We begin by deriving the constrained Cramér-Rao Bound (CCRB) on the low-rank decomposition parameters, and then extend the CCRB to bounds on the mean square error in the resulting estimates of the PMF tensor’s elements, as well as on the mean Kullback-Leibler divergence (KLD) between the estimated and true PMFs. The asymptotic tightness of these bounds is demonstrated by comparing them to the performance of the Maximum Likelihood estimate in a small-scale simulation example.

We consider the blind estimation / equalization of a Moving Average (MA) channel over a finite field. In this framework, the channel’s input and output signals, as well as its coefficients, belong to a finite (Galois) field, and all summation and multiplication operations are calculated modulo the field’s prime order. The input is assumed to be a sequence of independent, identically distributed (iid) samples with an unknown distribution, and the goal is to estimate the channel coefficients based on its observed output only. We derive two different estimation approaches: One is based on sequential identification of factors of the channel’s associated polynomial; The other is based on an attempted factorization of the empirical characteristic function of the channel’s output signal. We explain the trade-offs between the methods and demonstrate their performance by simulation.

It is common to assess the "memory strength" of a stationary process by looking at how fast the normalized log– determinant of its covariance submatrices (i.e., entropy rate) decreases. In this work, we propose an alternative characterization in terms of the normalized trace–inverse of the covariance submatrices. We show that this sequence is monotonically non-decreasing and is constant if and only if the process is white. Furthermore, while the entropy rate is associated with one-sided prediction errors (present from past), the new measure is associated with two-sided prediction errors (present from past and future). Minimizing this measure is then used as an alternative to Burg’s maximum-entropy principle for spectral estimation.

It is common to assess the “memory strength” of a stationary process by looking at how fast the normalized log–determinant of its covariance submatrices (i.e., entropy rate) decreases. In this work, we propose an alternative characterization in terms of the normalized trace–inverse of the covariance submatrices. We show that this sequence is monotonically non-decreasing and is constant if and only if the process is white. Furthermore, while the entropy rate is associated with one-sided prediction errors (present from past), the new measure is associated with two-sided prediction errors (present from past and future). Minimizing this measure is then used as an alternative to Burg’s maximum-entropy principle for spectral estimation. We also propose a counterpart for non-stationary processes, by looking at the average trace–inverse of subsets.

One-bit quantization has recently become an attractive option for data acquisition in cutting edge applications, due to the increasing demand for low power and higher sampling rates. Subsequently, the rejuvenated one-bit array processing field is now receiving more attention, as "classical" array processing techniques are adapted / modified accordingly. However, array calibration, often an instrumental preliminary stage in array processing, has so far received little attention in its one-bit form. In this paper, we present a novel solution approach for the blind calibration problem, namely, without using known calibration signals. In order to extract information within the second-order statistics of the quantized measurements, we propose to estimate the unknown sensors' gains and phases offsets according to a Kullback-Leibler Divergence (KLD) covariance fitting criterion. We then provide a quasi-Newton solution algorithm, with a consistent initial estimate, and demonstrate the improved accuracy of our KLD-based estimates in simulations.

Blind calibration of sensors arrays (without using calibration signals) is an important, yet challenging problem in array processing. While many methods have been proposed for "classical" array structures, such as uniform linear arrays, not as many are found in the context of the more "modern" sparse arrays. In this paper, we present a novel blind calibration method for 2-level nested arrays. Specifically, and despite recent contradicting claims in the literature, we show that the Least-Squares (LS) approach can in fact be used for this purpose with such arrays. Moreover, the LS approach gives rise to optimally-weighted LS joint estimation of the sensors' gains and phases offsets, which leads to more accurate calibration, and in turn, to higher accuracy in subsequent estimation tasks (e.g., direction-of-arrival). Our method, which can be extended to K-level arrays (K > 2), is superior to the current state of the art both in terms of accuracy and computational efficiency, as we demonstrate in simulation.

We propose a novel framework called IDEA that exploits electromagnetic (EM) side-channel signals to detect malicious activity on embedded and cyber-physical systems (CPS). IDEA first records EM emanations from an uncompromised reference device to establish a baseline of reference EM patterns. IDEA then monitors the target device's EM emanations. When the observed EM emanations deviate from the reference patterns, IDEA reports this as an anomalous or malicious activity. IDEA does not require any resource or infrastructure on, or any modification to, the monitored system itself. In fact, IDEA is isolated from the target device, and monitors the device without any physical contact. We evaluate IDEA by monitoring the target device while it is executing embedded applications with malicious code injections such as Distributed Denial of Service (DDoS), Ransomware and code modification. We further implement a control-flow hijack attack, an advanced persistent threat, and a firmware modification on three CPSs: an embedded medical device called SyringePump, an industrial Proportional-Integral-Derivative (PID) Controller, and a Robotic Arm, using a popular embedded system, Arduino UNO. The results demonstrate that IDEA can detect different attacks with excellent accuracy (AUC > 99.5%, and 100 percent detection with less than 1 percent false positives) from distances up to 3 m.

We study the blind calibration problem of uniform linear arrays of acoustic vector sensors for narrowband Gaussian signals, and propose an improved, asymptotically optimal blind calibration scheme. Following recent work by Ramamohan et al., we exploit the special (block-Toeplitz) structure of the underlying signals' spatial covariance matrix. However, we offer a substantial improvement over their ordinary Least Squares (LS)-based approach: Using asymptotic approximations we obtain Optimally-Weighted LS estimates of the sensors' gains and phases offsets. We show via simulations that our estimates exhibit near-optimal performance, with improvements reaching more than an order of magnitude in the mean squared estimation errors of the calibration parameters, as well as in directions of-arrival estimation.

We address the problem of source separation from noisy mixtures in a semi-blind scenario, with stationary, temporally-diverse Gaussian sources and known spectra. In such noisy models, a dilemma arises regarding the desired objective. On one hand, a "maximally separating" solution, providing the minimal attainable Interference-to-Source-Ratio (ISR), would often suffer from significant residual noise. On the other hand, optimal Minimum Mean Square Error (MMSE) estimation would yield estimates which are the "least distorted" versions of the true sources, often at the cost of compromised ISR. Based on Maximum Likelihood (ML) estimation of the unknown underlying model parameters, we propose two ML-based estimates of the sources. One asymptotically coincides with the MMSE estimate of the sources, whereas the other asymptotically coincides with the (unbiased) "least-noisy maximally-separating" solution for this model. We prove the asymptotic optimality of the latter and present the corresponding Cramér-Rao lower bound. We discuss the differences in principal properties of the proposed estimates and demonstrate them empirically using simulation results.

A novel blind estimate of the number of sources from noisy, linear mixtures is proposed in this letter. Based on Székely et al.'s distance correlation measure, we define the sources' dependence criterion (SDC), from which our estimate arises. Unlike most previously proposed estimates, the SDC estimate exploits the full independence of the sources and noise, as well as the non-Gaussianity of the sources (as opposed to the Gaussianity of the noise), via implicit use of high-order statistics. This leads to a more robust, resilient, and stable estimate w.r.t. the mixing matrix and the noise covariance structure. Empirical simulation results demonstrate these virtues on top of superior performance in comparison with current state-of-the-art estimates.

We consider the problem of estimating the Probability Mass Function (PMF) of a discrete random vector (RV) from partial observations, namely when some elements in each observed realization may be missing. Since the PMF takes the form of a multi-way tensor, under certain model assumptions the problem becomes closely associated with tensor factorization. Indeed, in recent studies it was shown that a low-rank PMF tensor can be fully recovered (under some mild conditions) by applying a low-rank (approximate) joint factorization to all estimated joint PMFs of subsets of fixed cardinality larger than two (e.g., triplets). The joint factorization is based on a Least Squares (LS) fit to the estimated lower-order sub-tensors. In this letter we take a different estimation approach by fitting the partial factorization directly to the observed partial data in the sense of Kullback-Leibler divergence (KLD). Consequently, we avoid the need for particular selection and direct estimation of sub-tensors of a particular order, as we inherently apply proper weighting to all the available partial data. We show that our approach essentially attains the Maximum Likelihood estimate of the full PMF tensor (under the low-rank model) and therefore enjoys its well-known properties of consistency and asymptotic efficiency. In addition, based on the Bayesian model interpretation of the low-rank model, we propose an Estimation-Maximization (EM) based approach, which is computationally cheap per iteration. Simulation results demonstrate the advantages of our proposed KLD-based hybrid approach (combining alternating-directions minimization with EM) over LS fitting of sub-tensors.

Tensor decompositions are regarded as a powerful tool for multidimensional signal processing. In this contribution, we focus on the well-known Canonical Polyadic (CP) decomposition and present a first-order perturbation analysis of the SEmi-algebraic framework for approximate CP decompositions via SImultaneous matrix diagonalization with Generalized Unfoldings (SECSI-GU), which is advantageous for tensors of an order higher than three. Numerical results indicate that the analytical relative Mean Square Factor Error (rMSFE) of the estimated factor matrices resulting from each generalized unfolding considered in SECSI -GU matches the empirical rMSFE very well. As SECSI -GU considers all possible partitionings of the tensor modes resulting in a large number of candidate factor matrix estimates, an exhaustive search-based criterion to select the final factor matrix estimates leads to a prohibitive computational complexity. The accurate performance prediction achieved by the first-order perturbation analysis conducted in this paper will significantly facilitate the selection of the final factor matrix estimates in an efficient manner and will therefore contribute to a low-complexity enhancement of SECSI-GU.

Electrocardiography (ECG) measured using wearable wireless sensors is already commonly used for several years, as one of the products of the emerging Telemedicine field, which is one the main branches in eHealth applications. In this work we address the problem of missing samples recovery of such ECG (digital) signals, resulting from temporally-local communication dropouts. We propose a new model for the ECG signal based on its conspicuous quasi-periodical characteristics in short time intervals, along with a compatible estimation procedure tailored to the proposed model. We extend the autoregressive (AR) model, previously proposed by Prieto-Guerrero et al., to a cyclostationary AR model, and our proposed estimation scheme incorporates a first phase of model parameters estimation, followed by a Linear Minimum Mean Squared Error (LMMSE) estimation phase of the missing samples. We demonstrate significant improvement compared to the AR method in simulation experiments using real ECG data.

In its basic, fully blind form, Independent Component Analysis (ICA) does not rely on a particular statistical model of the sources, but only on their mutual statistical independence, and therefore does not admit a Maximum Likelihood (ML) estimation framework. In semi-blind scenarios statistical models of the sources are available, enabling ML separation. Quasi-ML (QML) methods operate in the (more realistic) fully-blind scenarios, simply by presuming some hypothesized statistical models, thereby obtaining QML separation. When these models are (or are assumed to be) Gaussian with distinct temporal covariance matrices, the (quasi-)likelihood equations take the form of a “Sequentially Drilled Joint Congruence” (SeDJoCo) transformation problem. In this work we state some mild conditions on the sources' true and presumed covariance matrices, which guarantee consistency of the QML separation when the SeDJoCo solution is asymptotically unique. In addition, we derive a necessary “Mutual Diversity” condition on these matrices for the asymptotic uniqueness of the SeDJoCo solution. Finally, we demonstrate the consistency of QML in various simulation scenarios.

Joint EigenValue Decomposition (JEVD) algorithms are widely used in many application scenarios. These algorithms can be divided into different categories based on the cost function that needs to be minimized. Most of the frequently used algorithms in the literature use indirect least square (LS) criteria as a cost function. In this work, we perform a first order perturbation analysis for the JEVD algorithms based on the indirect LS criterion. We also present closed-form expressions for the eigenvector and eigenvalue matrices. The obtained expressions are asymptotic in the signal-to-noise ratio (SNR). Additionally, we use these results to obtain a statistical analysis, where we only assume that the noise has finite second order moments. The simulation results show that the proposed analytical expressions match well to the empirical results of JEVD algorithms which are based on the LS cost function.

The “Sequentially Drilled” Joint Congruence (SeDJoCo) transformation is a set of matrix transformation equations, which coincide with the Likelihood Equations for semi-blind source separation, when each source is modeled as a zero-mean Gaussian process with a known (and distinct) temporal covariance matrix. Therefore, with such a model a solution of SeDJoCo can lead to the Maximum Likelihood (ML) estimate of the separating matrix, which is asymptotically optimal. However, as we have shown in previous work, multiple solutions of SeDJoCo may exist, and the selection of the optimal solution among these (corresponding to the global maximum of the likelihood function) is therefore of considerable interest. In this paper we further extend our results by proposing a new ML approach for the identification and correction of a sub-optimal solution, assuming sources of unrestricted, general temporal covariance structures. We demonstrate the resulting improvement in simulation with non-stationary sources.

This paper presents a new perspective of beamforming designs in Coordinated Multi-Point (CoMP) downlink systems that are combined with multicast schemes. The beamformer computation is expressed as a joint matrix transformation that can be regarded as an extension of the "Sequentially Drilled" Joint Congruence (SeDJoCo) decomposition. A solution of the proposed joint matrix transformation is devised that takes into account the elimination of the multiuser interference as well as the maximization of the desired signal components. Therefore, it leads to a very effective semi-algebraic solution of beamforming designs for the multicast CoMP downlink, which is evident in the numerical simulations.

In the context of Maximum Likelihood (ML) source separation in a semi-blind scenario, where the spectra of the sources are known and distinct, the likelihood equations amount to a set of matrix decompositions (known as the "Sequentially Drilled" Joint Congruence Transformation (SeDJoCo)). However, quite often multiple solutions of SeDJoCo exist, only one of which is the optimal solution, corresponding to the global maximum. In this paper we characterize the different solutions and propose a procedure for detecting whether a given solution is sub-optimal. Moreover, for such sub-optimal solutions we propose a procedure for re-initializing an iterative solver so as to converge to the optimal solution. Using simulation, we present the empirical probability to encounter a sub-optimal solution (by a given iterative algorithm), as well as the resulting separation improvement when applying our proposed re-initialization approach in such cases.

We consider self-localization in an ad-hoc, asynchronous sensors network. A mobile beacon transmits a short wideband signal from a few locations, unknown to the sensors. Each of the sensors receives the transmissions and estimates their Times of Arrival (TOAs) relative to its own timebase, which has an unknown relative synchronization offset. If the positions of the beacon were known, each sensor could estimate its own time-offset and position. Since the beacon's positions are unknown, the sensors need to collaborate in order to estimate these positions along with their own. We propose a collaborative iterative scheme, where in each iteration each sensor announces its current estimate of the beacon's positions, along with an associated uncertainty covariance matrix. This information is received by neighboring sensors, and each sensor exploits the received information to refine its own estimates of the beacon's positions, as well as of its own time-offset and position. We show simulation results indicating successful self-localization using this scheme.

We consider the problem of blind identification and equalization of a Linear, Time-Invariant (LTI) system, where the input and output signals, as well as the linear operations, all reside in a finite (Galois) field. We point out some fundamental differences from the classical version of this problem. We show that if the input process is a sequence of independent, identically distributed random variables, the system is identifiable if and only if the (marginal) distribution of the input is non-uniform. For an autoregressive (AR) channel a finite impulse response equalizer can be found by minimizing the marginal entropy of its output signal. However, an exhaustive search for the minimizing equalizer, although theoretically possible, is not necessary: Based on somewhat surprising properties of the AR channel's output (not shared by the classical case), we show that the equalizer can be found directly from the empirical characteristic tensor of this output. We demonstrate the success rate of the proposed methods in simulation.

Traditional passive localization based on Time-Difference of Arrival (TDOA) or Frequency-Difference of Arrival (FDOA) usually involves several remote sensors, which require precise time-synchronization and frequency-locking among them. The need for such time or frequency alignment sometimes poses a serious operational challenge on the system. In addition, it is often desired to keep the number of sensors to a minimum. In this work we look into the operationally-simplest scenario in this context: using only two sensors, without any synchronization or locking. When at least one of the sensors, or the transmitting target, is moving at some considerable speed, it is still possible to localize the target, based on a few TDOA and / or FDOA measurements, by considering the time- and frequency-offsets as additional unknown parameters. We analyze the associated performance bound and propose a Maximum Likelihood estimation approach. The attainable accuracy and its dependence on geometry are demonstrated numerically and in simulation.

Passive estimation of the Time-Difference of Arrival (TDOA) of a common signal at two (or more) sensors is a fundamental problem in signal processing, with applications mainly in emitter localization. A common approach to TDOA estimation is the maximization of the sample cross-correlation between the received signals. For various reasons, this correlation is sometimes computed via the frequency-domain, following a Discrete Fourier Transform (DFT) of the signals - in which case the linear correlation is essentially replaced with a cyclic correlation. Although the two computations differ merely by some relatively short “edge-effects”, these edge-effects can entail more impact than commonly predicted by their relative (usually negligible) effective durations. In this work we analyze the mean square TDOA estimation error resulting from the use of cyclic instead of linear correlations, showing that for some signals the loss can be more severe than what would be predicted by a simple linear dependence on the delay value.

We consider the blind separation of sources with general (e.g., not necessarily stationary) temporal covariance structures. When the sources' temporal covariance matrices are known, the maximum-likelihood (ML) separation scheme (for Gaussian sources) conveniently exploits this knowledge. However, in the more practical case, when these matrices are unknown, ML separation calls for their estimation from the available observations. When multiple snapshots of the mixtures are available (synchronized to some external stimulus), such estimation is possible, but might require a huge number of snapshots for attaining reasonable accuracy. Rather than estimate high-dimensional covariance matrices, we propose here a more practical (“partial”-ML) approach, based on estimation of much smaller covariance matrices. These are covariances of low-dimensional vectors, consisting of respective off-diagonal terms of spatial sample-correlation matrices. Weighted joint diagonalization of these correlation matrices (using the estimated low-dimensional covariances for the weighting) significantly improves the separation performance over alternative options, as we demonstrate in simulation.

We consider the framework of Independent Component Analysis (ICA) for the case where the independent sources and their linear mixtures all reside in a Galois field of prime orderP. Similarities and differences from the classical ICA framework (over the real field) are explored. We show that a necessary and sufficient identifiability condition is that none of the sources should have a uniform distribution. We also show that pairwise independence of the mixtures implies their full mutual independence (namely a nonmixing condition) in the binary (P=2) and ternary (P=3) cases, but not necessarily in higher order (P>; 3) cases. We propose two different iterative separation (or identification) algorithms: One is based on sequential identification of the smallest-entropy linear combinations of the mixtures and is shown to be equivariant with respect to the mixing matrix; the other is based on sequential minimization of the pairwise mutual information measures. We provide some basic performance analysis for the binary (P=2) case, supplemented by simulation results for higher orders, demonstrating advantages and disadvantages of the proposed separation approaches.

We consider passive joint estimation of the time-difference of arrival (TDOA) and frequency-difference of arrival (FDOA) of an unknown signal at two sensors. The classical approach for deriving the Cramér-Rao bound (CRB) in this context assumes that the signal (as well as the noise) is Gaussian and stationary. As a result, the obtained Fisher information matrix with respect to the TDOA and FDOA is diagonal, implying that the respective estimation errors are uncorrelated (under asymptotic conditions). However, for some specific (non-Gaussian, non-stationary) signals, especially chirp-like signals, these errors can be strongly correlated. In this work we derive a “signal-specific” (or a “conditional”) CRB for this problem: Modeling the signal as a deterministic unknown, we obtain a bound which, given any particular signal, can reflect the possible signal-induced correlation between the TDOA and FDOA estimates. We further demonstrate that this bound is instrumental for proper weighting when combining joint TDOA and FDOA estimates from independent intervals.

We consider the problem of time difference of arrival (TDOA) estimation for cyclostationary signals in additive white Gaussian noise. Classical approaches to the problem either ignore the cyclostationarity and use ordinary cross-correlations, or exploit the cyclostationarity by using cyclic cross-correlations, or combine these approaches into a multicycle approach. Despite contradicting claims in the literature regarding the performance-ranking of these approaches, there has been almost no analytical comparative performance study. We propose to regard the estimated (ordinary or cyclic) correlations as the ldquofront-endrdquo data, and based on their asymptotically Gaussian distribution, to compute the asymptotic Cramer-Rao bounds (CRB) for the various combinations (ordinary/single-cycle/multi-cycle). Using our cyclic-correlations-based CRB (termed ldquoCRBCRBrdquo), we can bound the performance of any (unbiased) estimator which exploits a given set of correlations. Moreover, we propose an approximate maximum likelihood estimator (with respect to the correlations), and show that it attains our CRBCRB asymptotically in simulations, outperforming the competitors.

We propose a novel blind source separation algorithm called Block AutoRegressive Blind Identification (BARBI). The algorithm is asymptotically efficient in separation of instantaneous linear mixtures of blockwise stationary Gaussian autoregressive processes. A novel closed-form formula is derived for a Cramér Rao lower bound on elements of the corresponding Interference-to-Signal Ratio (ISR) matrix. This theoretical ISR matrix can serve as an estimate of the separation performance on the particular data. In simulations, the algorithm is shown to be applicable in blind separation of a linear mixture of speech signals.

When an autoregressive (AR) process is observed through a sparse multipath environment, its AR parameters may be estimated by searching for a symmetric finite impulse response (FIR) filter, which, when convolved with the observed signal's autocorrelation sequence, yields the sparsest output. The zeros of that filter would then correspond to the poles of the AR process. When the lscr0-norm of the output is used as a measure of its sparsity, consistency of the resulting estimate (under some simple conditions) is readily obtained. However, due to problematic aspects of lscr0-norm minimization, it is often more convenient to resort to lscr1-norm minimization. A question of major interest in this context is whether (and if so, under what conditions) consistency of the resulting estimate is maintained. By analyzing the perturbations of the lscr1-norm about the desired solution, we derive (and illustrate) specific conditions for consistency. We show that when the multipath reflections are sufficiently sparse, consistency is guaranteed for a very wide range of AR parameters and reflection gains.

We propose a new algorithm for Approximate Joint Diagonalization (AJD) with two main advantages over existing state-of-the-art algorithms: Improved overall running speed, especially in large-scale (high-dimensional) problems; and an ability to incorporate specially structured weight-matrices into the AJD criterion. The algorithm is based on approximate Gauss iterations for successive reduction of a weighted Least Squares off-diagonality criterion. The proposed Matlab® implementation allows AJD of ten 100 × 100 matrices in 3–4 seconds (for the unweighted case) on a common PC (Pentium M, 1.86GHz, 2GB RAM), generally 3–5 times faster than the fastest competitor. The ability to incorporate weights allows fast large-scale realization of optimized versions of classical blind source separation algorithms, such as Second-Order Blind Identification (SOBI), whose weighted version (WASOBI) yields significantly improved separation performance.

The Shalvi-Weinstein super-exponential algorithm for blind channel equalization employs empirical high-order cross-cumulants between the equalizer's input and output for iterative updates of the equalizer. When the source signal has (nearly) null cumulants of the required order, the algorithm's performance may be severely degraded. Rather than resort to even higher-order cumulants in such cases, we propose to employ an alternative statistic, based on second-order derivatives (Hessians, evaluated away from the origin) of the joint log-characteristic function of the equalizer's input and output. These Hessians admit straightforward empirical estimates, maintain the "philosophy of operation" of the algorithm, and, as we demonstrate in simulation, can significantly improve its performance in such (and in other) cases.

We consider the blind separation of source images from linear mixtures thereof, involving different relative spatial shifts of the sources in each mixture. Such mixtures can be caused, e.g., by the presence of a semi-reflective medium (such as a window glass) across a photographed scene, due to slight movements of the medium (or of the sources) between snapshots. Classical separation approaches assume either a static mixture model or a fully convolutive mixture model, which are, respectively, either under- or over-parameterized for this problem. In this paper, we develop a specially parameterized scheme for approximate joint diagonalization of estimated spectrum matrices, aimed at estimating the succinct set of mixture parameters: the static (gain) coefficients and the shift values. The estimated parameters are, in turn, used for convenient frequency-domain separation. As we demonstrate using both synthetic mixtures and real-life photographs, the advantage of the ability to incorporate spatial shifts is twofold: Not only does it enable separation when such shifts are present, but it also warrants deliberate introduction of such shifts as a simple source of added diversity whenever the static mixing coefficients form a singular matrix-thereby enabling separation in otherwise inseparable scenes.

Blind inversion of a linear and instantaneous mixture of source signals is a problem often encountered in many signal processing applications. Efficient fastICA (EFICA) offers an asymptotically optimal solution to this problem when all of the sources obey a generalized Gaussian distribution, at most one of them is Gaussian, and each is independent and identically distributed (i.i.d.) in time. Likewise, weights-adjusted second-order blind identification (WASOBI) is asymptotically optimal when all the sources are Gaussian and can be modeled as autoregressive (AR) processes with distinct spectra. Nevertheless, real-life mixtures are likely to contain both Gaussian AR and non-Gaussian i.i.d. sources, rendering WASOBI and EFICA severely suboptimal. In this paper, we propose a novel scheme for combining the strengths of EFICA and WASOBI in order to deal with such hybrid mixtures. Simulations show that our approach outperforms competing algorithms designed for separating similar mixtures.

Estimation of the autoregressive (AR) parameters of an AR process often involves applying Yule-Walker (YW) equations to the estimated correlations. When the process is Gaussian, the resulting estimate is asymptotically optimal, coinciding with the maximum-likelihood (ML) estimate. However, for non-Gaussian processes, applying the YW equations to the estimated correlations may be significantly sub-optimal, whereas computation of the exact ML estimate may be prohibitively cumbersome. In this paper we show how the YW equations may be applied to an alternative statistic, namely to off-origin Hessians of the second characteristic function. Although still not optimal, we show in simulation that the resulting estimate can significantly outperform the classical correlation-based estimate, as well as a cumulants-based estimate.

The performance of blind source separation algorithms is commonly measured by the output interference-to-signal ratio (ISR). In this paper, we derive an asymptotic bound on the attainable ISR for the case of Gaussian parametric auto-regressive (AR), moving-average (MA), or auto-regressive moving-average (ARMA) processes. Our bound is induced by the Crameacuter-Rao bound on estimation of the mixing matrix. We point out the relation to some previously obtained results, and provide a concise expression with some associated important insights. Using simulation, we demonstrate that the bound is attained asymptotically by some asymptotically efficient algorithms

We present a dictionary attack that is based on keyboard acoustic emanations. We combine signal processing and efficient data structures and algorithms, to successfully reconstruct single words of 7-13 characters from a recording of the clicks made when typing them on a keyboard. Our attack does not require any training, and works on an individual recording of the typed word (may be under 5 seconds of sound). The attack is very efficient, taking under 20 seconds per word on a standard PC. We demonstrate a 90% or better success rate of finding the correct word in the top 50 candidates identified by the attack, for words of 10 or more characters, and a success rate of 73% over all the words we tested. We show that the dominant factors affecting the attack's success are the word length, and more importantly, the number of repeated characters within the word. Our attack can be used as an effective acoustic-based password cracker. Our attack can also be used as part of an acoustic long-text reconstruction method, that is much more efficient and requires much less text than previous approaches.

We address the problem of blind separation of image mixtures (resulting, e.g., from reflections through window glass) consisting of pure unknown relative spatial-shifts in addition to scalar mixing coefficients. Most (and maybe all) existing approaches to the problem assume static mixtures, i.e., the spatial positions of the source images are assumed to remain fixed between snapshots. We propose an integrated method to estimate both the mixing coefficients and the spatial-shifts using a second-order statistics based algorithm, which uses specially parameterized approximate joint diagonalization of two-dimensional-spectra matrices. The accommodation of spatial shifts allows the exploitation of further diversity between snapshots, and thus enables to attain improved separation, especially when the static mixing coefficients are ill-conditioned - as we demonstrate using simulations results

We address the problem of blind separation of sources, mixed subject to possible Doppler frequency-shifts, differing between sources and between sensors. This situation is likely to occur, e.g. in scenarios involving mobile sensors and/or sources, but so far the multiple sources case does not seem to have been addressed (at least not in open literature, to our knowledge). We propose a batch-type iterative procedure for the estimation of the (static) mixing parameters and the frequency-shifts, followed by application of the inverse system to reconstruct the sources. The estimation procedure can be regarded as parameterized joint diagonalization of a bulk of rank-one matrices. Somewhat surprisingly, correlation matrices at zero lag are generally sufficient for separation, and consequently, the separation of white Gaussian sources is generally possible in this framework - unlike the situation in classical static mixing - as we demonstrate in simulation.

We propose a novel, noniterative approach for the problem of nonunitary, least-squares (LS) approximate joint diagonalization (AJD) of several Hermitian target matrices. Dwelling on the fact that exact joint diagonalization (EJD) of two Hermitian matrices can almost always be easily obtained in closed form, we show how two "representative matrices" can be constructed out of the original set of all target matrices, such that their EJD would be useful in the AJD of the original set. Indeed, for the two-by-two case, we show that the EJD of the representative matrices yields the optimal AJD solution. For larger-scale cases, the EJD can provide a suboptimal AJD solution, possibly serving as a good initial guess for a subsequent iterative algorithm. Additionally, we provide an informative lower bound on the attainable LS fit, which is useful in gauging the distance of prospective solutions from optimality.

We address the problem of passive blind estimation of time-delays for several mutually uncorrelated source signals received by a similar number of sensors. The mixtures at the receivers are modeled as unknown linear combinations of differently delayed versions of the source signals. The standard tools used in blind source separation (BSS) for either static or convolutive mixtures are inappropriate for this problem: The former is obviously under-parameterized, while the latter is over-parameterized and poorly suited for accommodating pure fractional delays. Thus, in this paper we propose a hybrid algorithm, which uses a specially parameterized approximate joint diagonalization of spectral matrices to estimate the delays (as well as the unknown mixing coefficients). The joint diagonalization algorithm is an extension of the iterative "AC-DC" algorithm, previously proposed in the context of BSS with static mixtures. We provide analytic expressions for all required steps for the two sensors/two sources case, and demonstrate the performance using simulations results.

High-order statistics have become a common tool in blind identification of nonminimum phase systems. In this paper we present a new, alternative tool, namely the first-order derivatives of the observations' second characteristic function, evaluated at arbitrary (off-origin) locations. The estimation of these derivatives reduces plainly into specially-weighted empirical averages, from which the identification of the system's zeros is nearly straightforward. We show that despite the addition of some nuisance parameters, this approach generates more equations than unknowns, and thus enables a well-averaged least-squares solution. We demonstrate, using simulation results, the potential improvement in estimation accuracy over cumulants-based estimation.

Traditionally, high-order statistics are used for blind estimation of nonminimum phase finite impulse response (FIR) channels. In this paper we present a new, alternative approach, that uses first- or second-order derivatives of the observations' second generalized characteristic function, evaluated at arbitrary (off-origin) locations. The estimation of these derivatives reduces plainly into specially-weighted empirical mean and covariance. We show that despite the addition of some nuisance parameters, this approach generates more equations than unknowns, and thus enables a well-averaged least-squares solution. A simulation example demonstrates the potential improvement in estimation accuracy over cumulants-based estimation.

A new algorithm for blind source separation is presented, which does not require any iterations with the raw data, and is therefore of a "closed-form" type. The algorithm is based on estimating the second-derivative matrices of the second joint characteristic function of the observations. These derivatives can be consistently estimated at various points, termed "processing points". A consistent estimate of the mixing matrix can in turn be obtained by applying approximate joint diagonalization to the estimated derivative matrices. Performance depends strongly on the choice of processing points, and can compare favorably to other BSS algorithms. We demonstrate the superior performance using simulations results.

Blind separation of Gaussian sources with different spectra can be attained using second-order statistics. The second-order blind identification (SOBI) algorithm, proposed by Belouchrani et al. (1997), uses approximate joint diagonalization. We show that substantial improvement over SOBI can be attained when the joint diagonalization is transformed into a properly weighted nonlinear least squares problem. We provide an iterative solution and derive the optimal weights for our weights-adjusted SOBI (WASOBI) algorithm. The improvement is demonstrated by analysis and simulations.

Approximate model equations often relate given measurements to unknown parameters whose estimate is sought. The least-squares (LS) estimation criterion assumes the measured data to be exact, and seeks parameters which minimize the model errors. Existing extensions of LS, such as the total LS (TLS) and constrained TLS (CTLS) take the opposite approach, namely assume the model equations to be exact, and attribute all errors to measurement inaccuracies. We introduce the extended LS (XLS) criterion, which accommodates both error sources. We define 'pseudo-linear' models, with which we provide an iterative algorithm for minimization of the XLS criterion. Under certain statistical assumptions, we show that XLS coincides with a statistical criterion, which we term the 'joint maximum-a-posteriori-maximum-likelihood' (JMAP-ML) criterion. We identify the differences between the JMAP-ML and ML criteria, and explain the observed superiority of JMAP-ML over ML under non-asymptotic conditions.