Existing trackers based on Poisson measurement process often struggle with efficiency and accuracy in large-scale tracking under heavy clutter. To overcome this, we introduce PiVoT, a scalable, robust multi-object tracker capable of efficiently detecting and tracking a large, varying number of objects, along with their shapes, existence probabilities, and measurement rates, even in heavy clutter. PiVoT employs a novel two-stage variational inference routine to achieve inference tractability and closed-form, parallelisable updates. Efficiency is further enhanced by early identification and removal of ineffective birth objects and designing highly simplified, much faster, yet equivalent variational updates. Additionally, PiVoT inherently offers efficient clutter-robust clustering, an innovation that can also enhance existing trackers that depend on supplementary clustering techniques. Experiments demonstrate PiVoT's clear accuracy and efficiency gains over existing methods, while also highlighting its ability to track a thousand closely spaced objects in under a second on a standard laptop without gating.

Recently, many self-supervised learning methods for image reconstruction have been proposed that can learn from noisy data alone, bypassing the need for ground-truth references. Most existing methods cluster around two classes: i) Stein's Unbiased Risk Estimate (SURE) and similar approaches that assume full knowledge of the noise distribution, and ii) Noise2Self and similar cross-validation methods that require very mild knowledge about the noise distribution. The first class of methods tends to be impractical, as the noise level is often unknown in real-world applications, and the second class is often suboptimal compared to supervised learning. In this paper, we provide a theoretical framework that characterizes this expressivity-robustness trade-off and propose a new approach based on SURE, but unlike the standard SURE, does not require knowledge about the noise level. Throughout a series of experiments, we show that the proposed estimator outperforms other existing self-supervised methods on various imaging inverse problems.

This paper explores the increasing demand for accurate and resilient multi-sensor fusion techniques, particularly within 3D tracking systems enhanced by drone technology. Employing the adaptive kernel Kalman filter (AKKF) methodology within the Stone Soup framework, our research seeks to develop robust fusion approaches capable of seamlessly amalgamating data from a multi-sensor arrangement with fixed ground sensors and dynamic sensors mounted on drones. By capitalising on the adaptive nature of the $A K K F$, we aim to refine the precision and dependability of 3D object tracking in intricate scenarios. Through empirical evaluations, we illustrate the effectiveness of our proposed AKKF-based fusion strategies in enhancing tracking performance within the Stone Soup framework, thus contributing to the advancement of multi-sensor fusion methodologies within this framework.

Photon counting lidar has become an invaluable tool for 3D depth imaging due to the fine-precision it can achieve over long ranges. However, high frame rate, high resolution lidar devices produce an enormous amount of time-of-flight (ToF) data which can hinder the deployment of real-time systems. In this paper, an efficient photon acquisition approach is proposed that exploits the simplicity of piecewise polynomial splines to form a hardware-friendly compressed statistic, or spline sketch, of the ToF data. We show that a piecewise linear or quadratic spline sketch, requires minimal on-chip arithmetic computation per photon detection and can reconstruct real-world depth images using a simple closed form solution. Further, by building range-walk correction into the proposed estimation algorithms, it is demonstrated that the spline sketches can be made robust to photon pile-up effects.

Solving an ill-posed linear inverse problem requires knowledge about the underlying signal model. In many applications, this model is a priori unknown and has to be learned from data. However, it is impossible to learn the model using observations obtained via a single incomplete measurement operator, as there is no information about the signal model in the nullspace of the operator, resulting in a chicken-and-egg problem: to learn the model we need reconstructed signals, but to reconstruct the signals we need to know the model. Two ways to overcome this limitation are using multiple measurement operators or assuming that the signal model is invariant to a certain group action. In this paper, we present necessary and sufficient sensing conditions for learning the signal model from measurement data alone which only depend on the dimension of the model and the number of operators or properties of the group action that the model is invariant to. As our results are agnostic of the learning algorithm, they shed light into the fundamental limitations of learning from incomplete data and have implications in a wide range set of practical algorithms, such as dictionary learning, matrix completion and deep neural networks.

Many modern imaging applications can be modeled as compressed sensing linear inverse problems. When the measurement operator involved in the inverse problem is sufficiently random, denoising Scalable Message Passing (SMP) algorithms have a potential to demonstrate high efficiency in recovering compressed data. One of the key components enabling SMP to achieve fast convergence, stability and predictable dynamics is the Onsager correction that must be updated at each iteration of the algorithm. This correction involves the denoiser’s divergence that is traditionally estimated via the Black-Box Monte Carlo (BB-MC) method. While the BB-MC method demonstrates satisfying accuracy of estimation, it requires heuristic tuning and executing the denoiser additional times at each iteration and might lead to a substantial increase in computational cost of the SMP algorithms. In this work we develop two Large System Limit models of the Onsager correction for denoisers operating within SMP algorithms and use these models to propose practical black-box methods for divergence estimation that require no additional executions of the denoiser and demonstrate similar correction compared to the BB-MC method.

Deep networks provide state-of-the-art performance in multiple imaging inverse problems ranging from medical imaging to computational photography. However, most existing networks are trained with clean signals which are often hard or impossible to obtain. Equivariant imaging (EI) is a recent self-supervised learning framework that exploits the group invariance present in signal distributions to learn a reconstruction function from partial measurement data alone. While EI results are impressive, its performance degrades with increasing noise. In this paper, we propose a Robust Equivariant Imaging (REI) framework which can learn to image from noisy partial measurements alone. The proposed method uses Stein's Unbiased Risk Estimator (SURE) to obtain a fully unsupervised training loss that is robust to noise. We show that REI leads to considerable performance gains on linear and nonlinear inverse problems, thereby paving the way for robust unsupervised imaging with deep networks. Code is available at https://github.com/edongdongchen/REI.

Single-photon light detection and ranging (lidar) captures depth and intensity information of a 3D scene. Reconstructing a scene from observed photons is a challenging task due to spurious detections associated with background illumination sources. To tackle this problem, there is a plethora of 3D reconstruction algorithms which exploit spatial regularity of natural scenes to provide stable reconstructions. However, most existing algorithms have computational and memory complexity proportional to the number of recorded photons. This complexity hinders their real-time deployment on modern lidar arrays which acquire billions of photons per second. Leveraging a recent lidar sketching framework, we show that it is possible to modify existing reconstruction algorithms such that they only require a small sketch of the photon information. In particular, we propose a sketched version of a recent state-of-the-art algorithm which uses point cloud denoisers to provide spatially regularized reconstructions. A series of experiments performed on real lidar datasets demonstrates a significant reduction of execution time and memory requirements, while achieving the same reconstruction performance than in the full data case.

Vector Approximate Message Passing (VAMP) provides the means of solving a linear inverse problem in a Bayes-optimal way assuming the measurement operator is sufficiently random. However, VAMP requires implementing the linear minimum mean squared error (LMMSE) estimator at every iteration, which makes the algorithm intractable for large-scale problems. In this work, we present a class of warm-started (WS) methods that provides a scalable approximation of LMMSE within VAMP. We show that a Message Passing (MP) algorithm equipped with a method from this class can converge to the fixed point of VAMP while having a per-iteration computational complexity proportional to that of AMP. Additionally, we provide the Onsager correction and a multi-dimensional State Evolution for MP utilizing one of the WS methods. Lastly, we show that the approximation approach used in the recently proposed Memory AMP (MAMP) algorithm is a special case of the developed class of WS methods.

In many real-world inverse problems, only incomplete measurement data are available for training which can pose a problem for learning a reconstruction function. Indeed, unsupervised learning using a fixed incomplete measurement process is impossible in general, as there is no information in the nullspace of the measurement operator. This limitation can be overcome by using measurements from multiple operators. While this idea has been successfully applied in various applications, a precise characterization of the conditions for learning is still lacking. In this paper, we fill this gap by presenting necessary and sufficient conditions for learning the underlying signal model needed for reconstruction which indicate the interplay between the number of distinct measurement operators, the number of measurements per operator, the dimension of the model and the dimension of the signals. Furthermore, we propose a novel and conceptually simple unsupervised learning loss which only requires access to incomplete measurement data and achieves a performance on par with supervised learning when the sufficient condition is verified. We validate our theoretical bounds and demonstrate the advantages of the proposed unsupervised loss compared to previous methods via a series of experiments on various imaging inverse problems, such as accelerated magnetic resonance imaging, compressed sensing and image inpainting.

This letter incorporates the adaptive kernel Kalman filter (AKKF) into the belief propagation (BP) algorithm for multi-target tracking (MTT) in single-sensor systems. The algorithm is capable of tracking an unknown and time-varying number of targets, in the presence of false alarms, clutter and measurement-to-target association uncertainty. Experiment results reveal that the proposed method has a favourable tracking performance using the generalized optimal sub-patten assignment (GOSAP) metrics at substantially less computation cost than the particle filter (PF) based multi-target tracking (MTT) BP algorithm.

High-resolution wide-swath synthetic aperture radar (HRWS-SAR) imaging is highly desirable since it allows one to produce high-resolution SAR images of large areas during a short visit time. In this article, staggered coprime pulse repetition frequencies synthetic aperture radar (SCopSAR) is proposed. It divides the time during which a scatterer is illuminated by the antenna beam pattern into two halves where, in each half, pulses are transmitted at the rate of one of two sub-Nyquist pulse repetition frequencies (PRFs). Such PRFs are related to the Nyquist PRF using two coprime subsampling factors. This allows extending the maximum range swath width that can be imaged by a number of times that equals the smaller subsampling factor at the expense of a reduction in the azimuth resolution by half. It further allows for a reduction in the amount of data to be stored and communicated. SCopSAR is an imaging modality suitable for scenes that contain a small number of bright scatterers over a dark background which, for instance, is the case when imaging ships in a calm sea background. Compared with the techniques recently proposed in the literature, SCopSAR simplifies the radar requirements since it uses only one carrier frequency, one waveform, and one channel. Simulations and real ERS-2 satellite raw data are used to validate the theoretical findings presented in this article.

The Recently proposed Vector Approximate Message Passing (VAMP) algorithm demonstrates a great reconstruction potential at solving compressed sensing related linear inverse problems. VAMP provides high per-iteration improvement, can utilize powerful denoisers like BM3D, has rigorously defined dynamics and is able to recover signals measured by highly undersampled and ill-conditioned linear operators. Yet, its applicability is limited to relatively small problem sizes due to the necessity to compute the expensive LMMSE estimator at each iteration. In this work we consider the problem of upscaling VAMP by utilizing Conjugate Gradient (CG) to approximate the intractable LMMSE estimator. We propose a rigorous method for correcting and tuning CG withing CG-VAMP to achieve a stable and efficient reconstruction. To further improve the performance of CG-VAMP, we design a warm-starting scheme for CG and develop theoretical models for the Onsager correction and the State Evolution of Warm-Started CG-VAMP (WS-CG-VAMP). Additionally, we develop robust and accurate methods for implementing the WS-CG-VAMP algorithm. The numerical experiments on large-scale image reconstruction problems demonstrate that WS-CG-VAMP requires much fewer CG iterations compared to CG-VAMP to achieve the same or superior level of reconstruction.

Deep convolutional neural networks (CNNs) have emerged as a new paradigm for Mammogram diagnosis. Contemporary CNN-based computer-aided-diagnosis systems (CADs) for breast cancer directly extract latent features from input mammogram image and ignore the importance of morphological features. In this paper, we introduce a novel end-to-end deep learning framework for mammogram image processing, which computes mass segmentation and simultaneously predicts diagnosis results. Specifically, our method is constructed in a dual-path architecture that solves the mapping in a dual-problem manner, with an additional consideration of important shape and boundary knowledge. One path, called the Locality Preserving Learner (LPL), is devoted to hierarchically extracting and exploiting intrinsic features of the input. Whereas the other path, called the Conditional Graph Learner (CGL), focuses on generating geometrical features via modeling pixel-wise image to mask correlations. By integrating the two learners, both the cancer semantics and cancer representations are well learned, and the component learning paths in return complement each other, contributing an improvement to the mass segmentation and cancer classification problem at the same time. In addition, by integrating an automatic detection set-up, the DualCoreNet achieves fully automatic breast cancer diagnosis practically. Experimental results show that in benchmark DDSM dataset, DualCoreNet has outperformed other related works in both segmentation and classification tasks, achieving 92.27% DI coefficient and 0.85 AUC score. In another benchmark INbreast dataset, DualCoreNet achieves the best mammography segmentation (93.69% DI coefficient) and competitive classification performance (0.93 AUC score).

Convolutional Neural Networks (CNNs) are now a well-established tool for solving computational imaging problems. Modern CNN-based algorithms obtain state-of-the-art performance in diverse image restoration problems. Furthermore, it has been recently shown that, despite being highly overparameterized, networks trained with a single corrupted image can still perform as well as fully trained networks. We introduce a formal link between such networks through their neural tangent kernel (NTK), and well-known non-local filtering techniques, such as non-local means or BM3D. The filtering function associated with a given network architecture can be obtained in closed form without need to train the network, being fully characterized by the random initialization of the network weights. While the NTK theory accurately predicts the filter associated with networks trained using standard gradient descent, our analysis shows that it falls short to explain the behaviour of networks trained using the popular Adam optimizer. The latter achieves a larger change of weights in hidden layers, adapting the non-local filtering function during training. We evaluate our findings via extensive image denoising experiments1.

In various imaging problems, we only have access to compressed measurements of the underlying signals, hindering most learning-based strategies which usually require pairs of signals and associated measurements for training Learning only from compressed measurements is impossible in general, as the compressed observations do not contain information outside the range of the forward sensing operator. We propose a new end-to-end self-supervised framework that overcomes this limitation by exploiting the equivariances present in natural signals. Our proposed learning strategy performs as well as fully supervised methods. Experiments demonstrate the potential of this frame- work on inverse problems including sparse-view X-ray computed tomography on real clinical data and image inpainting on natural images. Code has been made available at: https://github.com/edongdongchen/EI.

In this work, we introduce a novel stochastic proximal alternating linearized minimization algorithm [J. Bolte, S. Sabach, and M. Teboulle, Math. Program., 146 (2014), pp. 459--494] for solving a class of nonsmooth and nonconvex optimization problems. Large-scale imaging problems are becoming increasingly prevalent due to the advances in data acquisition and computational capabilities. Motivated by the success of stochastic optimization methods, we propose a stochastic variant of proximal alternating linearized minimization. We provide global convergence guarantees, demonstrating that our proposed method with variance-reduced stochastic gradient estimators, such as SAGA [A. Defazio, F. Bach, and S. Lacoste-Julien, Advances in Neural Information Processing Systems, 2014, pp. 1646--1654] and SARAH [L. M. Nguyen, J. Liu, K. Scheinberg, and M. Takáĉ, Proceedings of the 34th International Conference on Machine Learning, PMLR 70, 2017, pp. 2613--2621], achieves state-of-the-art oracle complexities. We also demonstrate the efficacy of our algorithm via several numerical examples including sparse nonnegative matrix factorization, sparse principal component analysis, and blind image-deconvolution.

In this article, we present a new algorithm for fast, online 3D reconstruction of dynamic scenes using times of arrival of photons recorded by single-photon detector arrays. One of the main challenges in 3D imaging using single-photon lidar in practical applications is the presence of strong ambient illumination which corrupts the data and can jeopardize the detection of peaks/surface in the signals. This background noise not only complicates the observation model classically used for 3D reconstruction but also the estimation procedure which requires iterative methods. In this work, we consider a new similarity measure for robust depth estimation, which allows us to use a simple observation model and a non-iterative estimation procedure while being robust to mis-specification of the background illumination model. This choice leads to a computationally attractive depth estimation procedure without significant degradation of the reconstruction performance. This new depth estimation procedure is coupled with a spatio-temporal model to capture the natural correlation between neighboring pixels and successive frames for dynamic scene analysis. The resulting online inference process is scalable and well suited for parallel implementation. The benefits of the proposed method are demonstrated through a series of experiments conducted with simulated and real single-photon lidar videos, allowing the analysis of dynamic scenes at 325 m observed under extreme ambient illumination conditions.

In this paper we consider the problem of recovering a signal x of size N from noisy and compressed measurements y = Ax + w of size M, where the measurement matrix A is right-orthogonally invariant (ROI). Vector Approximate Message Passing (VAMP) demonstrates great reconstruction results for even highly ill-conditioned matrices A in relatively few iterations. However, performing each iteration is challenging due to either computational or memory point of view. On the other hand, a recently proposed Conjugate Gradient (CG) Expectation Propagation (CG-EP) framework is able to sacrifice some performance for efficiency, but requires access to exact singular spectrum of A. In this work we develop a CG-VAMP algorithm that does not require such information, is feasible to implement and converges to the neighborhood of the original VAMP.

X-ray computed tomography (CT) reconstruction from a sparse number of views is a useful way to reduce either the radiation dose or the acquisition time, for example in fixed-gantry CT systems; however, this results in an ill-posed inverse problem whose solution is typically computationally demanding. Approximate message passing (AMP) techniques represent the state of the art for solving undersampling compressed sensing problems with random linear measurements, but there are still not clear solutions on how AMP should be modified and how it performs with real world problems. This paper investigates the question of whether we can employ an AMP framework for real sparse view CT imaging. The proposed algorithm for approximate inference in tomographic reconstruction incorporates a number of advances from within the AMP community, resulting in the denoising generalized approximate message passing CT algorithm (D-GAMP-CT). Specifically, this exploits the use of sophisticated image denoisers to regularize the reconstruction. While in order to reduce the probability of divergence the (Radon) system and the Poisson nonlinear noise model are treated separately, exploiting the existence of efficient preconditioners for the former and the generalized noise modeling in GAMP for the latter. Experiments with simulated and real CT baggage scans confirm that the performance of the proposed algorithm outperforms statistical CT optimization solvers.

This letter analyzes the performances of a simple reconstruction method, namely the Projected Back-Projection (PBP), for estimating the direction of a sparse signal from its phase-only (or amplitude-less) complex Gaussian random measurements, i.e., an extension of one-bit compressive sensing to the complex field. To study the performances of this algorithm, we show that complex Gaussian random matrices respect, with high probability, a variant of the Restricted Isometry Property (RIP) relating to the ℓ1-norm of the sparse signal measurements to their ℓ2-norm. This property allows us to upper-bound the reconstruction error of PBP in the presence of phase noise. Monte Carlo simulations are performed to highlight the performance of our approach in this phase-only acquisition model when compared to error achieved by PBP in classical compressive sensing.

In this paper, we present an algorithm for online 3D reconstruction of dynamic scenes using individual times of arrival (ToA) of photons recorded by single-photon detector arrays. One of the main challenges in 3D imaging using single-photon Lidar is the integration time required to build ToA histograms and reconstruct reliably 3D profiles in the presence of non-negligible ambient illumination. This long integration time also prevents the analysis of rapid dynamic scenes using existing techniques. We propose a new method which does not rely on the construction of ToA histograms but allows, for the first time, individual detection events to be processed online, in a parallel manner in different pixels, while accounting for the intrinsic spatiotemporal structure of dynamic scenes. Adopting a Bayesian approach, a Bayesian model is constructed to capture the dynamics of the 3D profile and an approximate inference scheme based on assumed density filtering is proposed, yielding a fast and robust reconstruction algorithm able to process efficiently thousands to millions of frames, as usually recorded using single-photon detectors. The performance of the proposed method, able to process hundreds of frames per second, is assessed using a series of experiments conducted with static and dynamic 3D scenes and the results obtained pave the way to a new family of real-time 3D reconstruction solutions.

This paper discusses Expectation-Propagation (EP) methods for approximate Bayesian inference in the context of linear regression with Poisson noise. We review two main factor graphs used for generalized linear models and discuss how different EP algorithms can be derived. The estimation performance based on EP approximations is compared to the performance using Monte Carlo sampling from the exact posterior distribution. In particular, we observe that using locally independent or isotropic approximate factors enables more robust and scalable algorithms while providing reliable posterior means and marginal variances.

Current popular methods for Magnetic Resonance Fingerprint (MRF) recovery are bottlenecked by the heavy storage and computation requirements of a dictionary-matching (DM) step due to the growing size and complexity of the fingerprint dictionaries in multi-parametric quantitative MRI applications. In this paper we study a deep learning approach to address these shortcomings. Coupled with a dimensionality reduction first layer, the proposed MRF-Net is able to reconstruct quantitative maps by saving more than 60 times in memory and computations required for a DM baseline. Fine-grid manifold enumeration i.e. the MRF dictionary is only used for training the network and not during image reconstruction. We show that the MRF-Net provides a piece-wise affine approximation to the Bloch response manifold projection and that rather than memorizing the dictionary, the network efficiently clusters this manifold and learns a set of hierarchical matched-filters for affine regression of the NMR characteristics in each segment.

We present, for the first time, a novel deep neural network architecture called DualCoreNet with a dual-path connection between the input image and output class label for mammogram image processing. This architecture is built upon U-Net, which non-linearly maps the input data into a deep latent space. One path of the DualCoreNet, the locality preserving learner, is devoted to hierarchically extracting and exploiting intrinsic features of the input, while the other path, called the conditional graph learner, focuses on modeling the input-mask correlations. The learned mask is further used to improve classification results, and the two learning paths complement each other. By integrating the two learners our new architecture provides a simple but effective way to jointly learn the segmentation and predict the class label. Benefiting from the powerful expressive capacity of deep neural networks a more discriminative representation can be learned, in which both the semantics and structure are well preserved. Experimental results show that DualCoreNet achieves the best mammography segmentation and classification simultaneously, outperforming recent state-of-the-art models.

In this paper, we consider the estimation of hyperparameters for regularization terms commonly used for obtaining structured sparse parameters in signal estimation problems, such as signal denoising. By considering the convex regularization terms as negative log-densities, we propose approximate maximum likelihood estimation for estimating parameters for continuous log-supermodular distributions, which is a key property that many sparse priors have. We then show how "perturb-and-MAP" ideas based on the Gumbel distribution and efficient discretization can be used to approximate the log-partition function for these models, which is a crucial step for approximate maximum likelihood estimation. We illustrate our estimation procedure on a set of experiments with flow-based priors and signal denoising.

In this work we investigate the practicability of stochastic gradient descent and recently introduced variants with variance-reduction techniques in imaging inverse problems, such as space-varying image deblurring. Such algorithms have been shown in machine learning literature to have optimal complexities in theory, and provide great improvement empirically over the full gradient methods. Surprisingly, in some tasks such as image deblurring, many of such methods fail to converge faster than the accelerated full gradient method (FISTA), even in terms of epoch counts. We investigate this phenomenon and propose a theory-inspired mechanism to characterize whether a given inverse problem should be preferred to be solved by stochastic optimization technique with a known sampling pattern. Furthermore, to overcome another key bottleneck of stochastic optimization which is the heavy computation of proximal operators while maintaining fast convergence, we propose an accelerated primal-dual SGD algorithm and demonstrate the effectiveness of our approach in image deblurring experiments.

This paper presents a processing framework to separate moving targets from the clutter, under multichannel synthetic aperture radar (SAR) scenarios, and addresses the moving target imaging and velocity estimation problems for ground moving target indication (GMTI) applications. A practical implementation is introduced to break the SAR/GMTI problem into two processing stages, and the sparsity of the moving targets in the observed scene is exploited throughout the stages. The two-stage process extracts the moving targets from the monitored region via a sparsity-based iterative decomposition algorithm and subsequently estimates the complete velocity vectors of moving targets by enforcing sparsity constraints. The model is sufficiently versatile to incorporate digital elevation map information, which further improves the moving target relocation accuracy. The effectiveness of the presented framework is demonstrated using the Air Force Research Laboratory Gotcha GMTI challenge data.

We propose a structure-adaptive variant of the state-of-the-art stochastic variance-reduced gradient algorithm Katyusha for regularized empirical risk minimization. The proposed method is able to exploit the intrinsic low-dimensional structure of the solution, such as sparsity or low rank which is enforced by a non-smooth regularization, to achieve even faster convergence rate. This provable algorithmic improvement is done by restarting the Katyusha algorithm according to restricted strong-convexity constants. We demonstrate the effectiveness of our approach via numerical experiments.

We study the convergence of the iterative projected gradient (IPG) algorithm for arbitrary (possibly non-convex) sets when both the gradient and projection oracles are computed approximately. We consider different notions of approximation of which we show that the progressive fixed precision and the (1+ ε)-optimal oracles can achieve the same accuracy as for the exact IPG algorithm. We show that the former scheme is also able to maintain the (linear) rate of convergence of the exact algorithm under the same embedding assumption. In contrast, the (1+ε)-approximate oracle requires a stronger embedding condition, moderate compression ratios and it typically slows down the convergence. We apply our results to accelerate solving a class of data driven compressed sensing problems, where we replace iterative exhaustive searches over large data sets by fast approximate nearest neighbor search strategies based on the cover tree data structure. For data sets with low intrinsic dimensions, our proposed algorithm achieves a complexity logarithmic in terms of the data set population as opposed to the linear complexity of a brute force search. By running several numerical experiments, we conclude similar observations as predicted by our theoretical analysis.

We propose a randomized first order optimization algorithm Gradient Projection Iterative Sketch (GPIS) and an accelerated variant for efficiently solving large scale constrained Least Squares (LS). We provide the first theoretical convergence analysis for both algorithms. An efficient implementation using a tailored line-search scheme is also proposed. We demonstrate our methods’ computational efficiency compared to the classical accelerated gradient method, and the variance-reduced stochastic gradient methods through numerical experiments in various large synthetic/real data sets.

We consider the problem of constructing a linear map from a Hilbert space H (possibly infinite dimensional) to ℝmthat satisfies a restricted isometry property (RIP) on an arbitrary signal model, i.e., a subset of H. We present a generic framework that handles a large class of low-dimensional subsets but also unstructured and structured linear maps. We provide a simple recipe to prove that a random linear map satisfies a general RIP with high probability. We also describe a generic technique to construct linear maps that satisfy the RIP. Finally, we detail how to use our results in several examples, which allow us to recover and extend many known compressive sampling results.

One of the important classes of sparse signals is the non-negative signals. Many algorithms have already been proposed to recover such non-negative representations, where greedy and convex relaxed algorithms are among the most popular methods. The greedy techniques have been modified to incorporate the non-negativity of the representations. One such modification has been proposed for Orthogonal Matching Pursuit (OMP), which first chooses positive coefficients and uses a non-negative optimisation technique as a replacement for the orthogonal projection onto the selected support. Beside the extra computational costs of the optimisation program, it does not benefit from the fast implementation techniques of OMP. These fast implementations are based on the matrix factorisations. We here first investigate the problem of positive representation, using pursuit algorithms. We will then describe a new implementation, which can fully incorporate the positivity constraint of the coefficients, throughout the selection stage of the algorithm. As a result, we present a novel fast implementation of the Non-Negative OMP, which is based on the QR decomposition and an iterative coefficients update. We will empirically show that such a modification can easily accelerate the implementation by a factor of ten in a reasonable size problem.

Inspired by the recently proposed Magnetic Resonance Fingerprinting technique, we develop a principled compressed sensing framework for quantitative MRI. The three key components are: a random pulse excitation sequence following the MRF technique; a random EPI subsampling strategy and an iterative projection algorithm that imposes consistency with the Bloch equations. We show that, as long as the excitation sequence possesses an appropriate form of persistent excitation, we are able to achieve accurate recovery of the proton density, T1, T2and off-resonance maps simultaneously from a limited number of samples.

In this paper, we extend the idea of the seeding matrix design and introduce the modulated matrix framework for compressed sensing. The 1-D state evolution equation is derived to track the sample distortion performance as a function of the signal distribution and the rescaling matrix. A special example, the two-block matrix, is presented as a generalization of the hybrid zeroing matrix. The first order phase transition is further studied to better understand the dynamics. With the two-block matrix, exact recovery can be achieved in the region where the homogeneous Gaussian matrix is not optimal for the sparse signals. For compressible signals, the reconstruction quality can also be effectively improved.

The problem of efficient sampling of wideband Radar signals for Electronic Support Measures (ESM) is investigated in this paper. Wideband radio frequency sampling generally needs a sampling rate at least twice the maximum frequency of the signal, i.e. Nyquist rate, which is generally very high. However, when the signal is highly structured, like wideband Radar signals, we can use the fact that signals do not occupy the whole spectrum and instead, there exists a parsimonious structure in the time-frequency domain. Here, we use this fact and introduce a novel low complexity sampling system, which has a recovery guarantee, assuming that received RF signals follow a particular structure. The proposed technique is inspired by the compressive sampling of sparse signals and it uses a multi-coset sampling setting, however it does not involve a computationally expensive reconstruction step. We call this here Low-Complexity Multi-Coset (LoCoMC) sampling technique. Simulation results, show that the proposed sub-Nyquist sampling technique works well in simulated ES scenarios.

Inspired by the recently proposed magnetic resonance fingerprinting (MRF) technique, we develop a principled compressed sensing framework for quantitative MRI. The three key components are a random pulse excitation sequence following the MRF technique, a random EPI subsampling strategy, and an iterative projection algorithm that imposes consistency with the Bloch equations. We show that, theoretically, as long as the excitation sequence possesses an appropriate form of persistent excitation, we are able to accurately recover the proton density, T1, T2, and off-resonance maps simultaneously from a limited number of samples. These results are further supported through extensive simulations using a brain phantom.

Within the framework of compressed sensing, we consider dense signals, which contain both discrete as well as continuous-amplitude components. We demonstrate by a comprehensive numerical study–to the best of our knowledge the first of its kind in the literature–that dense signals can be recovered from noisy, incomplete linear measurements by simple iterative algorithms that are inspired by or are implementations of approximate message passing. Those iterative algorithms are shown to significantly outperform all other algorithms presented so far, when they use a novel noise-adaptive thresholding function that is proposed in this contribution.

The primary challenge in linear inverse problems is to design stable and robust decoders to reconstruct high-dimensional vectors from a low-dimensional observation through a linear operator. Sparsity, low-rank, and related assumptions are typically exploited to design decoders, whose performance is then bounded based on some measure of deviation from the idealized model, typically using a norm. This paper focuses on characterizing the fundamental performance limits that can be expected from an ideal decoder given a general model, i.e., a general subset of simple vectors of interest. First, we extend the so-called notion of instance optimality of a decoder to settings where one only wishes to reconstruct some part of the original high-dimensional vector from a low-dimensional observation. This covers practical settings, such as medical imaging of a region of interest, or audio source separation, when one is only interested in estimating the contribution of a specific instrument to a musical recording. We define instance optimality relatively to a model much beyond the traditional framework of sparse recovery, and characterize the existence of an instance optimal decoder in terms of joint properties of the model and the considered linear operator. Noiseless and noise-robust settings are both considered. We show somewhat surprisingly that the existence of noise-aware instance optimal decoders for all noise levels implies the existence of a noise-blind decoder. A consequence of our results is that for models that are rich enough to contain an orthonormal basis, the existence of an ℓ2/ℓ2instance optimal decoder is only possible when the linear operator is not substantially dimension-reducing. This covers well-known cases (sparse vectors, low-rank matrices) as well as a number of seemingly new situations (structured sparsity and sparse inverse covariance matrices for instance). We exhibit an operator-dependent norm which, under a model-specific generalization of the restricted isometry property, always yields a feasible instance optimality property. This norm can be upper bounded by an atomic norm relative to the considered model.

We model the compressible signal with the two states Gaussian mixture distribution and consider the sample distortion function for the recently proposed Bayesian optimal AMP decoder. By leveraging the rigorous analysis of the AMP algorithm, we are able to derive the theoretical SD function and a sample allocation scheme for multi-resolution statistical image model. We then adopt the “turbo” message passing method to integrate the bandwise sample allocation with the exploitation of the hidden Markov tree structure of wavelet coefficients. Experiments on natural image show that the combination outperforms either of them working alone.

We cast the under-determined convolutive speech separation as sparse approximation of the spatial spectra of the mixing sources. In this framework we compare and contrast the major practical algorithms for structured sparse recovery of speech signal. Specific attention is paid to characterization of the measurement matrix. We first propose how it can be identified using the Image model of multipath effect where the acoustic parameters are estimated by localizing a speaker and its images in a free space model. We further study the circumstances in which the coherence of the projections induced by microphone array design tend to affect the recovery performance.

This paper investigates analysis operator learning for the recently introduced cosparse signal model that is a natural analysis complement to the more traditional sparse signal model. Previous work on such analysis operator learning has relied on access to a set of clean training samples. Here we introduce a new learning framework which can use training data which is corrupted by noise and/or is only approximately cosparse. The new model assumes that a p-cosparse signal exists in an epsilon neighborhood of each data point. The operator is assumed to be uniformly normalized tight frame (UNTF) to exclude some trivial operators. In this setting, an alternating optimization algorithm is introduced to learn a suitable analysis operator.

The authors propose an advanced synthetic aperture radar (SAR) image formation framework based on iterative inversion algorithms that approximately solve a regularised least squares problem. The framework provides improved image reconstructions, compared to the standard methods, in certain imaging scenarios, for example when the SAR data are under-sampled. Iterative algorithms also allow prior information to be used to solve additional problems such as the correction of unknown phase errors in the SAR data. However, for an iterative inversion framework to be feasible, fast algorithms for the generative model and its adjoint must be available. The authors demonstrate how fast, N2 log2 N complexity, (re/back)-projection algorithms can be used as accurate approximations for the generative model and its adjoint, without the limiting geometric approximations of other N2 log2 N methods, for example, the polar format algorithm. Experimental results demonstrate the effectiveness of their framework using publicly available SAR datasets.

This letter considers sufficient conditions for sparse recovery in the sparse multiple measurement vector (MMV) problem for some recently proposed rank aware greedy algorithms. Specifically we consider the compressed sensing framework with Gaussian random measurement matrices and show that the rank of the measurement matrix in the noiseless sparse MMV problem allows such algorithms to reduce the effect of the lognterm that is present in traditional OMP recovery.

This paper revisits the sparse multiple measurement vector (MMV) problem, where the aim is to recover a set of jointly sparse multichannel vectors from incomplete measurements. This problem is an extension of single channel sparse recovery, which lies at the heart of compressed sensing. Inspired by the links to array signal processing, a new family of MMV algorithms is considered that highlight the role of rank in determining the difficulty of the MMV recovery problem. The simplest such method is a discrete version of MUSIC which is guaranteed to recover the sparse vectors in the full rank MMV setting, under mild conditions. This idea is extended to a rank aware pursuit algorithm that naturally reduces to Order Recursive Matching Pursuit (ORMP) in the single measurement case while also providing guaranteed recovery in the full rank setting. In contrast, popular MMV methods such as Simultaneous Orthogonal Matching Pursuit (SOMP) and mixed norm minimization techniques are shown to be rank blind in terms of worst case analysis. Numerical simulations demonstrate that the rank aware techniques are significantly better than existing methods in dealing with multiple measurements.

We develop a principled way of identifying probability distributions whose independent and identically distributed realizations are compressible, i.e., can be well approximated as sparse. We focus on Gaussian compressed sensing, an example of underdetermined linear regression, where compressibility is known to ensure the success of estimators exploiting sparse regularization. We prove that many distributions revolving around maximum a posteriori (MAP) interpretation of sparse regularized estimators are in fact incompressible, in the limit of large problem sizes. We especially highlight the Laplace distribution and ^1 regularized estimators such as the Lasso and basis pursuit denoising. We rigorously disprove the myth that the success of ^1 minimization for compressed sensing image reconstruction is a simple corollary of a Laplace model of images combined with Bayesian MAP estimation, and show that in fact quite the reverse is true. To establish this result, we identify nontrivial undersampling regions where the simple least-squares solution almost surely outperforms an oracle sparse solution, when the data are generated from the Laplace distribution. We also provide simple rules of thumb to characterize classes of compressible and incompressible distributions based on their second and fourth moments. Generalized Gaussian and generalized Pareto distributions serve as running examples.

In this paper, we consider power spectral density estimation of bandlimited, wide-sense stationary signals from sub-Nyquist sampled data. This problem has recently received attention from within the emerging field of cognitive radio for example, and solutions have been proposed that use ideas from compressed sensing and the theory of digital alias-free signal processing. Here we develop a compressed sensing based technique that employs multi-coset sampling and produces multi-resolution power spectral estimates at arbitrarily low average sampling rates. The technique applies to spectrally sparse and nonsparse signals alike, but we show that when the wide-sense stationary signal is spectrally sparse, compressed sensing is able to enhance the estimator. The estimator does not require signal reconstruction and can be directly obtained from a straightforward application of nonnegative least squares.

In this paper we present a algorithm designed to detect and characterise the signal coming from Frequency Modulation Continuous Wave radars. The signals are made of linear frequency modulations. A few relevant coefficients of the chirplet transform are selected, and then gathered into chirps whose starting time, length, and chirprate are estimated. An example is provided on a synthetic signal.

In the past decade there has been a great interest in a synthesis-based model for signals, based on sparse and redundant representations. Such a model assumes that the signal of interest can be composed as a linear combination of few columns from a given matrix (the dictionary). An alternative analysis-based model can be envisioned, where an analysis operator multiplies the signal, leading to a cosparse outcome. In this paper, we consider this analysis model, in the context of a generic missing data problem (e.g., compressed sensing, inpainting, source separation, etc.). Our work proposes a uniqueness result for the solution of this problem, based on properties of the analysis operator and the measurement matrix. This paper also considers two pursuit algorithms for solving the missing data problem, an L1-based and a new greedy method. Our simulations demonstrate the appeal of the analysis model, and the success of the pursuit techniques presented.

This paper describes frequency-domain nonlinear mixture of beamformers that can extract a speech source from a known direction when there are fewer microphones than sources (the underdetermined case). Our approach models the data in each frequency bin via Gaussian mixture distributions, which can be learned using the expectation maximization algorithm. The model learning is performed using the observed mixture signals only, and no prior training is required. Nonlinear beamformers are then developed based on this model. The proposed estimators are a nonlinear weighted sum of linear minimum mean square error or minimum variance distortionless response beamformers. The resulting nonlinear beamformers do not need to know or estimate the number of sources, and can be applied to microphone arrays with two or more microphones. We test and evaluate the described methods on underdetermined speech mixtures.

A new dictionary selection approach for sparse coding, called parametric dictionary design, has recently been introduced. The aim is to choose a dictionary from a class of admissible dictionaries which can be presented parametrically. The designed dictionary satisfies a constraint, here the incoherence property, which can help conventional sparse coding methods to find sparser solutions in average. In this paper, an extra constraint will be applied on the parametric dictionaries to find a structured dictionary. Various structures can be imposed on dictionaries to promote a correlation between the atoms. We intentionally choose a structure to implement the dictionary using a set of filter banks. This indeed helps to implement the dictionary-signal multiplications more efficiently. The price we pay for the extra structure is that the designed dictionary is not as incoherent as unstructured parametric designed dictionaries.

Sparse representations have proved a powerful tool in the analysis and processing of audio signals and already lie at the heart of popular coding standards such as MP3 and Dolby AAC. In this paper we give an overview of a number of current and emerging applications of sparse representations in areas from audio coding, audio enhancement and music transcription to blind source separation solutions that can solve the “cocktail party problem.” In each case we will show how the prior assumption that the audio signals are approximately sparse in some time-frequency representation allows us to address the associated signal processing task.

When sampling signals below the Nyquist rate, efficient and accurate reconstruction is nevertheless possible, whenever the sampling system is well behaved and the signal is well approximated by a sparse vector. This statement has been formalised in the recently developed theory of compressed sensing, which developed conditions on the sampling system and proved the performance of several efficient algorithms for signal reconstruction under these conditions. In this paper, we prove that a very simple and efficient algorithm, known as Iterative Hard Thresholding, has near optimal performance guarantees rivalling those derived for other state of the art approaches.

Sparse modeling of signals has recently received a lot of attention. Often, a linear under-determined generative model for the signals of interest is proposed and a sparsity constraint imposed on the representation. When the generative model is not given, choosing an appropriate generative model is important, so that the given class of signals has approximate sparse representations. In this paper we introduce a new scheme for dictionary learning and impose an additional constraint to reduce the dictionary size. Small dictionaries are desired for coding applications and more likely to ldquoworkrdquo with suboptimal algorithms such as Basis Pursuit. Another benefit of small dictionaries is their faster implementation, e.g. a reduced number of multiplication/addition in each matrix vector multiplication, which is the bottleneck in sparse approximation algorithms.

Compressed sensing is an emerging signal acquisition technique that enables signals to be sampled well below the Nyquist rate, given that the signal has a sparse representation in an orthonormal basis. In fact, sparsity in an orthonormal basis is only one possible signal model that allows for sampling strategies below the Nyquist rate. In this paper, we consider a more general signal model and assume signals that live on or close to the union of linear subspaces of low dimension. We present sampling theorems for this model that are in the same spirit as the Nyquist-Shannon sampling theorem in that they connect the number of required samples to certain model parameters. Contrary to the Nyquist-Shannon sampling theorem, which gives a necessary and sufficient condition for the number of required samples as well as a simple linear algorithm for signal reconstruction, the model studied here is more complex. We therefore concentrate on two aspects of the signal model, the existence ofonetoonemaps to lower dimensional observation spaces and the smoothness of the inverse map. We show thatalmostalllinear maps areonetoonewhen the observation space is at least of the same dimension as the largest dimension of the convex hull of the union ofanytwosubspaces in the model. However, we also show that in order for the inverse map to have certain smoothness properties such as a given finite Lipschitz constant, the required observation dimension necessarily depends logarithmically on the number of subspaces in the signal model. In other words, while unique linear sampling schemes require a small number of samples depending only on the dimension of the subspaces involved, in order to have stable sampling methods, the number of samples depends necessarily logarithmically on the number of subspaces in the model. These results are then applied to two examples, the standard compressed sensing signal model in which the signal has a sparse representation in an orthonormal basis and to a sparse signal model with additional tree structure.

This paper investigates conditions under which the solution of an underdetermined linear system with minimal lscrpnorm, 02marbitrarily close to 1/radic2 ap 0.707 where sparse recovery with p = 1 fails for at least one m-sparse vector, as well as matrices with delta2marbitrarily close to one where lscr1minimization succeeds for any m-sparse vector. This highlights the pessimism of sparse recovery prediction based on the RIC, and indicates that there is limited room for improving over the best known positive results of Foucart and Lai, which guarantee that lscr1minimization recovers all m-sparse vectors for any matrix with delta2mprecovery (0 les p les 1) with matrices of unit spectral norm, which are expressed in terms of the minimal singular values of 2m-column submatrices. Compared to lscr1minimization, lscrpminimization recovery failure is shown to be only slightly delayed in terms of the RIC values. Furthermore in this case the minimization is nonconvex and it is important to consider the specific minimization algorithm being used. It is shown that when lscrpoptimization is attempted using an iterative reweighted lscr1scheme, failure can still occur for delta2marbitrarily close to 1/radic2.

We attempt to quantify the possible gains that can be achieved by examining a rate-distortion competition between a conventional and a compressive sampling solution to data rate reduction. Simple approximate expression are developed for the minimum bit rate required to obtain the best achievable average performance from the compressive sensing system and the performance that would be achieved if that rate requirement was met. An example of a signal that contains a small number of phasors in Gaussian white noise is used to validate these results and to compare the degradation in performance of the 2 systems when lower bit rates than those required by the theory are employed.

In real time communications, system performance and computational complexities play key roles. Reducing the computational load and providing accurate performances are the main challenges in present systems. In this paper, a blind equalization (BE) with affordable complexity and good performance in large constellation MIMO systems is proposed. Saving computational cost happens both in the signal separation part and in signal detection part. First, based on binary phase shift keying (BPSK) or quadrature amplitude modulation (QAM) signal characteristics, an efficient and simple nonlinear function for the independent component analysis (ICA) is introduced. Second, using the idea of the sphere decoding (SD), we choose the soft information of channels smartly and overcome the so-called curse of dimensionality of the expectation maximization (EM) algorithm to enhance the final results.

This correspondence explores the efficiency of embedding information within scalar quantizers. The aim being to efficiently include additional information within an existing coding scheme. Two different information embedding problems are considered: one using post-quantization embedding (PQE) and the other using joint quantization embedding. High-resolution bounds are derived for the achievable embedding rates in both these problems. Surprisingly these results show that it is possible to ldquosoak uprdquo the inefficiencies in scalar quantization through the embedding process. The correspondence concludes by considering some practical examples of low embedding rate coding strategies.

Sparse signal approximations are approximations that use only a small number of elementary waveforms to describe a signal. In this paper we proof the convergence of an iterative hard thresholding algorithm and show, that the fixed points of that algorithm are local minima of the sparse approximation cost function, which measures both, the reconstruction error and the number of elements in the representation. Simulation results suggest that the algorithm is comparable in performance to a commonly used alternative method.

Sparse coding is a new field in signal processing with possible applications to source coding. In this paper we present a new method that combines the problems of sparse signal approximation with coefficient quantization. This method uses overcomplete dictionaries and exploits signal redundancy. The proposed method will be derived as an extension of a recently presented method (iterative thresholding) to find sparse representations of signals. Because in digital communication and storage we need a quantized representation of the signal, instead of quantization of sparse representations a posteriori, we propose a refined method that combines sparse approximation and quantization. To compare the proposed method to a posteriori quantization, we present an audio example.

Redundancy reduction has been proposed as the main computational process in the primary sensory pathways in the mammalian brain. This idea has led to the development of sparse coding techniques, which are exploited in this article to extract salient structure from musical signals. In particular, we use a sparse coding formulation within a generative model that explicitly enforces shift-invariance. Previous work has applied these methods to relatively small problem sizes. In this paper, we present a subset selection step to reduce the computational complexity of these methods, which then enables us to use the sparse coding approach for many real world applications. We demonstrate the algorithm's potential on two tasks in music analysis: the extraction of individual notes from polyphonic piano music and single-channel blind source separation.

We use Bayesian statistics to study the dictionary learning problem in which an over-complete generative signal model has to be adapted for optimally sparse signal representations. With such a formulation we develop a stochastic gradient learning algorithm based on importance sampling techniques to minimise the negative marginal log-likelihood. As this likelihood is not available analytically, approximations have to be utilised. The importance sampling Monte Carlo marginalisation proposed here improves on previous methods and addresses three main issues: (1) bias of the gradient estimate; (2) multi-modality of the distribution to be approximated; and (3) computational efficiency. Experimental results show the advantages of the new method when compared to previous techniques. The gained efficiency allows the treatment of large scale problems in a statistically sound framework as demonstrated here by the extraction of individual piano notes from a polyphonic piano recording.

Note onset detection and localization is useful in a number of analysis and indexing techniques for musical signals. The usual way to detect onsets is to look for "transient" regions in the signal, a notion that leads to many definitions: a sudden burst of energy, a change in the short-time spectrum of the signal or in the statistical properties, etc. The goal of this paper is to review, categorize, and compare some of the most commonly used techniques for onset detection, and to present possible enhancements. We discuss methods based on the use of explicitly predefined signal features: the signal's amplitude envelope, spectral magnitudes and phases, time-frequency representations; and methods based on probabilistic signal models: model-based change point detection, surprise signals, etc. Using a choice of test cases, we provide some guidelines for choosing the appropriate method for a given application.

Many time-series in engineering arise from a sparse mixture of individual components. Sparse coding can be used to decompose such signals into a set of functions. Most sparse coding algorithms divide the signal into blocks. The functions learned from these blocks are, however, not independent of the temporal alignment of the blocks. We present a fast algorithm for sparse coding that does not depend on the block location. To reduce the dimensionality of the problem, a subspace selection step is used during signal decomposition. Due to this reduction, an iterative reweighted least squares method can be used for the constrained optimisation. We demonstrate the algorithm's abilities by learning functions from a polyphonic piano recording. The found functions represent individual notes and a sparse signal decomposition leads to a transcription of the piano signal.

Video encoding standards, namely MPEG-2, store large amounts of information obtained for compression purposes that can be accessed with minimal decoding. This paper shows that, with proper filtering of motion vectors and DCT coefficients, accurate segmentation results can be achieved by combining both reliable motion estimation and background subtraction. We further present a fine segmentation step that exploits specific blob characteristics to reduce segmentation noise and solve some occlusion problems. Examples using real videos from underground station CCTV cameras show that compressed domain information can be the key for successful surveillance applications where very fast algorithms with high accuracy are required.

We present a study on the combined use of energy and phase information for the detection of onsets in musical signals. The resulting method improves upon both energy-based and phase-based approaches. The detection function, generated from the analysis of the signal in the complex frequency domain is sharp at the position of onsets and smooth everywhere else. Results on a database of recordings show high detection rates for low rates of errors. The approach is more robust than its predecessors both theoretically and practically.

We consider the identifiability of the statistical model for noisy independent component analysis showing that while the mixing process is identifiable, the noise covariance is only partially so. This raises questions as to the performance of certain maximum-likelihood algorithms for blind source separation in the presence of noise.

The problem of separation of audio sources recorded in a real world situation is well established in modern literature. A method to solve this problem is blind source separation (BSS) using independent component analysis (ICA). The recording environment is usually modeled as convolutive. Previous research on ICA of instantaneous mixtures provided solid background for the separation of convolved mixtures. The authors revise current approaches on the subject and propose a fast frequency domain ICA framework, providing a solution for the apparent permutation problem encountered in these methods.

In this article, we give an overview of a range of approaches to the analysis and separation of musical audio. In particular, we consider the problems of automatic music transcription and audio source separation, which are of particular interest to our group. Monophonic music transcription, where a single note is present at one time, can be tackled using an autocorrelation-based method. For polyphonic music transcription, with several notes at any time, other approaches can be used, such as a blackboard model or a multiple-cause/sparse coding method. The latter is based on ideas and methods related to independent component analysis (ICA), a method for sound source separation.