Abstract. In this work we study the behavior of the forward-backward (FB) algorithm when the proximity operator is replaced by a subiterative procedure to approximate a Gaussian denoiser, in a Plug-and-Play (PnP) fashion. Specifically, we consider both analysis and synthesis Gaussian denoisers within a dictionary framework, obtained by unrolling dual-FB iterations or FB iterations, respectively. We analyze the associated minimization problems as well as the asymptotic behavior of the resulting FB-PnP iterations. In particular, we show that the synthesis Gaussian denoising problem can be viewed as a proximity operator. For each case, analysis, and synthesis, we show that the FB-PnP algorithms solve the same problem whether we use only one or an infinite number of subiteration to solve the denoising problem at each iteration. To this aim, we show that each “one subiteration” strategy within the FB-PnP can be interpreted as a primal-dual algorithm when a warm-restart strategy is used. We further present similar results when using a Moreau–Yosida smoothing of the global problem, for an arbitrary number of subiterations. Finally, we provide numerical simulations to illustrate our theoretical results. In particular we first consider a toy compressive sensing example, as well as an image restoration problem in a deep dictionary framework.

Convolutional Dictionary Learning (CDL) is a dictionary learning technique exploiting the translation invariance of elementary signals. In the time-frequency domain, the repetition of elementary frequency patterns can be exploited through the "nonnegative matrix factorization" (NMF) decompositions and extensions, such as semi or complex-NMF, of the spectrogram. We study the links between these two approaches here, and we show in particular that a signal which admits a Convolutive Sparse Coding decomposition admits time-frequency synthesis coefficients that can be decomposed in semi-NMF or complex-NMF. The different approaches are then compared experimentally on synthetic signals.

When dealing with electro or magnetoencephalography records, many supervised prediction tasks are solved by working with covariance matrices to summarize the signals. Learning with these matrices requires the usage of Riemanian geometry to account for their structure. In this paper, we propose a new method to deal with distributions of covariance matrices, and demonstrate its computational efficiency on M/EEG multivariate time series. More specifically, we define a Sliced-Wasserstein distance between measures of symmetric positive definite matrices that comes with strong theoretical guarantees. Then, we take advantage of its properties and kernel methods to apply this discrepancy to brain-age prediction from MEG data, and compare it to state-of-the-art algorithms based on Riemannian geometry. Finally, we show that it is an efficient surrogate to the Wasserstein distance in domain adaptation for Brain Computer Interface applications.

Given a vector $y \in \mathbb{R}^n$ and a matrix $H \in \mathbb{R}^{n\times m}$, the sparse approximation problem $\mathcal P_{0/p}$ asks for a point $x$ such that $\|y - Hx\|_p \leq \alpha$, for a given scalar $\alpha$, minimizing the size of the support $\|x\|_0 := \#\{j \ |\ x_j \neq 0 \}$. Existing convex mixed-integer programming formulations for $\mathcal P_{0/p}$ are of a kind referred to as “big-$M$”, meaning that they involve the use of a bound $M$ on the values of $x$. When a proper value for $M$ is not known beforehand, these formulations are not exact, in the sense that they may fail to recover the wanted global minimizer. In this work, we study the polytopes arising from these formulations and derive valid inequalities for them. We first use these inequalities to design a branch-and-cut algorithm for these models. Additionally, we prove that these inequalities are sufficient to describe the set of feasible supports for $\mathcal P_{0/p}$. Based on this result, we introduce a new (and the first to our knowledge) $M$-independent integer linear programming formulation for $\mathcal P_{0/p}$, which guarantees the recovery of the global minimizer. We propose a practical approach to tackle this formulation, which has exponentially many constraints. The proposed methods are then compared in computational experimentation to test their potential practical contribution.

Sparse coding is now one of the state-of-art approaches for solving inverse problems. In combination with (Fast) Iterative Shrinkage Thresholding Algorithm (ISTA), among other algorithms, one can efficiently get a nice estimator of the sought sparse signal. However, the major drawback of these methods is the tuning of the so-called hyperparameter. In this paper, we first provide an Expectation-Maximization (EM) algorithm to estimate the parameters of a Bernoulli-Gaussian model for denoising a sparse signal corrupted by a white Gaussian noise. Then, building on the Expectation-Maximization interpretation of ISTA, we provide a simple iterative algorithm to blindly estimate all the model parameters in the linear inverse problem context, including the hyperparameter involved in the popular ℓ0regularized minimization. Moreover, the algorithm directly yields an estimator of the sparse signal.

We consider the problem of blind source separation for underdetermined convolutive mixtures. Based on the multiplicative narrowband approximation in the time-frequency domain with the help of the short-time-Fourier-transform (STFT) and the sparse representation of the source signals, we formulate the separation problem in an optimization framework. This framework is then generalized based on the recently investigated convolutive narrowband approximation and the statistics of the room impulse response. Algorithms with convergence proof are then employed to solve the proposed optimization problems. The evaluation of the proposed frameworks and algorithms for synthesized and live recorded mixtures are illustrated. The proposed approaches are also tested for mixtures with input noise. Numerical evaluations show the advantages of the proposed methods.

One of the most general models of music signals considers that such signals can be represented as a sum of two distinct components: a tonal part that is sparse in frequency and temporally stable and a transient (or percussive) part that is composed of short-term broadband sounds. In this paper, we propose a novel hybrid method built upon nonnegative matrix factorization (NMF) that decomposes the time frequency representation of an audio signal into such two components. The tonal part is estimated by a sparse and orthogonal nonnegative decomposition, and the transient part is estimated by a straightforward NMF decomposition constrained by a pre-learned dictionary of smooth spectra. The optimization problem at the heart of our method remains simple with very few hyperparameters and can be solved thanks to simple multiplicative update rules. The extensive benchmark on a large and varied music database against four state of the art harmonic/percussive source separation algorithms demonstrate the merit of the proposed approach.

This paper presents a new method for blind source separation problem in reverberant environments with more sources than microphones. Based on the sparsity property in the time-frequency domain and the low-rank assumption of the spectrogram of the source, the STRAUSS (SparsiTy and low-Rank AmplitUde based Source Separation) method is developed. Numerical evaluations show that the proposed method outperforms the existing multichannel NMF approaches, while it is exclusively based on amplitude information.

In this paper, we propose a supervised multilayer factorization method designed for harmonic/percussive source separation and drum extraction. Our method decomposes the audio signals in sparse orthogonal components which capture the harmonic content, while the drum is represented by an extension of non negative matrix factorization which is able to exploit time-frequency dictionaries to take into account non stationary drum sounds. The drum dictionaries represent various real drum hits and the decomposition has more physical sense and allows for a better interpretation of the results. Experiments on real music data for a harmonic/percussive source separation task show that our method outperforms other state of the art algorithms. Finally, our method is very robust to non stationary harmonic sources that are usually poorly decomposed by existing methods.

We consider the problem of extracting the source signals from an under-determined convolutive mixture, assuming known filters. We start from its formulation as a minimization of a convex functional, combining a classical I2discrepancy term between the observed mixture and the one reconstructed from the estimated sources, and a sparse regularization term of source coefficients in a time-frequency domain. We then introduce a first kind of structure, using a hybrid model. Finally, we embed the previously introduced Windowed-Group-Lasso operator into the iterative thresholding/shrinkage algorithm, in order to take into account some structures inside each layers of time-frequency representations. Intensive numerical studies confirm the benefits of such an approach.

We consider the audio declipping problem by using iterative thresholding algorithms and the principle of social sparsity. This recently introduced approach features thresholding/shrinkage operators which allow to model dependencies between neighboring coefficients in expansions with time-frequency dictionaries. A new unconstrained convex formulation of the audio declipping problem is introduced. The chosen structured thresholding operators are the so called windowed group-Lasso and the persistent empirical Wiener. The usage of these operators significantly improves the quality of the reconstruction, compared to simple soft-thresholding. The resulting algorithm is fast, simple to implement, and it outperforms the state of the art in terms of signal to noise ratio.

Imaging inverse problems can be formulated as an optimization problem and solved thanks to algorithms such as forward-backward or ISTA (Iterative Shrinkage/Thresholding Algorithm) for which non smooth functionals with sparsity constraints can be minimized efficiently. However, the soft thresholding operator involved in this algorithm leads to a biased estimation of large coefficients. That is why a step allowing to reduce this bias is introduced in practice. Indeed, in the statistical community, a large variety of thresholding operators have been studied to avoid the biased estimation of large coefficients; for instance, the non negative Garrote or the the SCAD thresholding. One can associate a non convex penalty to these operators. We study the convergence properties of ISTA, possibly relaxed, with any thresholding rule and show that they correspond to a semi-convex penalty. The effectiveness of this approach is illustrated on image inverse problems.

The main contribution presented here is an adaptive/unsupervised iterative thresholding algorithm for sparse representation of signals which can be modeled as the sum of two components. Such Hybrid or Morphological representations are known to be well adapted for applications in image or audio signal processing. The proposed algorithm uses a Bernoulli-Gaussian prior on the synthesis coefficients of the signal, with morphological depending parameters. Using an EM-framework introduced by Figueiredo and Nowak in the case of the convex ℓ1prior, we derive an unsupervised algorithm in the spirit of ISTA, with iteratively adapted thresholding/shrinkage.

We consider the problem of extracting the source signals from an under-determined convolutive mixture assuming known mixing filters. State-of-the-art methods operate in the time-frequency domain and rely on narrowband approximation of the convolutive mixing process by complex-valued multiplication in each frequency bin. The source signals are then estimated by minimizing either a mixture fitting cost or aℓ1source sparsity cost, under possible constraints on the number of active sources. In this paper, we define a wideband ℓ2mixture fitting cost circumventing the above approximation and investigate the use of a ℓ1,2mixed-norm cost promoting disjointness of the source time-frequency representations. We design a family of convex functionals combining these costs and derive suitable optimization algorithms. Experiments indicate that the proposed wideband methods result in a signal-to-distortion ratio improvement of 2 to 5 dB compared to the state-of-the-art on reverberant speech mixtures.

We address the problem of learning classifiers using several kernel functions. On the contrary to many contributions in the field of learning from different sources of information using kernels, we here do not assume that the kernels used are positive definite. The learning problem that we are interested in involves a misclassification loss term and a regularization term that is expressed by means of a mixed norm. The use of a mixed norm allows us to enforce some sparsity structure, a particular case of which is, for instance, the Group Lasso. We solve the convex problem by employing proximal minimization algorithms, which can be viewed as refined versions of gradient descent procedures capable of naturally dealing with nondifferentiability. A numerical simulation on a Uci dataset shows the modularity of our approach.

We study the behavior of hybrid random waveform models for audio signals, involving sparse random series of waveforms, with random coefficients. Similar approaches have been considered in the recent years. However, these do generally not rely on explicit models, and are of more "algorithmical" nature. The models we propose allow us to analyze mathematical properties of such signals and corresponding estimators, and derive estimation algorithms, which do not rely on complex optimization techniques