Weighted least squares are often used to approximate log-likelihoods when solving inverse problems involving non-Gaussian noise as they are more appealing from an optimization perspective. Although a theoretical expression of the weights can be derived for specific noises, this may become intractable for more general noises. Moreover, such theoretical weights can be detrimental to the efficiency of optimization algorithms. To remedy these issues, we propose in this work to learn the weights from data so as to adapt to any general noise while maintaining the efficiency of optimization. The proposed pipeline combines a weight estimation module with an unrolled optimization algorithm. The weight estimation module and a few parameters of the unrolled algorithm are trained together in an end-to-end manner. We demonstrate the effectiveness of the proposed methodology in the context of Poisson image deconvolution.

Abstract We consider the minimization of $$\ell _0$$ ℓ 0 -regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence and the logistic regression, possibly combined with an $$\ell _2$$ ℓ 2 regularization. We first prove the existence of global minimizers for such problems and characterize their local minimizers. Then, we propose a new class of continuous relaxations of the $$\ell _0$$ ℓ 0 pseudo-norm, termed as $$\ell _0$$ ℓ 0 Bregman Relaxations (B-rex). They are defined in terms of suitable Bregman distances and lead to exact continuous relaxations of the original $$\ell _0$$ ℓ 0 -regularized problem in the sense that they do not alter its set of global minimizers and reduce its non-convexity by eliminating certain local minimizers. Both features make such relaxed problems more amenable to be solved by standard non-convex optimization algorithms. In this spirit, we consider the proximal gradient algorithm and provide explicit computation of proximal points for the B-rex penalty in several cases. Finally, we report a set of numerical results illustrating the geometrical behavior of the proposed B-rex penalty for different choices of the underlying Bregman distance, its relation with convex envelopes, as well as its exact relaxation properties in 1D/2D and higher dimensions.

Given a forward mapping Φ : R^N → R^M and a point x* ∈ R^N , the region {x ∈ R^N , ||Φ(x) − Φ(x*)|| ≤ ε}, where ε ≥ 0 is a perturbation amplitude, represents the set of all possible inputs x that could have produced the measurement Φ(x*) within an acceptable error margin. This set is related to uncertainty analysis, a key challenge in inverse problems. In this work, we develop a numerical algorithm called Jackpot (Jacobian Kernel Projection Optimization) which approximates this set with a low-dimensional adversarial manifold. The proposed algorithm leverages automatic differentation, allowing it to handle complex, high dimensional mappings such as those found when dealing with dynamical systems or neural networks. We demonstrate the effectiveness of our algorithm on various challenging large-scale, non-linear problems including parameter identification in dynamical systems and blind image deblurring.

Denoising is omnipresent in image processing. It is usually addressed with algorithms relying on a set of hyperparameters that control the quality of the recovered image. Manual tuning of those parameters can be a daunting task, which calls for the development of automatic tuning methods. Given a denoising algorithm, the best set of parameters is the one that minimizes the error between denoised and ground-truth images. Clearly, this ideal approach is unrealistic, as the ground-truth images are unknown in practice. In this work, we propose unsupervised cost functions — i.e., that only require the noisy image — together with reach this ideal gold standard performance. Specifically, the proposed approach makes it possible to obtain an average PSNR output within less than 1% of the best achievable PSNR

Hyper-parameter tuning, and especially regularisation parameter estimation, is a challenging but essential task when solving inverse problems. The solution is obtained here through the minimization of a functional composed of a data fidelity term and a regularization term. Those terms are balanced through a (or several) regularisation parameter(s) whose estimation is made under an unrolled strategy together with the inverse problem solving. The resulting network is trained while incorporating information on the model through Maximum a Posteriori estimation which drastically decreases the amount of data needed for the training and results in better estimation results. The performances are demonstrated in a deconvolution context where the regularisation is performed in the wavelet domain.

The Gap safe screening technique is a powerful tool to accelerate the convergence of sparse optimization solvers. Its performance is largely based on the ability to determine the smallest “sphere”, centered at a given feasible dual point, that contains the dual solution. This can be achieved through an inner sphere refinement loop, applied at each screening step. In this work, we show that this refinement loop actually converges to the solution of a fixed-point equation for which we derive a closed-form expression for two common loss functions. This allows us to develop an analytic (i.e., non iterative), more concise and theoretically-grounded variant of the sphere refinement step.

Safe screening rules are powerful tools to accelerate iterative solvers in sparse regression problems. They allow early identification of inactive coordinates (i.e., those not belonging to the support of the solution) which can thus be screened out in the course of iterations. In this paper, we extend the GAP Safe screening rule to the ℓ1-regularized Kullback-Leibler divergence which does not fulfil the regularity assumptions made in previous works. The proposed approach is experimentally validated on synthetic and real count data sets.

Sparse optimization problems are ubiquitous in many fields such as statistics, signal/image processing and machine learning. This has led to the birth of many iterative algorithms to solve them. A powerful strategy to boost the performance of these algorithms is known as safe screening: it allows the early identification of zero coordinates in the solution, which can then be eliminated to reduce the problem's size and accelerate convergence. In this work, we extend the existing Gap Safe screening framework by relaxing the global strong-concavity assumption on the dual cost function. Instead, we exploit local regularity properties, that is, strong concavity on well-chosen subsets of the domain. The non-negativity constraint is also integrated to the existing framework. Besides making safe screening possible to a broader class of functions that includes $\beta$-divergences (e.g., the Kullback-Leibler divergence), the proposed approach also improves upon the existing Gap Safe screening rules on previously applicable cases (e.g., logistic regression). The proposed general framework is exemplified by some notable particular cases: logistic function, $\beta=1.5$ and Kullback-Leibler divergences. Finally, we showcase the effectiveness of the proposed screening rules with different solvers (coordinate descent, multiplicative-update and proximal gradient algorithms) and different datasets (binary classification, hyperspectral and count data).

On-grid based direction-of-arrival (DOA) estimation methods rely on the resolution of a difficult group-sparse optimization problem that involves the ℓ2,0pseudo-norm. In this work, we show that an exact relaxation of this problem can be obtained by replacing the ℓ2,0term with a group minimax concave penalty with suitable parameters. This relaxation is more amenable to non-convex optimization algorithms as it is continuous and admits less local (not global) minimizers than the initial ℓ2,0-regularized criteria. We then show on numerical simulations that the minimization of the proposed relaxation with an iteratively reweighted ℓ2,0algorithm leads to an improved performance over traditional approaches.

Non-negative matrix factorization with transform learning (TL-NMF) aims at estimating a short-time orthogonal transform that projects temporal data into a domain that is more amenable to NMF than off-the-shelf time-frequency transforms. In this work, we study the identifiability of TL-NMF under the Gaussian composite model. We prove that one can uniquely identify row-spaces of the orthogonal transform by optimizing the likelihood function of the model. This result is illustrated on a toy source separation problem which demonstrates the ability of TL-NMF to learn a suitable orthogonal basis.

Single-particle cryo-electron microscopy (cryo-EM) reconstructs the three-dimensional (3D) structure of biomolecules from a large set of 2D projection images with random and unknown orientations. A crucial step in the single-particle cryo-EM pipeline is 3D refinement, which resolves a highresolution 3D structure from an initial approximate volume by refining the estimation of the orientation of each projection. In this work, we propose a new approach that refines the projection angles on the continuum. We formulate the optimization problem over the density map and the orientations jointly. The density map is updated using the efficient alternating-direction method of multipliers, while the orientations are updated through a semicoordinate- wise gradient descent for which we provide an explicit derivation of the gradient. Our method eliminates the requirement for a fine discretization of the orientation space and does away with the classical but computationally expensive templatematching step. Numerical results demonstrate the feasibility and performance of our approach compared to several baselines.

Lemma 4.4 in [E. Soubies, L. Blanc-Féraud and G. Aubert, SIAM J. Imaging Sci., 8 (2015), pp. 1607--1639] is wrong for local minimizers of the continuous exact $\ell_0$ (CEL0) functional. The argument used to conclude the proof of this lemma is not sufficient in the case of local minimizers. In this note, we supply a revision of this lemma where new results are established for local minimizers. Theorem 4.8 in that paper remains unchanged but the proof has to be rewritten according to the new version of the lemma. Finally, some remarks of this paper are also rewritten using the corrected lemma.

Within the framework of the $\ell_0$ regularized least squares problem, we focus, in this paper, on nonconvex continuous penalties approximating the $\ell_0$-norm. Such penalties are known to better promote sparsity than the $\ell_1$ convex relaxation. Based on some results in one dimension and in the case of orthogonal matrices, we propose the continuous exact $\ell_0$ penalty (CEL0) leading to a tight continuous relaxation of the $\ell_2-\ell_0$ problem. The global minimizers of the CEL0 functional contain the global minimizers of $\ell_2 - \ell_0$, and from each global minimizer of CEL0 one can easily identify a global minimizer of $\ell_2 - \ell_0$. We also demonstrate that from each local minimizer of the CEL0 functional, a local minimizer of $\ell_2 - \ell_0$ is easy to obtain. Moreover, some strict local minimizers of the initial functional are eliminated with the proposed tight relaxation. Then solving the initial $\ell_2 - \ell_0$ problem is equivalent, in a sense, to solving it by replacing the $\ell_0$-norm with the CEL0 which provides better properties for the objective function in terms of minimization, such as the continuity and the convexity with respect to each direction of the standard $\mathbb{R}^N$ basis, although the problem remains nonconvex. Finally, recent nonsmooth nonconvex algorithms are used to address this relaxed problem within a macro algorithm ensuring the convergence to a critical point of the relaxed functional which is also a (local) optimum of the initial problem.

Super-resolution microscopy techniques allow to overstep the diffraction limit of conventional optics. Theses techniques are very promising since they give access to the visualisation of finer structures which is of fundamental importance in biology. In this paper we deal with Multiple-Angle Total Internal Reflection Microscopy (MA-TIRFM) which allows to reconstruct 3D sub-cellular structures of a single layer of ~ 300 nm behind the glass coverslip with a high axial resolution. The 3D volume reconstruction from a set of 2D measurements is an ill-posed inverse problem and a regularization is essential. Our aim in this work is to propose a new reconstruction method for sparse structures robust to Poisson noise and background fluorescence. The sparse property of the solution can be seen as a regularization using the `£° norm'. In order to solve this combinatorial problem, we propose a new algorithm based on smoothed `£° norm' allowing minimizing a non convex energy, composed of the Kullback-Leibler divergence data term and the £° regularization term, in a Graduated Non Convexity framework.