State-space models are a statistical framework for modelling temporal phenomena via a hidden state. In this framework, the hidden state is not observed, and instead a series of related observations are obtained. A state-space model is defined by the state dynamics, which is encoded as a distribution. The parameters of this distribution are often unknown, and must be estimated in order to perform inference. In real-world systems, it is common that not all dimensions of the hidden state directly interact, which implies a sparse system. Most parameter estimation methods for state-space models cannot recover sparsity. In this work, we propose PolyGrad, a fully automatic method for obtaining sparse estimates of the state interactions of a non-linear state-space model via a polynomial approximation. This novel versatile methodology allows to infer both the structure and values of a generic parameterisation of a state-space model. The proposed method is computationally efficient and can represent a large class of complex systems.

We study the variational inference problem of minimizing a regularized Rényi divergence over an exponential family. We propose to solve this problem with a Bregman proximal gradient algorithm. We propose a sampling-based algorithm to cover the black-box setting, corresponding to a stochastic Bregman proximal gradient algorithm with biased gradient estimator. We show that the resulting algorithms can be seen as relaxed moment-matching algorithms with an additional proximal step. Using Bregman updates instead of Euclidean ones allows us to exploit the geometry of our approximate model. We prove strong convergence guarantees for both our deterministic and stochastic algorithms using this viewpoint, including monotonic decrease of the objective, convergence to a stationary point or to the minimizer, and geometric convergence rates. These new theoretical insights lead to a versatile, robust, and competitive method, as illustrated by numerical experiments

Adaptive importance sampling (AIS) algorithms are a rising methodology in signal processing, statistics, and machine learning. An effective adaptation of the proposals is key for the success of AIS. Recent works have shown that gradient information about the involved target density can greatly boost performance, but its applicability is restricted to differentiable targets. In this letter, we propose a proximal Newton adaptive importance sampler for the estimation of expectations with respect to non-smooth target distributions. We implement a scaled Newton proximal gradient method to adapt the proposal distributions, enabling efficient and optimized moves even when the target distribution lacks differentiability. We show the good performance of the algorithm in two scenarios: one with convex constraints and another with non-smooth sparse priors.

Antimicrobial resistance (AMR) is a significant global health challenge caused by the misuse and overuse of antibiotics in various sectors, leading to the development of resistant bacteria. In such infections, the first-line antibiotics intended for specific diseases become ineffective, necessitating the repurposing of other antibiotics for treatment. To address this, we have developed a new algorithm for general-purpose drug repositioning based on a matrix completion framework on graphs. Our probabilistic approach combines deep matrix factorization with graph learning to achieve precise drug repurposing. In this study, we curated a new dataset on antibiotic-bacteria associations. Applying our proposed method to this dataset demonstrates that our approach outperforms benchmarks in both general-purpose drug repositioning and three specific AMR case studies.

black-box global optimization aims at minimizing an objective function whose analytical form is not known. To do so, many state-of-the-art methods rely on sampling-based strategies, where sampling distributions are built in an iterative fashion, so that their mass concentrate where the objective function is low. Despite empirical success, the theoretical study of these methods remains difficult. In this work, we introduce a new framework, based on divergence-decrease conditions, to study and design black-box global optimization algorithms. Our approach allows to establish and quantify the improvement of sampling distributions at each iteration, in terms of expected value or quantile of the objective. We show that the information-geometric optimization approach fits within our framework, yielding a new approach for its analysis. We also establish sampling distribution improvement results for two novel algorithms, one related with the cross-entropy approach with mixture models, and another one using heavy-tailed sampling distributions.

Adaptive importance sampling (AIS) algorithms are widely used to approximate expectations with respect to complicated target probability distributions. When the target has heavy tails, existing AIS algorithms can provide inconsistent estimators or exhibit slow convergence, as they often neglect the target’s tail behaviour. To avoid this pitfall, we propose an AIS algorithm that approximates the target by Student-t proposal distributions. We adapt location and scale parameters by matching the escort moments - which are defined even for heavy-tailed distributions - of the target and proposal. These updates minimize the $\alpha$-divergence between the target and the proposal, thereby connecting with variational inference. We then show that the $\alpha$-divergence can be approximated by a generalized notion of effective sample size and leverage this new perspective to adapt the tail parameter with Bayesian optimization. We demonstrate the efficacy of our approach through applications to synthetic targets and a Bayesian Student-t regression task on a real example with clinical trial data.

State-space models (SSMs) are common tools in time-series analysis for inference and prediction. SSMs are versatile probabilistic models that allow for Bayesian inference by describing a (generally Markovian) latent process. However, the parameters of that latent process are often unknown and must be estimated. In this paper, we consider the parameter estimation in a SSM with a non-Markovian linear-Gaussian latent process. This process is described as a vector auto-regressive with p unknown matrices. Our algorithm LaGrangEM estimates these matrices through an expectation-maximization algorithm that exploits a graphical interpretation of the latent process in order to define prior knowledge about the unknown parameters. We connect the new algorithm with existing approaches such as Granger causality and graphical inference in SSMs. We discuss the strong potential of the algorithm to bring interpretability, e.g., in estimating causal relationships and their delays. The numerical experiments also show a superiority in performance.

Time-series datasets are central in machine learning with applications in numerous fields of science and engineering, such as biomedicine, Earth observation, and network analysis. Extensive research exists on state-space models (SSMs), which are powerful mathematical tools that allow for probabilistic and interpretable learning on time series. Learning the model parameters in SSMs is arguably one of the most complicated tasks, and the inclusion of prior knowledge is known to both ease the interpretation but also to complicate the inferential tasks. Very recent works have attempted to incorporate a graphical perspective on some of those model parameters, but they present notable limitations that this work addresses. More generally, existing graphical modeling tools are designed to incorporate either static information, focusing on statistical dependencies among independent random variables (e.g., graphical Lasso approach), or dynamic information, emphasizing causal relationships among time series samples (e.g., graphical Granger approaches). However, there are no joint approaches combining static and dynamic graphical modeling within the context of SSMs. This work proposes a novel approach to fill this gap by introducing a joint graphical modeling framework that bridges the graphical Lasso model and a causal-based graphical approach for the linear-Gaussian SSM. We present DGLASSO (Dynamic Graphical Lasso), a new inference method within this framework that implements an efficient block alternating majorization-minimization algorithm. The algorithm's convergence is established by departing from modern tools from nonlinear analysis. Experimental validation on various synthetic data showcases the effectiveness of the proposed model and inference algorithm. This work will significantly contribute to the understanding and utilization of time-series data in diverse scientific and engineering applications where incorporating a graphical approach is essential to perform the inference.

State-space models (SSMs) are a common tool for modeling multi-variate discrete-time signals. The linear-Gaussian (LG) SSM is widely applied as it allows for a closed-form solution at inference, if the model parameters are known. However, they are rarely available in real-world problems and must be estimated. Promoting sparsity of these parameters favours both interpretability and tractable inference. In this work, we propose GraphIT, a majorization-minimization (MM) algorithm for estimating the linear operator in the state equation of an LG-SSM under sparse prior. A versatile family of non-convex regularization potentials is proposed. The MM method relies on tools inherited from the expectation-maximization methodology and the iterated reweighted-l1 approach. In particular, we derive a suitable convex upper bound for the objective function, that we then minimize using a proximal splitting algorithm. Numerical experiments illustrate the benefits of the proposed inference technique.

Independent vector analysis (IVA) generalizes independent component analysis (ICA) to multiple datasets, and when used with a multivariate Gaussian model (IVA-G), provides a powerful tool for joint analysis of multiple datasets in an array of applications. While IVA-G enjoys uniqueness guarantees, the current solution to the problem exhibits significant variability across runs necessitating the use of a scheme for selecting the most consistent one, which is costly. In this paper, we present a penalized maximum-likelihood framework for the problem, which enables us to derive a non-convex cost function that depends on the precision matrices of the source component vectors, the main mechanism by which IVA-G leverages correlation across the datasets. By adding a quadratic regularization, a block-coordinate proximal algorithm is shown to offer a suitable solution to this minimization problem. The proposed method also provides convergence guarantees that are lacking in other state-of-the-art approaches to the problem. This also allows us to obtain overall slightly better performance, and in particular, we show that our method yields better estimation in average than the current IVA-G algorithm for various source numbers, datasets, and degrees of correlation across the data.

Simulated annealing is a popular approach to solve nonconvex and black-box optimization problems. It consists in running a non-homogeneous Markov chain to sample from a sequence of Boltzmann probability distributions. This sequence is controlled by a cooling schedule, which governs the concentration of the mass of the Boltzmann distributions around the global minimizers. However, convergence is often slow, difficult to assess, and requires a fixed cooling schedule. We propose here a new simulated annealing algorithm with adaptive cooling schedule, which draws samples from variational approximations of the Boltzmann distributions. Our approach is theoretically sound and relies on an alternating Bregman proximal-gradient scheme minimizing a regularized Rényi divergence. Numerical experiments illustrate the performance of the method.

Abstract. Most optimization problems arising in imaging science involve high-dimensional linear operators and their adjoints. In the implementations of these operators, changes may be introduced for various practical considerations (e.g., memory limitation, computational cost, convergence speed), leading to an adjoint mismatch. This occurs for the X-ray tomographic inverse problems found in computed tomography (CT), where a surrogate operator often replaces the adjoint of the measurement operator (called the projector). The resulting adjoint mismatch can jeopardize the convergence properties of iterative schemes used for image recovery. In this paper, we study the theoretical behavior of a panel of primal-dual proximal algorithms, which rely on forward-backward-(forward) splitting schemes when an adjoint mismatch occurs. We analyze these algorithms by focusing on the resolution of possibly nonsmooth convex penalized minimization problems in an infinite-dimensional setting. Using tools from fixed point theory, we show that they can solve monotone inclusions beyond minimization problems. Such findings indicate that these algorithms can be seen as a generalization of classical primal-dual formulations. The applicability of our findings is also demonstrated through two numerical experiments in the context of CT image reconstruction.

Denoising, detrending, deconvolution: usual restoration tasks, traditionally decoupled. Coupled formulations entail complex ill-posed inverse problems. We propose PENDANTSS for joint trend removal and blind deconvolution of sparse peak-like signals. It blends a parsimonious prior with the hypothesis that smooth trend and noise can somewhat be separated by low-pass filtering. We combine the generalized quasi-norm ratio SOOT/SPOQ sparse penalties $\ell_p/\ell_q$ with the BEADS ternary assisted source separation algorithm. This results in a both convergent and efficient tool, with a novel Trust-Region block alternating variable metric forward-backward approach. It outperforms comparable methods, when applied to typically peaked analytical chemistry signals. Reproducible code is provided.

In this paper, we introduce a variational Bayesian algorithm (VBA) for image blind deconvolution. Our VBA generic framework incorporates smoothness priors on the unknown blur/image and possible affine constraints (e.g., sum to one) on the blur kernel, integrating the VBA within a neural network paradigm following an unrolling methodology. The proposed architecture is trained in a supervised fashion, which allows us to optimally set two key hyperparameters of the VBA model and leads to further improvements in terms of resulting visual quality. Various experiments involving grayscale/color images and diverse kernel shapes, are performed. The numerical examples illustrate the high performance of our approach when compared to state-of-the-art techniques based on optimization, Bayesian estimation, or deep learning.

Co-administration of two or more drugs simultaneously can result in adverse drug reactions. Identifying drug-drug interactions (DDIs) is necessary, especially for drug development and for repurposing old drugs. DDI prediction can be viewed as a matrix completion task, for which matrix factorization (MF) appears as a suitable solution. This paper presents a novel Graph Regularized Probabilistic Matrix Factorization (GRPMF) method, which incorporates expert knowledge through a novel graph-based regularization strategy within an MF framework. An efficient and sounded optimization algorithm is proposed to solve the resulting non-convex problem in an alternating fashion. The performance of the proposed method is evaluated through the DrugBank dataset, and comparisons are provided against state-of-the-art techniques. The results demonstrate the superior performance of GRPMF when compared to its counterparts.

Simulated annealing (SA) is a widely used approach to solve global optimization problems in signal processing. The initial non-convex problem is recast as the exploration of a sequence of Boltzmann probability distributions, which are increasingly harder to sample from. They are parametrized by a temperature that is iteratively decreased, following the so-called cooling schedule. Convergence results of SA methods usually require the cooling schedule to be set a priori with slow decay. In this work, we introduce a new SA approach that selects the cooling schedule on the fly. To do so, each Boltzmann distribution is approximated by a proposal density, which is also sequentially adapted. Starting from a variational formulation of the problem of joint temperature and proposal adaptation, we derive an alternating Bregman proximal algorithm to minimize the resulting cost, obtaining the sequence of Boltzmann distributions and proposals. Numerical experiments in an idealized setting illustrate the potential of our method compared with state-of-the-art SA algorithms.

This work addresses the problem of hyperspectral image classification when the number of labeled samples is very small (few shot learning). Our work is based on the recently proposed framework of convolutional transform learning. In this work, we propose a semi-supervised version of deep convolutional transform learning. We compare with four recent studies which are tailored for solving the few-shot learning problem in hyperspectral classification. Results show that our method can improve over the state-of-the-art.

Conventional Convolutional Neural Network (CNN) based clustering formulations are based on the encoder-decoder based framework, where the clustering loss is incorporated after the encoder network. The problem with this approach is that it requires training an additional decoder network; this, in turn, means learning additional weights which can lead to over-fitting in data constrained scenarios. This work introduces a Deep Convolutional Transform Learning (DCTL) based clustering framework. The advantage of our proposed formulation is that we do not require learning the additional decoder network. Therefore our formulation is less prone to over-fitting. Comparison with state-of-the-art deep learning based clustering solutions on benchmark image datasets shows that our proposed method improves over the rest in challenging scenarios where there are many clusters with limited samples.

Positron emission tomography (PET) is a quantitative imaging modality widely used in oncology, neurology, and pharmacology. The data acquired by a PET scanner correspond to projections of the concentration activity, assumed to follow a Poisson distribution. The reconstruction of images from tomographic projections corrupted by Poisson noise is a challenging ill-posed large-scale inverse problem. Several available solvers use the majorization-minimization (MM) principle, though relying on various construction strategies with a lack of unifying framework. This work fills the gap by introducing the concept of Bregman majorization. This leads to a unified view of MM-based methods for image reconstruction in the presence of Poisson noise. From this general approach, we exhibit three algorithmic solutions and compare their computational efficiency on a problem of dynamic PET image reconstruction, either using GPU or CPU processing.

Standard few-shot benchmarks are often built upon simplifying assumptions on the query sets, which may not always hold in practice. In particular, for each task at testing time, the classes effectively present in the unlabeled query set are known a priori, and correspond exactly to the set of classes represented in the labeled support set. We relax these assumptions and extend current benchmarks, so that the query-set classes of a given task are unknown, but just belong to a much larger set of possible classes. Our setting could be viewed as an instance of the challenging yet practical problem of extremely imbalanced $K$-way classification, $K$ being much larger than the values typically used in standard benchmarks, and with potentially irrelevant supervision from the support set. Expectedly, our setting incurs drops in the performances of state-of-the-art methods. Motivated by these observations, we introduce a \textbf{P}rim\textbf{A}l \textbf{D}ual Minimum \textbf{D}escription \textbf{LE}ngth (\textbf{PADDLE}) formulation, which balances data-fitting accuracy and model complexity for a given few-shot task, under supervision constraints from the support set. Our constrained MDL-like objective promotes competition among a large set of possible classes, preserving only effective classes that befit better the data of a few-shot task. It is hyper-parameter free, and could be applied on top of any base-class training. Furthermore, we derive a fast block coordinate descent algorithm for optimizing our objective, with convergence guarantee, and a linear computational complexity at each iteration. Comprehensive experiments over the standard few-shot datasets and the more realistic and challenging \textit{i-Nat} dataset show highly competitive performances of our method, more so when the numbers of possible classes in the tasks increase. Our code is publicly available at \url{https://github.com/SegoleneMartin/PADDLE}.

This work addresses the resolution of penalized least-squares problems using the proximal gradient algorithm (PGA). PGA can be accelerated by preconditioning strategies. However, typical effective choices of preconditioners may correspond to intricate matrices that are not easily inverted, leading to increased complexity in the computation of the proximity step. To relax these requirements, we propose an unmatched preconditioning approach where two metrics are used in the gradient step and the proximity step. We provide convergence conditions for this new iterative scheme and characterize its limit point. Simulations for tomographic image reconstruction from undersampled measurements show the benefits of our approach for various simple choices of metrics.

Accelerated population ageing experienced in the last few decades is an unprecedented phenomenon. Currently, this is more in the developing countries. Soon three-fourths of the elderly will be in the developing world. From 1990 to 2025, the elderly population in Asia will rise from 50 per cent of the world's elderly to 58 per cent, in Africa and Latin America from 5 to 7 per cent, but in Europe the figure will drop from 19 to 12 per cent of the world's elderly. The life span has increased in India from 32 yr in 1947 to more than 62 yr now. From the morbidity point of view, almost 50 per cent of the Indian elderly have chronic diseases and 5 per cent suffer from immobility. There are several vulnerable groups and a big disadvantaged lot are elderly females who are one of the fastest growing segments, which will increase to become 4 times the current figure, by 2025. In spite of professional disinterest in the speciality, recent trends indicate the beginning of sensitization of medical teachers, advancing speciality of psychosocial gerontology and availability of some research funds. Importance of training of health professionals and priorities in gerontological research are also under consideration. Infections still take a heavy toll of our elderly population apart from well known degenerative disorders. Limitations of a developing country further influence the morbidity pattern in various ways. Nutritional deficiencies are common and often subclinical thus escaping the desired interventions. Coronary heart disease, hypertension, mental and many other disorders in the elderly have been reported as isolated observations highlighting differences from those made in the Western countries. Socio-economically, the traditional support of extended families is rapidly undergoing erosion making the elderly further vulnerable. This causes more emotional and psychological problems while the State finds itself helpless in providing a comprehensive care to its large chunk of elderly population. It will be important to surmise and predetermine the future factors that are going to modify the diverse patterns of morbidity, disability and mortality in regional context.

The objective of this letter is to propose a novel computational method to learn the state of an appliance (ON / OFF) given the aggregate power consumption recorded by the smart-meter. We formulate a multi-label classification problem where the classes correspond to the appliances. The proposed approach is based on our recently introduced framework of convolutional transform learning. We propose a deep supervised version of it relying on an original multi-label cost. Comparisons with state-of-the-art techniques show that our proposed method improves over the benchmarks on popular non-intrusive load monitoring datasets.

Based on its great successes in inference and denosing tasks, Dictionary Learning (DL) and its related sparse optimization formulations have garnered a lot of research interest. While most solutions have focused on single layer dictionaries, the recently improved Deep DL methods have also fallen short on a number of issues. We hence propose a novel Deep DL approach where each DL layer can be formulated and solved as a combination of one linear layer and a Recurrent Neural Network, where the RNN is flexibly regraded as a layer-associated learned metric. Our proposed work unveils new insights between the Neural Networks and Deep DL, and provides a novel, efficient and competitive approach to jointly learn the deep transforms and metrics. Extensive experiments are carried out to demonstrate that the proposed method can not only outperform existing Deep DL, but also state-of-the-art generic Convolutional Neural Networks.

Curbing hate speech is undoubtedly a major challenge for online microblogging platforms like Twitter. While there have been studies around hate speech detection, it is not clear how hate speech finds its way into an online discussion. It is important for a content moderator to not only identify which tweet is hateful but also to predict which tweet will be responsible for accumulating hate speech. This would help in prioritizing tweets that need constant monitoring. Our analysis reveals that for hate speech to manifest in an ongoing discussion, the source tweet may not necessarily be hateful; rather, there are plenty of such non-hateful tweets which gradually invoke hateful replies, resulting in the entire reply threads becoming provocative.

Subspace clustering assumes that the data is separable into separate subspaces. Such a simple assumption, does not always hold. We assume that, even if the raw data is not separable into subspaces, one can learn a representation (transform coefficients) such that the learnt representation is separable into subspaces. To achieve the intended goal, we embed subspace clustering techniques (locally linear manifold clustering, sparse subspace clustering and low rank representation) into transform learning. The entire formulation is jointly learnt; giving rise to a new class of methods called transformed subspace clustering (TSC). In order to account for non-linearity, kernelized extensions of TSC are also proposed. To test the performance of the proposed techniques, benchmarking is performed on image clustering and document clustering datasets. Comparison with state-of-the-art clustering techniques shows that our formulation improves upon them.

Convolutional transform learning is an unsupervised framework we introduced recently, for feature generation based on learnt convolutions. In this work, we propose a supervised formulation for convolutional transform so as to address the multi-label classification problem. Unlike the simple multiclass classification, in multi-label problems, each sample can correspond to multiple classes simultaneously, making the problem quite challenging. We propose to make use of a label consistency penalty and develop a suitable minimization algorithm for the training step. We illustrate the performance of the developed formulation on the practical problem of nonintrusive load monitoring. Comparisons with popular techniques show that our proposed approach yields better results on benchmark datasets.

Modeling and inference with multivariate sequences is central in a number of signal processing applications such as acoustics, social network analysis, biomedical, and finance, to name a few. The linear-Gaussian state-space model is a common way to describe a time series through the evolution of a hidden state, with the advantage of presenting a simple inference procedure due to the celebrated Kalman filter. A fundamental question when analyzing a multivariate sequence is the search for relationships between its entries (or the entries of the modeled hidden state), especially when the inherent structure is a non-fully connected graph. In such context, graphical modeling combined with parsimony constraints allows to limit the proliferation of parameters and enables a compact data representation which is easier to interpret by the experts. In this work, we propose a novel expectation-maximization algorithm for estimating the linear matrix operator in the state equation of a linear-Gaussian state-space model. Lasso regularization is included in the M-step, that we solve using a proximal splitting Douglas-Rachford algorithm. Numerical experiments illustrate the benefits of the proposed model and inference technique, named GraphEM, over competitors relying on Granger causality.

Modern 3D image recovery problems require powerful optimization frameworks to handle high dimensionality while providing reliable numerical solutions in a reasonable time. In this perspective, asynchronous parallel optimization algorithms have received an increasing attention by overcoming memory limitation issues and communication bottlenecks. In this work, we propose a block distributed Majorize-Minorize Memory Gradient (BD3MG) optimization algorithm for solving large scale non-convex differentiable optimization problems. Assuming a distributed memory environment, the algorithm casts the efficient 3MG scheme into smaller dimension subproblems where blocks of variables are addressed in an asynchronous manner. Convergence of the sequence built by the proposed BD3MG method is established under mild assumptions. Application to the restoration of 3D images degraded by a depth-variant blur shows that our method yields significant computational time reduction compared to several synchronous and asynchronous competitors, while exhibiting great scalability potential.

Population Monte Carlo (PMC) algorithms are a family of adaptive importance sampling (AIS) methods for approximating integrals in Bayesian inference. In this paper, we propose a novel PMC algorithm that combines recent advances in the AIS and the optimization literatures. In such a way, the proposal densities are adapted according to the past weighted samples via a local resampling that preserves the diversity, but we also exploit the geometry of the targeted distribution. A scaled Langevin strategy with Newton-based scaling metric is retained for this purpose, allowing to adapt jointly the means and the covariances of the proposals, without needing to tune any extra parameter. The performance of the proposed technique is clearly superior in two numerical examples at the cost of a reasonable computational complexity increment.

In drug target interaction (DTI) the interactions of some (a subset) drugs on some (a subset) targets are known. The goal is to predict the interactions of all drugs on all targets. One approach is to formulate this as a matrix completion problem, where the matrix of interactions having drugs along the rows and targets along the columns is partially filled. So far standard matrix completion approaches such as nuclear norm minimization and matrix factorization have been used to address the problem. In this work, we propose a deep matrix factorization approach to improve the prediction results. Experiments have been performed on benchmark databases and comparison carried out with some state-of-the-art algorithms. Empirically our proposed deep method, outperforms all the techniques compared against.

Image restoration is frequently addressed by resorting to variational methods which account for some prior knowledge about the solution. The success of these methods, however, heavily depends on the estimation of a set of hyperparameters. Deep learning architectures are, on the contrary, very generic and efficient, but they offer limited control over their output. In this paper, we present iRestNet, a neural network architecture which combines the benefits of both approaches. iRestNet is obtained by unfolding a proximal interior point algorithm. This enables enforcing hard constraints on the pixel range of the restored image thanks to a logarithmic barrier strategy, without requiring any parameter setting. Explicit expressions for the involved proximity operator, and its differential, are derived, which allows training iRestNet with gradient descent and backpropagation. Numerical experiments on image deblurring show that the proposed approach provides good image quality results compared to state-of-the-art variational and machine learning methods.

In this letter, we propose a probabilistic optimization method, named probabilistic incremental proximal gradient (PIPG) method, by developing a probabilistic interpretation of the incremental proximal gradient algorithm. We explicitly model the update rules of the incremental proximal gradient method and develop a systematic approach to propagate the uncertainty of the solution estimate over iterations. The PIPG algorithm takes the form of Bayesian filtering updates for a state-space model constructed by using the cost function. Our framework makes it possible to utilize well-known exact or approximate Bayesian filters, such as Kalman or extended Kalman filters, to solve large-scale regularized optimization problems.

Joint deconvolution and segmentation of ultrasound images is a challenging problem in medical imaging. By adopting a hierarchical Bayesian model, we propose an accelerated Markov chain Monte Carlo scheme where the tissue reflectivity function is sampled thanks to a recently introduced proximal unadjusted Langevin algorithm. This new approach is combined with a forward-backward step and a preconditioning strategy to accelerate the convergence, and with a method based on the majorization-minimization principle to solve the inner nonconvex minimization problems. As demonstrated in numerical experiments conducted on both simulated and in vivo ultrasound images, the proposed method provides high-quality restoration and segmentation results and is up to six times faster than an existing Hamiltonian Monte Carlo method.

In recent years, there has been a growing interest in problems in graph estimation and model selection, which all share very similar matrix variational formulations, the most popular one being probably GLASSO. Unfortunately, the standard GLASSO formulation does not take into account noise corrupting the data: this shortcoming leads us to propose a novel criterion, where the regularization function is decoupled in two terms, one acting only on the eigenvalues of the matrix and the other on the matrix elements. Incorporating noise information into the model has the side-effect to make the cost function non-convex. To overcome this difficulty, we adopt a majorization-minimization approach, where at each iteration a convex approximation of the original cost function is minimized via the Douglas-Rachford procedure. The achieved results are very promising w.r.t. classical approaches.

Mass spectrometry (MS) is a fundamental technology of analytical chemistry for measuring the structure of molecules, with many application fields such as clinical biomarker analysis or pharmacokinetics. In the context of proteomic analysis with MS, the superposition of the isotopic patterns of different proteins, in various charge-states produces MS spectra difficult to decipher. The complexity of the pattern models and the large size of the data again increase the difficulty of the analysis step. In this paper, we propose to formulate the problem of proteins characterization as the estimation of a positive-valued sparse signal thanks to a dictionary-based approach relying on the protein averagine concept. A proximal primal-dual splitting convex optimization method is considered to solve the resulting variational problem. Moreover, the large size of the dictionary matrix is circumvented by proposing a suitable block circulant approximation of it, allowing to limit the computational burden of the method. Numerical experiments on synthetic and real MS datasets illustrate the good performance of our approach.

Interior point methods have been known for decades to be useful for the resolution of small to medium size constrained optimization problems. These approaches have the benefit of ensuring feasibility of the iterates through a logarithmic barrier. We propose to incorporate a proximal forward-backward step in the resolution of the barrier subproblem to account for non-necessarily differentiable terms arising in the objective function. The combination of this scheme with a novel line-search strategy gives rise to the so-called Proximal Interior Point Algorithm (PIPA) suitable for the minimization of the sum of a smooth convex function and a non-smooth convex one under general convex constraints. The convergence of PIPA is secured under mild assumptions. As demonstrated by numerical experiments carried out on a large-scale hyperspectral image unmixing application, the proposed method outperforms the state-of-the-art.

In this paper, we present a new distributed algorithm for minimizing a sum of non-necessarily differentiable convex functions composed with arbitrary linear operators. The overall cost function is assumed strongly convex. Each involved function is associated with a node of a hypergraph having the ability to communicate with neighboring nodes sharing the same hyperedge. Our algorithm relies on a primal-dual splitting strategy with established convergence guarantees. We show how it can be efficiently implemented to take full advantage of a multicore architecture. The good numerical performance of the proposed approach is illustrated in a problem of video sequence denoising, where a significant speedup is achieved.

In the field of 3D image recovery, huge amounts of data need to be processed. Parallel optimization methods are then of main interest since they allow to overcome memory limitation issues, while benefiting from the intrinsic acceleration provided by recent multicore computing architectures. In this context, we propose a Block Parallel Majorize-Minimize Memory Gradient (BP3MG) algorithm for solving large scale optimization problems. This algorithm combines a block coordinate strategy with an efficient parallel update. The proposed method is applied to a 3D microscopy image restoration problem involving a depth-variant blur, where it is shown to lead to significant computational time savings with respect to a sequential approach.

State-of-the-art methods for solving smooth optimization problems are nonlinear conjugate gradient, low memory BFGS, and majorize-minimize (MM) subspace algorithms. The MM subspace algorithm that has been introduced more recently has shown good practical performance when compared with other methods on various optimization problems arising in signal and image processing. However, to the best of our knowledge, no general result exists concerning the theoretical convergence rate of the MM subspace algorithm. This paper aims at deriving such convergence rates both for batch and online versions of the and in particular, discusses the influence of the choice of the subspace.

Primal-dual proximal optimization methods have recently gained much interest for dealing with very large-scale data sets encoutered in many application fields such as machine learning, computer vision and inverse problems [1-3]. In this work, we propose a novel random block-coordinate version of such algorithms allowing us to solve a wide array of convex variational problems. One of the main advantages of the proposed algorithm is its ability to solve composite problems involving large-size matrices without requiring any inversion. In addition, the almost sure convergence to an optimal solution to the problem is guaranteed. We illustrate the good performance of our method on a mesh denoising application.

Inverse problems encountered in video processing often require to minimize criteria involving a high number of variables. Among available optimization techniques, proximal methods have shown their efficiency in solving large-scale possibly nonsmooth problems. When some of the proximity operators involved in these methods do not have closed form expressions, they may constitute a bottleneck in terms of computational complexity and memory requirements. In this paper, we address this problem and propose accelerated techniques for solving it. A new dual block-coordinate forward-backward algorithm computing the proximity operator of a sum of convex functions composed with linear operators is proposed and theoretically analyzed. The numerical performance of the approach is assessed through an application to deconvolution and super-resolution of interlaced video sequences.

The Poisson--Gaussian model can accurately describe the noise present in a number of imaging systems. However most existing restoration methods rely on approximations of the Poisson--Gaussian noise statistics. We propose a convex optimization strategy for the reconstruction of images degraded by a linear operator and corrupted with a mixed Poisson--Gaussian noise. The originality of our approach consists of considering the exact, mixed continuous-discrete model corresponding to the data statistics. After establishing the Lipschitz differentiability and convexity of the Poisson--Gaussian neg-log-likelihood, we derive a primal-dual iterative scheme for minimizing the associated penalized criterion. The proposed method is applicable to a large choice of convex penalty terms. The robustness of our scheme allows us to handle computational difficulties due to infinite sums arising from the computation of the gradient of the criterion. We propose finite bounds for these sums, that are dependent on the current image estimate, and thus adapted to each iteration of our algorithm. The proposed approach is validated on image restoration examples. Then, the exact data fidelity term is used as a reference for studying some of its various approximations. We show that in a variational framework the shifted Poisson and exponential approximations lead to very good restoration results.

The ℓ1/ℓ2ratio regularization function has shown good performance for retrieving sparse signals in a number of recent works, in the context of blind deconvolution. Indeed, it benefits from a scale invariance property much desirable in the blind context. However, the ℓ1/ℓ2function raises some difficulties when solving the nonconvex and nonsmooth minimization problems resulting from the use of such a penalty term in current restoration methods. In this paper, we propose a new penalty based on a smooth approximation to the ℓ1/ℓ2function. In addition, we develop a proximal-based algorithm to solve variational problems involving this function and we derive theoretical convergence results. We demonstrate the effectiveness of our method through a comparison with a recent alternating optimization strategy dealing with the exact ℓ1/ℓ2term, on an application to seismic data blind deconvolution.

Many inverse problems require to minimize a criterion being the sum of a non necessarily smooth function and a Lipschitz differentiable function. Such an optimization problem can be solved with the Forward-Backward algorithm which can be accelerated thanks to the use of variable metrics derived from the Majorize-Minimize principle. The convergence of this approach is guaranteed provided that the criterion satisfies some additional technical conditions. Combining this method with an alternating minimization strategy will be shown to allow us to address a broad class of optimization problems involving large-size signals. An application example to a nonconvex spectral unmixing problem will be presented.

Primal-dual interior-point methods are used in image processing to solve inversion problems that can be reduced to constrained convex minimization. Such iterative methods require the solution of a sequence of linear systems which are used to derive descent directions. This approach is very consuming in terms of computing time and memory usage for large-scale problems, unless the involved matrices have a specific structure. This is the case in the spectral unmixing problem where these matrices are block-diagonal when no spatial regularization is considered. Here, we consider the edge-preserving regularized case and we propose to tackle the linear system solving using a majorization-minimization (MM) approach based on separable quadratic majorant functions. The resulting systems have the same structure as in the non-regularized case and can thereby be solved efficiently. The interior-point algorithm is speeded-up while remaining convergent. An example of spectral unmixing is proposed to illustrate the efficiency of this approach.

With the development of new imaging systems delivering large-size data sets, phase retrieval has become recently the focus of much attention. The problem is especially challenging due to its intrinsically nonconvex formulation. In addition, the applicability of many existing solutions may be limited either by their estimation performance or by their computational cost, especially in the case of non-Fourier measurements. In this paper, we propose a novel phase retrieval approach, which is based on a smooth nonconvex approximation of the standard data fidelity term. In addition, the proposed method allows us to employ a wide range of convex separable regularization functions. The optimization process is performed by a block coordinate proximal algorithm which is amenable to solving large-scale problems. An application of this algorithm to an image reconstruction problem shows that it may be very competitive with respect to state-of-the-art methods.

This paper deals with the reconstruction of relaxation time distributions in Nuclear Magnetic Resonance (NMR) spectroscopy. This large scale and ill-posed inverse problem is solved by the iterative minimization of a regularized objective function allowing to encode some prior assumptions on the sought distribution. The numerical optimization of the criterion is performed using a primal-dual interior point algorithm allowing to handle the non-negativity constraint. The performances of the proposed approach are illustrated through the processing of real data from a two-dimensional NMR experiment.

Tomography is an image reconstruction task that may be viewed as a linear inverse problem akin to deconvolution. Recent progresses in optimization methods have made it possible to formulate this task so that fewer projections and higher amounts of noise can be dealt with, making use of a-priori information and domain constraints. In this article, we investigate 3MG, a new optimization method that is highly flexible and effective. In particular, we propose and compare convex and non-convex regularization potentials on both synthetic and real images. We further investigate the possibility to deal with continuous angular integration, i.e. where projections rays are no longer straight lines, but cones. This is encountered in a variety of real-life situations, but is difficult or impossible to deal with exactly using traditional reconstruction algorithms. We show that in this situation it may be beneficial to acquire fewer projections than would be required using classical methods.

In multichannel imaging, several observations of the same scene acquired in different spectral ranges are available. Very often, the spectral components are degraded by a blur modelled by a linear operator and an additive noise. In this paper, we address the problem of recovering the image components in a wavelet domain by adopting a variational approach. Our contribution is twofold. First, an appropriate multivariate penalty function is derived from a novel joint prior model of the probability distribution of the wavelet coefficients located at the same spatial position in a given subband through all the channels. Secondly, we address the challenging issue of computing the Maximum A Posteriori estimate by using a Majorize-Minimize optimization strategy. Simulation tests carried out on multispectral satellite images show that the proposed method outperforms conventional techniques.

In this work, we consider a class of differentiable criteria for sparse image computing problems, where a nonconvex regularization is applied to an arbitrary linear transform of the target image. As special cases, it includes edge-preserving measures or frame-analysis potentials commonly used in image processing. As shown by our asymptotic results, the $\ell_2-\ell_0$ penalties we consider may be employed to provide approximate solutions to $\ell_0$-penalized optimization problems. One of the advantages of the proposed approach is that it allows us to derive an efficient majorize-minimize subspace algorithm. The convergence of the algorithm is investigated by using recent results in nonconvex optimization. The fast convergence properties of the proposed optimization method are illustrated through image processing examples. In particular, its effectiveness is demonstrated on several data recovery problems.

A Poisson-Gaussian model accurately describes the noise present in many imaging systems such as CCD cameras or fluorescence microscopy. However most existing restoration strategies rely on approximations of the Poisson-Gaussian noise statistics. We propose a convex optimization algorithm for the reconstruction of signals degraded by a linear operator and corrupted with mixed Poisson-Gaussian noise. The originality of our approach consists of considering the exact continuous-discrete model corresponding to the data statistics. After establishing the Lipschitz differentiability of the Poisson-Gaussian log-likelihood, we derive a primal-dual iterative scheme for minimizing the associated penalized criterion. The proposed method is applicable to a large choice of penalty terms. The robustness of our scheme allows us to handle computational difficulties due to infinite sums arising from the computation of the gradient of the criterion. The proposed approach is validated on image restoration examples.

In this paper, we consider a class of differentiable criteria for sparse image recovery problems. The regularization is applied to a linear transform of the target image. As special cases, it includes edge preserving measures or frame analysis potentials. As shown by our asymptotic results, the considered ℓ2- ℓ0penalties may be employed to approximate solutions to ℓ0penalized optimization problems. One of the advantages of the approach is that it allows us to derive an efficient Majorize-Minimize Memory Gradient algorithm. The fast convergence properties of the proposed optimization algorithm are illustrated through image restoration examples.

This paper proposes accelerated subspace optimization methods in the context of image restoration. Subspace optimization methods belong to the class of iterative descent algorithms for unconstrained optimization. At each iteration of such methods, a stepsize vector allowing the best combination of several search directions is computed through a multidimensional search. It is usually obtained by an inner iterative second-order method ruled by a stopping criterion that guarantees the convergence of the outer algorithm. As an alternative, we propose an original multidimensional search strategy based on the majorize-minimize principle. It leads to a closed-form stepsize formula that ensures the convergence of the subspace algorithm whatever the number of inner iterations. The practical efficiency of the proposed scheme is illustrated in the context of edge-preserving image restoration.

This paper deals with the reconstruction of T1-T2 correlation spectra in Nuclear Magnetic Resonance (NMR) spectroscopy. The ill-posed character of this inverse problem and its large size are the main difficulties of the reconstruction. While maximum entropy is retained as an adequate regularization approach, the choice of an efficient optimization algorithm remains a challenging task. Our proposal is to apply a nonlinear conjugate gradient algorithm with two original features. Firstly, a theoretically well stated line search strategy suitable for the entropy function is applied to ensure a monotonic decrease of the criterion. Secondly, an appropriate preconditioning structure based on a truncated singular value decomposition of the forward model matrix is used to speed up the algorithm convergence. The resulting method reveals far more efficient than the classical Skilling and Bryan method and its applicability is illustrated through real NMR data processing.