Abstract. Stochastic gradient descent (SGD) and its variants are widely used and highly effective optimization methods in machine learning, especially for neural network training. By using a single datum or a small subset of the data, selected randomly at each iteration, SGD scales well to problem size and has been shown to be effective for solving large-scale inverse problems. In this work, we investigate SGD for solving nonlinear inverse problems in Banach spaces through the lens of iterative regularization. Under general assumptions, we prove almost sure convergence of the iterates to the minimum-distance solution and show the regularizing property in expectation under an a priori stopping rule. Further, we establish convergence rates under the conditional stability assumptions for both exact and noisy data. Numerical experiments on Schlieren tomography and electrical impedance tomography are presented to show distinct features of the method.

Abstract. In this work, we propose an innovative iterative direct sampling method to solve nonlinear elliptic inverse problems from a limited number of pairs of Cauchy data. It extends the original direct sampling method (DSM) by incorporating an iterative mechanism, enhancing its performance with a modest increase in computational effort but a clear improvement in its stability against data noise. The method is formulated in an abstract framework of operator equations and is applicable to a broad range of elliptic inverse problems. Numerical results on electrical impedance tomography, optical tomography, cardiac electrophysiology, etc., demonstrate its effectiveness and robustness, especially with an improved accuracy for identifying the locations and geometric shapes of inhomogeneities in the presence of large noise, when compared with the standard DSM.

Denoising diffusion models have emerged as the go-to generative framework for solving inverse problems in imaging. A critical concern regarding these models is their performance on out-of-distribution tasks, which remains an under-explored challenge. Using a diffusion model on an out-of-distribution dataset, realistic reconstructions can be generated, but with hallucinating image features that are uniquely present in the training dataset. To address this discrepancy and improve reconstruction accuracy, we introduce a novel test-time adaptation sampling framework called Steerable Conditional Diffusion. Specifically, this framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. Utilising the proposed method, we achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities, advancing the robust deployment of denoising diffusion models in real-world applications.

Invertible neural networks (INNs) represent an important class of deep neural network architectures that have been widely used in applications. The universal approximation properties of INNs have been established recently. However, the approximation rate of INNs is largely missing. In this work, we provide an analysis of the capacity of a class of coupling-based INNs to approximate bi-Lipschitz continuous mappings on a compact domain, and the result shows that it can well approximate both forward and inverse maps simultaneously. Furthermore, we develop an approach for approximating bi-Lipschitz maps on infinite-dimensional spaces that simultaneously approximate the forward and inverse maps, by combining model reduction with principal component analysis and INNs for approximating the reduced map, and we analyze the overall approximation error of the approach. Preliminary numerical results show the feasibility of the approach for approximating the solution operator for parameterized second-order elliptic problems.

Abstract. In recent years, new regularization methods based on (deep) neural networks have shown very promising empirical performance for the numerical solution of ill-posed problems, e.g., in medical imaging and imaging science. Due to the nonlinearity of neural networks, these methods often lack satisfactory theoretical justification. In this work, we rigorously discuss the convergence of a successful unsupervised approach that utilizes untrained convolutional neural networks to represent solutions to linear ill-posed problems. Untrained neural networks are particularly appealing for many applications because they do not require paired training data. The regularization property of the approach relies solely on the architecture of the neural network instead. Due to the vast overparameterization of the employed neural network, suitable early stopping is essential for the success of the method. We establish that the classical discrepancy principle is an adequate method for early stopping of two-layer untrained convolutional neural networks learned by gradient descent, and furthermore, it yields an approximation with minimax optimal convergence rates. Numerical results are also presented to illustrate the theoretical findings.

Abstract. In this work, we develop a novel approach using deep neural networks (DNNs) to reconstruct the conductivity distribution in elliptic problems from one measurement of the solution over the whole domain. The approach is based on a mixed reformulation of the governing equation and utilizes the standard least-squares objective, with DNNs as ansatz functions to approximate the conductivity and flux simultaneously. We provide a thorough analysis of the DNN approximations of the conductivity for both continuous and empirical losses, including rigorous error estimates that are explicit in terms of the noise level, various penalty parameters, and neural network architectural parameters (depth, width, and parameter bounds). We also provide multiple numerical experiments in two dimensions and multidimensions to illustrate distinct features of the approach, e.g., excellent stability with respect to data noise and capability of solving high-dimensional problems.

Abstract. In this work we consider stochastic gradient descent (SGD) for solving linear inverse problems in Banach spaces. SGD and its variants have been established as one of the most successful optimization methods in machine learning, imaging, and signal processing, to name a few. At each iteration SGD uses a single datum, or a small subset of data, resulting in highly scalable methods that are very attractive for large-scale inverse problems. Nonetheless, the theoretical analysis of SGD-based approaches for inverse problems has thus far been largely limited to Euclidean and Hilbert spaces. In this work we present a novel convergence analysis of SGD for linear inverse problems in general Banach spaces: we show the almost sure convergence of the iterates to the minimum norm solution and establish the regularizing property for suitable a priori stopping criteria. Numerical results are also presented to illustrate features of the approach.

Penalised PET image reconstruction algorithms are often accelerated during early iterations with the use of subsets. However, these methods may exhibit limit cycle behaviour at later iterations due to variations between subsets. Desirable converged images can be achieved for a subclass of these algorithms via the implementation of a relaxed step size sequence, but the heuristic selection of parameters will impact the quality of the image sequence and algorithm convergence rates. In this work, we demonstrate the adaption and application of a class of stochastic variance reduction gradient algorithms for PET image reconstruction using the relative difference penalty and numerically compare convergence performance to BSREM. The two investigated algorithms are: SAGA and SVRG. These algorithms require the retention in memory of recently computed subset gradients, which are utilised in subsequent updates. We present several numerical studies based on Monte Carlo simulated data and a patient data set for fully 3D PET acquisitions. The impact of the number of subsets, different preconditioners and step size methods on the convergence of regions of interest values within the reconstructed images is explored. We observe that when using constant preconditioning, SAGA and SVRG demonstrate reduced variations in voxel values between subsequent updates and are less reliant on step size hyper-parameter selection than BSREM reconstructions. Furthermore, SAGA and SVRG can converge significantly faster to the penalised maximum likelihood solution than BSREM, particularly in low count data.

In this paper, we propose and analyze a reconstruction algorithm for imaging an anisotropic conductivity tensor in a second-order elliptic PDE with a nonzero Dirichlet boundary condition from internal current densities. It is based on a regularized output least-squares formulation with the standard $L^2(\Omega)^{d,d}$ penalty, which is then discretized by the standard Galerkin finite element method. We establish the continuity and differentiability of the forward map with respect to the conductivity tensor in the $L^p(\Omega)^{d,d}$-norms, the existence of minimizers and optimality systems of the regularized formulation using the concept of H-convergence. Further, we provide a detailed analysis of the discretized problem, especially the convergence of the discrete approximations with respect to the mesh size, using the discrete counterpart of H-convergence. In addition, we develop a projected Newton algorithm for solving the first-order optimality system. We present extensive two-dimensional numerical examples to show the efficiency of the proposed method.

Recent advances in reconstruction methods for inverse problems leverage powerful data-driven models, e.g., deep neural networks. These techniques have demonstrated state-of-the-art performances for several imaging tasks, but they often do not provide uncertainty on the obtained reconstruction. In this work, we develop a scalable, data-driven, knowledge-aided computational framework to quantify the model uncertainty via Bayesian neural networks. The approach builds on, and extends deep gradient descent, a recently developed greedy iterative training scheme, and recasts it within a probabilistic framework. Scalability is achieved by being hybrid in the architecture: only the last layer of each block is Bayesian, while the others remain deterministic, and by being greedy in training. The framework is showcased on one representative medical imaging modality, viz. computed tomography with either sparse view or limited view data, and exhibits competitive performance with respect to state-of-the-art benchmarks, e.g., total variation, deep gradient descent and learned primal-dual.

In this note, we analyze an iterative soft/hard thresholding algorithm with homotopy continuation for recovering a sparse signal x†from noisy data of a noise level ε. Under suitable regularity and sparsity conditions, we design a path, along which the algorithm can find a solution x*, which admits a sharp reconstruction error ||x* - x†|| ℓ∞ = O(ε) with an iteration complexity O((ln ε)/(ln γ)np), where n and p are problem dimensionality and γ ε (0,1) controls the length of the path. Numerical examples are given to illustrate its performance.

This paper provides an analysis of the linearized inverse problem in multifrequency electrical impedance tomography. We consider an isotropic conductivity distribution with a finite number of unknown inclusions with different frequency dependence, as is often seen in biological tissues. We discuss reconstruction methods for both fully known and partially known spectral profiles and demonstrate in the latter case the successful employment of difference imaging. We also study the reconstruction with an imperfectly known boundary and show that the multifrequency approach can eliminate modeling errors and recover almost all inclusions. In addition, we develop an efficient group sparse recovery algorithm for the robust solution of related linear inverse problems. Several numerical simulations are presented to illustrate and validate the approach.

This work is concerned with nonlinear parameter identification in partial differential equations subject to impulsive noise. To cope with the non-Gaussian nature of the noise, we consider a model with $\mbox{L}^1$ fitting. However, the nonsmoothness of the problem makes its efficient numerical solution challenging. By approximating this problem using a family of smoothed functionals, a semismooth Newton method becomes applicable. In particular, its superlinear convergence is proved under a second-order condition. The convergence of the solution to the approximating problem as the smoothing parameter goes to zero is shown. A strategy for adaptively selecting the regularization parameter based on a balancing principle is suggested. The efficiency of the method is illustrated on several benchmark inverse problems of recovering coefficients in elliptic differential equations, for which one- and two-dimensional numerical examples are presented.

This paper considers the numerical solution of inverse problems with an $\mathrm{L}^1$ data fitting term, which is challenging due to the lack of differentiability of the objective functional. Utilizing convex duality, the problem is reformulated as minimizing a smooth functional with pointwise constraints, which can be efficiently solved using a semismooth Newton method. In order to achieve superlinear convergence, the dual problem requires additional regularization. For both the primal and the dual problems, the choice of the regularization parameters is crucial. We propose adaptive strategies for choosing these parameters. The regularization parameter in the primal formulation is chosen according to a balancing principle derived from the model function approach, whereas the one in the dual formulation is determined by a path-following strategy based on the structure of the optimality conditions. Several numerical experiments confirm the efficiency and robustness of the proposed method and adaptive strategy.