Abstract. The Transformer architecture has revolutionized the field of sequence modeling and underpins the recent breakthroughs in large language models (LLMs). However, a comprehensive mathematical theory that explains its structure and operations remains elusive. In this work, we propose a novel continuous framework that rigorously interprets the Transformer as a discretization of a structured integro-differential equation. Within this formulation, the self-attention mechanism emerges naturally as a nonlocal integral operator, and layer normalization is characterized as a projection to a time-dependent constraint. This operator-theoretic and variational perspective offers a unified and interpretable foundation for understanding the architecture’s core components, including attention, feedforward layers, and normalization. Our approach extends beyond previous theoretical analyses by embedding the entire Transformer operation in continuous domains for both token indices and feature dimensions. This leads to a principled and flexible framework that not only deepens on theoretical insight but also offers new directions for architecture design, analysis, and control-based interpretations. This new interpretation provides a step toward bridging the gap between deep learning architectures and continuous mathematical modeling, and contributes a foundational perspective to the ongoing development of interpretable and theoretically grounded neural network models.

Understanding the spatial distribution of palms in tropical forests is essential for ecological monitoring, conservation strategies, and the sustainable integration of natural forest products into local and global supply chains. However, the analysis of remotely sensed data are challenged by overlapping palm and tree crowns, uneven shading across the canopy surface, and the heterogeneous nature of the forest landscapes, which often affect the performance of palm detection and segmentation algorithms. To overcome these issues, we introduce PalmDSNet, a deep learning framework for efficient detection, segmentation, and counting of canopy palms. To model spatial patterns, we introduce a bimodal reproduction algorithm that simulates palm propagation based on PalmDSNet outputs. We used UAV-captured imagery to create orthomosaics from 21 sites across western Ecuadorian tropical forests, covering a gradient from the everwet Chocó forests near Colombia to the drier forests of southwestern Ecuador. Expert annotations were used to create a comprehensive dataset, including 7,356 bounding boxes on image patches and 7,603 palm centers across five orthomosaics, encompassing a total area of 449 hectares. By integrating detection and spatial modeling, we effectively simulate the spatial distribution of palms in diverse and dense tropical environments, validating its utility for advanced applications in tropical forest monitoring and remote sensing analysis. The dataset can be accessed at 10.5281/zenodo.13822508, and the code to replicate the study is available at github.com/ckn3/palm-ds-sp.

The vision and language generative models have been overgrown in recent years. For video generation, various open-sourced models and public-available services have been developed to generate high-quality videos. However, these methods often use a few metrics, e.g., FVD [56] or IS [45], to evaluate the performance. We argue that it is hard to judge the large conditional generative models from the simple metrics since these models are often trained on very large datasets with multi-aspect abilities. Thus, we propose a novel framework and pipeline for exhaustively evaluating the performance of the generated videos. Our approach involves generating a diverse and comprehensive list of 700 prompts for text-to-video generation, which is based on an analysis of real-world user data and generated with the assistance of a large language model. Then, we evaluate the state-of-the-art video generative models on our carefully designed benchmark, in terms of visual qualities, content qualities, motion qualities, and text-video alignment with 17 well-selected objective metrics. To obtain the finalleaderboard of the models, we further fit a series of coefficients to align the objective metrics to the users' opinions. Based on the proposed human alignment method, our final score shows a higher correlation than simply averaging the metrics, showing the effectiveness of the proposed evaluation method.

Abstract. Each coil image in a parallel magnetic resonance imaging (pMRI) system is an imaging slice modulated by the corresponding coil sensitivity. These coil images, structurally similar to each other, are stacked together as 3-dimensional (3D) image data, and their sparsity property can be explored via 3D directional Haar tight framelets. The features of the 3D image data from the 3D framelet systems are utilized to regularize sensitivity encoding (SENSE) pMRI reconstruction. Accordingly, a so-called SENSE3d algorithm is proposed to reconstruct images of high quality from the sampled [Formula: see text]-space data with a high acceleration rate by decoupling effects of the desired image (slice) and sensitivity maps. Since both the imaging slice and sensitivity maps are unknown, this algorithm repeatedly performs a slice step followed by a sensitivity step by using updated estimations of the desired image and the sensitivity maps. In the slice step, for the given sensitivity maps, the estimation of the desired image is viewed as the solution to a convex optimization problem regularized by the sparsity of its 3D framelet coefficients of coil images. This optimization problem, involving data from the complex field, is solved by a primal-dual three-operator splitting (PD3O) method. In the sensitivity step, the estimation of sensitivity maps is modeled as the solution to a Tikhonov-type optimization problem that favors the smoothness of the sensitivity maps. This corresponding problem is nonconvex and could be solved by a forward-backward splitting method. Experiments on real phantoms and in vivo data show that the proposed SENSE3d algorithm can explore the sparsity property of the imaging slices and efficiently produce reconstructed images of high quality with reduced aliasing artifacts caused by high acceleration rate, additive noise, and the inaccurate estimation of each coil sensitivity. To provide a comprehensive picture of the overall performance of our SENSE3d model, we provide the quantitative index (HaarPSI) and comparisons to some deep learning methods such as VarNet and fastMRI-UNet.

Abstract. For problems in image processing and many other fields, a large class of effective neural networks has encoder-decoder-based architectures. Although these networks have shown impressive performance, mathematical explanations of their architectures are still underdeveloped. In this paper, we study the encoder-decoder-based network architecture from the algorithmic perspective and provide a mathematical explanation. We use the two-phase Potts model for image segmentation as an example for our explanations. We associate the segmentation problem with a control problem in the continuous setting. Then, the continuous control model is time discretized by an operator-splitting scheme, the PottsMGNet, and space discretized by the multigrid method. We show that the resulting discrete PottsMGNet is equivalent to an encoder-decoder-based network. With minor modifications, it is shown that a number of the popular encoder-decoder-based neural networks are just instances of the proposed PottsMGNet. By incorporating the soft-threshold-dynamics into the PottsMGNet as a regularizer, the PottsMGNet has shown to be robust with the network parameters such as network width and depth and has achieved remarkable performance on datasets with very large noise. In nearly all our experiments, the new network always performs better than or as well as on accuracy and dice score compared to existing networks for image segmentation.

Abstract. The Mumford–Shah model is a classical segmentation model, but its objective function is nonconvex. The smoothing and thresholding (SaT) approach is a convex variant of the Mumford–Shah model, which seeks a smoothed approximation solution to the Mumford–Shah model. The SaT approach separates the segmentation into two stages: first, a convex energy function is minimized to obtain a smoothed image; then, a thresholding technique is applied to segment the smoothed image. The energy function consists of three weighted terms and the weights are called the regularization parameters. Selecting appropriate regularization parameters is crucial to achieving effective segmentation results. Traditionally, the regularization parameters are chosen by trial-and-error, which is a very time-consuming procedure and is not practical in real applications. In this paper, we apply a Bayesian inference approach to infer the regularization parameters and estimate the smoothed image. We analyze the convex variant Mumford–Shah variational model from a statistical perspective and then construct a hierarchical Bayesian model. A mean field variational family is used to approximate the posterior distribution. The variational density of the smoothed image is assumed to have a Gaussian density, and the hyperparameters are assumed to have Gamma variational densities. All the parameters in the Gaussian density and Gamma densities are iteratively updated. Experimental results show that the proposed approach is capable of generating high-quality segmentation results. Although the proposed approach contains an inference step to estimate the regularization parameters, it requires less CPU running time to obtain the smoothed image than previous methods.

Abstract. Ground-based astronomy utilizes modern telescopes to obtain information on the universe by analyzing recorded signals. Due to atmospheric turbulence, the reconstruction process requires solving a deconvolution problem with an unknown point spread function (PSF). The crucial step in PSF estimation is to obtain a high-resolution phase from low-resolution phase gradients, which is a challenging problem. In this paper, when multiple frames of low-resolution phase gradients are available, we introduce PhaseNet, a deep learning approach based on the Taylor frozen flow hypothesis. Our approach incorporates a data-driven residual regularization term, of which the gradient is parameterized by a network, into the Laplacian regularization based model. To solve the model, we unroll the Nesterov accelerated gradient algorithm so that the network can be efficiently and effectively trained. Finally, we evaluate the performance of PhaseNet under various atmospheric conditions and demonstrate its superiority over TV and Laplacian regularization based methods.

Hyperspectral images (HSIs) provide exceptional spatial and spectral resolution of a scene, crucial for various remote sensing applications. However, the high dimensionality, presence of noise and outliers, and the need for precise labels of HSIs present significant challenges to the analysis of HSIs, motivating the development of performant HSI clustering algorithms. This paper introduces a novel unsupervised HSI clustering algorithm—Superpixel-based and Spatially-regularized Diffusion Learning (S2DL)—which addresses these challenges by incorporating rich spatial information encoded in HSIs into diffusion geometry-based clustering. S2DL employs the Entropy Rate Superpixel (ERS) segmentation technique to partition an image into superpixels, then constructs a spatially-regularized diffusion graph using the most representative high-density pixels. This approach reduces computational burden while preserving accuracy. Cluster modes, serving as exemplars for underlying cluster structure, are identified as the highest-density pixels farthest in diffusion distance from other highest-density pixels. These modes guide the labeling of the remaining representative pixels from ERS superpixels. Finally, majority voting is applied to the labels assigned within each superpixel to propagate labels to the rest of the image. This spatial-spectral approach simultaneously simplifies graph construction, reduces computational cost, and improves clustering performance. S2DL’s performance is illustrated with extensive experiments on four publicly available, real-world HSIs: Indian Pines, Salinas, Salinas A, and WHU-Hi. Additionally, we apply S2DL to landscape-scale, unsupervised mangrove species mapping in the Mai Po Nature Reserve, Hong Kong, using a Gaofen-5 HSI. The success of S2DL in these diverse numerical experiments indicates its efficacy on a wide range of important unsupervised remote sensing analysis tasks.

K-Means algorithm is a popular clustering method. However, it has two limitations: 1) it gets stuck easily in spurious local minima, and 2) the number of clusters$k$has to be given a priori. To solve these two issues, a multi-prototypes convex merging based K-Means clustering algorithm (MCKM) is presented. First, based on the structure of the spurious local minima of the K-Means problem, a multi-prototypes sampling (MPS) is designed to select the appropriate number of multi-prototypes for data with arbitrary shapes. Then, a merging technique, called convex merging (CM), merges the multi-prototypes to get a better local minima without$k$being given a priori. Specifically, CM can obtain the optimal merging and estimate the correct$k$. By integrating these two techniques with K-Means algorithm, the proposed MCKM is an efficient and explainable clustering algorithm for escaping the undesirable local minima of K-Means problem without given$k$first. Two theoretical proofs are given to guarantee that the cost of MCKM (MPS+CM) can achieve a constant factor approximation to the optimal cost of the K-Means problem. Experimental results performed on synthetic and real-world data sets have verified the effectiveness of the proposed algorithm.

Abstract. Spherical image processing has been widely applied in many important fields, such as omnidirectional vision for autonomous cars, global climate modeling, and medical imaging. It is nontrivial to extend an algorithm developed for flat images to the spherical ones. In this work, we focus on the challenging task of spherical image inpainting with a deep learning-based regularizer. Instead of a naive application of existing models for planar images, we employ a fast directional spherical Haar framelet transform and develop a novel optimization framework based on a sparsity assumption of the framelet transform. Furthermore, by employing progressive encoder-decoder architecture, a new and better-performed deep CNN denoiser is carefully designed and works as an implicit regularizer. Finally, we use a plug-and-play method to handle the proposed optimization model, which can be implemented efficiently by training the CNN denoiser prior. Numerical experiments are conducted and show that the proposed algorithms can greatly recover damaged spherical images and achieve the best performance over purely using a deep learning denoiser and a plug-and-play model.

In this work, a novel algorithm called SVM with Shape-adaptive Reconstruction and Smoothed Total Variation (SaR-SVM-STV) is introduced to classify hyperspectral images, which makes full use of spatial and spectral information. The Shape-adaptive Reconstruction (SaR) is introduced to preprocess each pixel based on the Pearson Correlation be-tween pixels in its shape-adaptive (SA) region. Support Vector Machines (SVMs) are trained to estimate the pixel-wise probability maps of each class. Then the Smoothed Total Variation (STV) model is applied to denoise and generate the final classification map. Experiments show that SaR-SVM-STY outperforms the SVM-STV method with a few training labels, demonstrating the significance of reconstructing hy-perspectral images before classification.

We present Deep Tensor Canonical Correlation Analysis (DTCCA), a method to learn complex nonlinear transformations of multiple views (more than two) of data such that the resulting representations are linearly correlated in high order. The high-order correlation of given multiple views is modeled by covariance tensor, which is different from most CCA formulations relying solely on the pairwise correlations. Parameters of transformations of each view are jointly learned by maximizing the high-order canonical correlation. To solve the resulting problem, we reformulate it as the best sum of rank-1 approximation, which can be efficiently solved by existing tensor decomposition method. DTCCA is a nonlinear extension of tensor CCA (TCCA) via deep networks. Comparing with kernel TCCA, DTCCA not only can deal with arbitrary dimensions of the input data, but also does not need to maintain the training data for computing representations of any given data point. Hence, DTCCA as a unified model can efficiently overcome the scalable issue of TCCA for either high-dimensional multi-view data or a large amount of views, and it also naturally extends TCCA for learning nonlinear representation. Extensive experiments on four multi-view data sets demonstrate the effectiveness of the proposed method.

We present a probabilistic semi-supervised learning (SSL) framework based on sparse graph structure learning. Different from existing SSL methods with either a predefined weighted graph heuristically constructed from the input data or a learned graph based on the locally linear embedding assumption, the proposed SSL model is capable of learning a sparse weighted graph from the unlabeled high-dimensional data and a small amount of labeled data, as well as dealing with the noise of the input data. Our representation of the weighted graph is indirectly derived from a unified model of density estimation and pairwise distance preservation in terms of various distance measurements, where latent embeddings are assumed to be random variables following an unknown density function to be learned, and pairwise distances are then calculated as the expectations over the density for the model robustness to the data noise. Moreover, the labeled data based on the same distance representations are leveraged to guide the estimated density for better class separation and sparse graph structure learning. A simple inference approach for the embeddings of unlabeled data based on point estimation and kernel representation is presented. Extensive experiments on various data sets show promising results in the setting of SSL compared with many existing methods and significant improvements on small amounts of labeled data.

Adaptive optics is a commonly used technique to correct the phase distortions caused by the Earth's atmosphere to improve the image quality of the ground-based imaging systems. However, the observed images still suffer from the blur caused by the adaptive optics residual wavefront. In this paper, we propose a model for reconstructing the residual phase in high resolution from a sequence of deformable mirror data. Our model is based on the turbulence statistics and the Taylor frozen flow hypothesis with knowledge of the wind velocities in atmospheric turbulence layers. A tomography problem for the phase distortions from different altitudes is solved in order to get a high quality phase reconstruction. We also consider inexact tomography operators resulting from the uncertainty in the wind velocities. The wind velocities are estimated from the deformable mirror data and, additionally, by including them as unknowns in the objective function. We provide a theoretical analysis on the existence of a minimizer of the objective function. To solve the associated joint optimization problem, we use an alternating minimization method which results in a high resolution reconstruction algorithm with adaptive wind velocities. Numerical simulations are carried out to show the effectiveness of our approach.

We consider the high-resolution imaging problem of 3-dimensional (3D) point source image recovery from 2-dimensional data using a method based on point spread function (PSF) engineering. The method involves a new technique, recently proposed by Prasad, based on the use of a rotating PSF with a single lobe to obtain depth from defocus. The amount of rotation of the PSF encodes the depth position of the point source. Applications include high-resolution single molecule localization microscopy as well as the problem addressed in this paper on localization of space debris using a space-based telescope. The localization problem is discretized on a cubical lattice where the coordinates of nonzero entries represent the 3D locations and the values of these entries the fluxes of the point sources. Finding the locations and fluxes of the point sources is a large-scale sparse 3D inverse problem. A new non-convex regularization method with a data-fitting term based on Kullback--Leibler (KL) divergence is proposed for 3D localization for the Poisson noise model. In addition, we propose a new scheme of estimation of the source fluxes from the KL data-fitting term. Numerical experiments illustrate the efficiency and stability of the algorithms that are trained on a random subset of image data before being applied to other images. Our 3D localization algorithms can readily be applied to other kinds of depth-encoding PSFs as well.

Parallel magnetic resonance imaging (pMRI) is a technique to accelerate the magnetic resonance imaging process. The problem of reconstructing an image from the collected pMRI data is ill-posed. Regularization is needed to make the problem well-posed. In this paper, we first construct a two-dimensional tight framelet system whose filters have the same support as the orthogonal Haar filters and are able to detect edges of an image in the horizontal, vertical, and $\pm 45^o$ directions. This system is referred to as directional Haar framelet (DHF). We then propose a pMRI reconstruction model whose regularization term is formed by the DHF. This model is solved by a fast proximal algorithm with low computational complexity. The regularization parameters are updated adaptively and determined automatically during the iteration of the algorithm. Numerical experiments for in-silico and in-vivo data sets are provided to demonstrate the superiority of the DHF-based model and the efficiency of our proposed algorithm for pMRI reconstruction.

The alternating direction method of multipliers (ADMM) is a popular and efficient first-order method that has recently found numerous applications, and the proximal ADMM is an important variant of it. The main contributions of this paper are the proposition and the analysis of a class of inertial proximal ADMMs, which unify the basic ideas of the inertial proximal point method and the proximal ADMM, for linearly constrained separable convex optimization. This class of methods are of inertial nature because at each iteration the proximal ADMM is applied to a point extrapolated at the current iterate in the direction of last movement. The recently proposed inertial primal-dual algorithm [A. Chambolle and T. Pock, On the ergodic convergence rates of a first-order primal-dual algorithm, preprint, 2014, Algorithm 3] and the inertial linearized ADMM [C. Chen, S. Ma, and J. Yang, arXiv:1407.8238, eq. (3.23)] are covered as special cases. The proposed algorithmic framework is very general in the sense that the weighting matrices in the proximal terms are allowed to be only positive semidefinite, but not necessarily positive definite as required by existing methods of the same kind. By setting the two proximal terms to zero, we obtain an inertial variant of the classical ADMM, which is to the best of our knowledge new. We carry out a unified analysis for the entire class of methods under very mild assumptions. In particular, convergence, as well as asymptotic $o(1/\sqrt{k})$ and nonasymptotic $O(1/\sqrt{k})$ rates of convergence, are established for the best primal function value and feasibility residues, where $k$ denotes the iteration counter. The global iterate convergence of the generated sequence is established under an additional assumption. We also present extensive experimental results on total variation--based image reconstruction problems to illustrate the profits gained by introducing the inertial extrapolation steps.

In this paper, a two-stage method for segmenting blurry images in the presence of Poisson or multiplicative Gamma noise is proposed. The method is inspired by a previous work on two-stage segmentation and the usage of an I-divergence term to handle the noise. The first stage of our method is to find a smooth solution $u$ to a convex variant of the Mumford--Shah model where the $\ell_2$ data-fidelity term is replaced by an I-divergence term. A primal-dual algorithm is adopted to efficiently solve the minimization problem. We prove the convergence of the algorithm and the uniqueness of the solution $u$. Once $u$ is obtained, in the second stage, the segmentation is done by thresholding $u$ into different phases. The thresholds can be given by the users or can be obtained automatically by using any clustering method. In our method, we can obtain any $K$-phase segmentation ($K\geq 2$) by choosing $(K-1)$ thresholds after $u$ is found. Changing $K$ or the thresholds does not require $u$ to be recomputed. Experimental results show that our two-stage method performs better than many standard two-phase or multiphase segmentation methods for very general images, including antimass, tubular, magnetic resonance imaging, and low-light images.

Tight-frame, a generalization of orthogonal wavelets, has been used successfully in various problems in image processing, including inpainting, impulse noise removal, and superresolution image restoration. Segmentation is the process of identifying object outlines within images. There are quite a few efficient algorithms for segmentation such as model-based approaches, pattern recognition techniques, tracking-based approaches, and artificial intelligence--based approaches. In this paper, we propose applying the tight-frame approach to automatically identify tube-like structures in medical imaging, with the primary application of segmenting blood vessels in magnetic resonance angiography images. Our method iteratively refines a region that encloses the potential boundary of the vessels. At each iteration, we apply the tight-frame algorithm to denoise and smooth the potential boundary and sharpen the region. The cost per iteration is proportional to the number of pixels in the image. We prove that the iteration converges in a finite number of steps to a binary image whereby the segmentation of the vessels can be done straightforwardly. Numerical experiments on synthetic and real two-dimensional (2D) and three-dimensional (3D) images demonstrate that our method is more accurate when compared with some representative segmentation methods, and it usually converges within a few iterations.

The Mumford--Shah model is one of the most important image segmentation models and has been studied extensively in the last twenty years. In this paper, we propose a two-stage segmentation method based on the Mumford--Shah model. The first stage of our method is to find a smooth solution $g$ to a convex variant of the Mumford--Shah model. Once $g$ is obtained, then in the second stage the segmentation is done by thresholding $g$ into different phases. The thresholds can be given by the users or can be obtained automatically using any clustering methods. Because of the convexity of the model, $g$ can be solved efficiently by techniques like the split-Bregman algorithm or the Chambolle--Pock method. We prove that our method is convergent and that the solution $g$ is always unique. In our method, there is no need to specify the number of segments $K$ ($K\geq2$) before finding $g$. We can obtain any $K$-phase segmentations by choosing $(K-1)$ thresholds after $g$ is found in the first stage, and in the second stage there is no need to recompute $g$ if the thresholds are changed to reveal different segmentation features in the image. Experimental results show that our two-stage method performs better than many standard two-phase or multiphase segmentation methods for very general images, including antimass, tubular, MRI, noisy, and blurry images.

The total variation (TV) model is attractive in that it is able to preserve sharp attributes in images. However, the restored images from TV-based methods do not usually stay in a given dynamic range, and hence projection is required to bring them back into the dynamic range for visual presentation or for storage in digital media. This will affect the accuracy of the restoration as the projected image will no longer be the minimizer of the given TV model. In this paper, we show that one can get much more accurate solutions by imposing box constraints on the TV models and solving the resulting constrained models. Our numerical results show that for some images where there are many pixels with values lying on the boundary of the dynamic range, the gain can be as great as 10.28 decibel in the peak signal-to-noise ratio. One traditional hindrance using the constrained model is that it is difficult to solve. However, in this paper, we propose using the alternating direction method of multipliers (ADMM) to solve the constrained models. This leads to a fast and convergent algorithm that is applicable for both Gaussian and impulse noise. Numerical results show that our ADMM algorithm is better than some state-of-the-art algorithms for unconstrained models in terms of both accuracy and robustness with respect to the regularization parameter.

The composition vector (CV) method is an alignment-free method for sequence comparison. Because of its simplicity when compared with multiple sequence alignment methods, the method has been widely discussed lately; and some formulas based on probabilistic models, like Hao’s and Yu’s formulas, have been proposed. In this paper, we improve these formulas by using the entropy principle which can quantify the nonrandomness occurrence of patterns in the sequences. More precisely, existing formulas are used to generate a set of possible formulas from which we choose the one that maximizes the entropy. We give the closed-form solution to the resulting optimization problem. Hence, from any given CV formula, we can find the corresponding one that maximizes the entropy. In particular, we show that Hao’s formula is itself maximizing the entropy and we derive a new entropy-maximizing formula from Yu’s formula. We illustrate the accuracy of our new formula by using both simulated and experimental data sets. For the simulated data sets, our new formula gives the best consensus and significant values for three different kinds of evolution models. For the data set of tetrapod 18S rRNA sequences, our new formula groups the clades of bird and reptile together correctly, where Hao’s and Yu’s formulas failed. Using real data sets with different sizes, we show that our formula is more accurate than Hao’s and Yu’s formulas even for small data sets.

Image restoration is a computationally intensive problem as a large number of pixel values have to be determined. Since the pixel values of digital images can attain only a finite number of values (e.g., 8-bit images can have only 256 gray levels), one would like to recover an image within some dynamic range. This leads to the imposition of box constraints on the pixel values. The traditional gradient projection methods for constrained optimization can be used to impose box constraints, but they may suffer from either slow convergence or repeated searching for active sets in each iteration. In this paper, we develop a new box-constrained multiplicative iterative (BCMI) algorithm for box-constrained image restoration. The BCMI algorithm just requires pixelwise updates in each iteration, and there is no need to invert any matrices. We give the convergence proof of this algorithm and apply it to total variation image restoration problems, where the observed blurry images contain Poisson, Gaussian, or salt-and-pepper noises.

There are two key issues in successfully solving the image restoration problem: 1) estimation of the regularization parameter that balances data fidelity with the regularity of the solution and 2) development of efficient numerical techniques for computing the solution. In this paper, we derive a fast algorithm that simultaneously estimates the regularization parameter and restores the image. The new approach is based on the total-variation (TV) regularized strategy and Morozov's discrepancy principle. The TV norm is represented by the dual formulation that changes the minimization problem into a minimax problem. A proximal point method is developed to compute the saddle point of the minimax problem. By adjusting the regularization parameter adaptively in each iteration, the solution is guaranteed to satisfy the discrepancy principle. We will give the convergence proof of our algorithm and numerically show that it is better than some state-of-the-art methods in terms of both speed and accuracy.

Loss of information in a wavelet domain can occur during storage or transmission when the images are formatted and stored in terms of wavelet coefficients. This calls for image inpainting in wavelet domains. In this paper, a variational approach is used to formulate the reconstruction problem. We propose a simple but very efficient iterative scheme to calculate an optimal solution and prove its convergence. Numerical results are presented to show the performance of the proposed algorithm.

Image inpainting in wavelet domains refers to the recovery of an image from incomplete and/or inaccurate wavelet coefficients. To reconstruct the image, total variation (TV) models have been widely used in the literature, and they produce high-quality reconstructed images. In this paper, we consider an unconstrained, TV-regularized, $\ell_2$-data-fitting model to recover the image. The model is solved by the alternating direction method (ADM). At each iteration, the ADM needs to solve three subproblems, all of which have closed-form solutions. The per-iteration computational cost of the ADM is dominated by two Fourier transforms and two wavelet transforms, all of which admit fast computation. Convergence of the ADM iterative scheme is readily obtained. We also discuss extensions of this ADM scheme to solving two closely related constrained models. We present numerical results to show the efficiency and stability of the ADM for solving wavelet domain image inpainting problems. Numerical results comparing the ADM with some recent algorithms are also reported.

A two-phase image restoration method based upon total variation regularization combined with an L(1)-data-fitting term for impulse noise removal and deblurring is proposed. In the first phase, suitable noise detectors are used for identifying image pixels contaminated by noise. Then, in the second phase, based upon the information on the location of noise-free pixels, images are deblurred and denoised simultaneously. For efficiency reasons, in the second phase a superlinearly convergent algorithm based upon Fenchel-duality and inexact semismooth Newton techniques is utilized for solving the associated variational problem. Numerical results prove the new method to be a significantly advance over several state-of-the-art techniques with respect to restoration capability and computational efficiency.

A wavelet inpainting problem refers to the problem of filling in missing wavelet coefficients in an image. A variational approach was used by Chan et al. The resulting functional was minimized by the gradient descent method. In this paper, we use an optimization transfer technique which involves replacing their univariate functional by a bivariate functional by adding an auxiliary variable. Our bivariate functional can be minimized easily by alternating minimization: for the auxiliary variable, the minimum has a closed form solution, and for the original variable, the minimization problem can be formulated as a classical total variation (TV) denoising problem and, hence, can be solved efficiently using a dual formulation. We show that our bivariate functional is equivalent to the original univariate functional. We also show that our alternating minimization is convergent. Numerical results show that the proposed algorithm is very efficient and outperforms that of Chan et al.

We present novel wavelet-based inpainting algorithms. Applying ideas from anisotropic regularization and diffusion, our models can better handle degraded pixels at edges. We interpret our algorithms within the framework of forward-backward splitting methods in convex analysis and prove that the conditions for ensuring their convergence are fulfilled. Numerical examples illustrate the good performance of our algorithms.

This paper proposes an image statistic for detecting random-valued impulse noise. By this statistic, we can identify most of the noisy pixels in the corrupted images. Combining it with an edge-preserving regularization, we obtain a powerful two-stage method for denoising random-valued impulse noise, even for noise levels as high as 60%. Simulation results show that our method is significantly better than a number of existing techniques in terms of image restoration and noise detection.

A popular way to restore images comprising edges is to minimize a cost function combining a quadratic data-fidelity term and an edge-preserving (possibly nonconvex) regularizalion term. Mainly because of the latter term, the calculation of the solution is slow and cumbersome. Half-quadratic (HQ) minimization (multiplicative form) was pioneered by Geman and Reynolds (1992) in order to alleviate the computational task in the context of image reconstruction with nonconvex regularization. By promoting the idea of locally homogeneous image models with a continuous-valued line process, they reformulated the optimization problem in terms of an augmented cost function which is quadratic with respect to the image and separable with respect to the line process, hence the name "half quadratic." Since then, a large amount of papers were dedicated to HQ minimization and important results--including edge-preservation along with convex regularization and convergence-have been obtained. In this paper, we show that HQ minimization (multiplicative form) is equivalent to the most simple and basic method where the gradient of the cost function is linearized at each iteration step. In fact, both methods give exactly the same iterations. Furthermore, connections of HQ minimization with other methods, such as the quasi-Newton method and the generalized Weiszfeld's method, are straightforward.

Video clip consists of frames, and each frame can be considered as a transformed picture of the reference frame. In this paper, we briefly discuss a framelet method for high-resolution image reconstruction to enhance the resolution of video clips. The detailed discussion can be found in R.H. Chan et al. (2005). Experiments on an actual video clip show that our method can provide information that are not discernable from the given video clip.

Recently, a two-phase scheme for removing salt-and-pepper impulse noise has been proposed [R.H. Chan et al], In the first phase, an adaptive median filter is used to identify pixels which are likely to be contaminated by noise (noise candidates). In the second phase, the image is restored by minimizing a specialized regularization functional that applies only to those selected noise candidates. As an extension of this work, we propose an efficient method to accomplish the second phase. The speed of our method can be double as that of the method proposed in [R.H. Chan et al] for images contaminated by 30% salt-and-pepper noise and is faster for higher noise level.

This paper proposes a two-phase scheme for removing salt-and-pepper impulse noise. In the first phase, an adaptive median filter is used to identify pixels which are likely to be contaminated by noise (noise candidates). In the second phase, the image is restored using a specialized regularization method that applies only to those selected noise candidates. In terms of edge preservation and noise suppression, our restored images show a significant improvement compared to those restored by using just nonlinear filters or regularization methods only. Our scheme can remove salt-and-pepper-noise with a noise level as high as 90%.

This work proposes a two-stage iterative method for removing random-valued impulse noise. In the first phase, we use the adaptive center-weighted median filter to identify pixels which are likely to be corrupted by noise (noise candidates). In the second phase, these noise candidates are restored using a detail-preserving regularization method which allows edges and noise-free pixels to be preserved. These two phases are applied alternatively. Simulation results indicate that the proposed method is significantly better than those using just nonlinear filters or regularization only.

In PDE image restoration problems, one has to invert operators which is a sum of a blurring operator and an elliptic operator with highly varying coefficient. We present a preconditioner for such operators, which can be used with the conjugate gradient (CG) method, and compare it with Vogel and Oman's (see SIAM J. Sci. Stat. Comput., vol.17, p.227-38, 1996, and IEEE Trans. Image Processing, vol.7, p.813-24, 1998) product preconditioner.