Wind velocity estimation is a critical component of weather prediction, but accurately estimating atmospheric winds with in-place instruments is expensive, inefficient, and provides only sparse coverage. Consequently, methods which can extract wind patterns from remote sensing data would offer significant benefits—satellites and satellite-derived data in particular, provide comprehensive coverage of global weather phenomena. In this paper, we develop an optical flow method which successfully extracts dense wind velocity estimates from water vapor data, a commonly estimated quantity by satellites equipped with infrared or microwave instruments. The test-data comes from direct numerical simulations of a mesoscale convective system over the eastern Pacific and provides accompanying ground truth wind velocities, allowing us to quantitatively measure the performance of our methods.

Abstract. Motivated by Tadmor, Nezzar, and Vese‘s work [ 30 ] dedicated to multiscale image representation using hierarchical [Formula: see text] decompositions, we propose transposing their approach to the case of registration, a task which consists in determining a smooth deformation aligning the salient constituents visible in an image into their counterpart in another. The underlying goal is to obtain a hierarchical decomposition of the deformation in the form of a composition of intermediate deformations: the coarser one, computed from versions of the two images capturing the essential features, encodes the main structural/geometrical deformation, while iterating the procedure and refining the versions of the two images yields more accurate deformations that faithfully map small-scale features. The proposed model falls within the framework of variational methods and hyperelasticity by viewing the shapes to be matched as Ogden materials. The material behavior is described by means of a specifically tailored strain energy density function, complemented by [Formula: see text]-penalizations ensuring that the computed deformation is a bi-Lipschitz homeomorphism. Theoretical results emphasizing the mathematical soundness of the model are provided, among which the existence of minimizers/asymptotic results, and a suitable numerical algorithm is supplied, along with numerical simulations demonstrating the ability of the model to produce accurate hierarchical representations of deformations. A very preliminary version of this work has been accepted for publication in the Eighth International Conference on Scale Space and Variational Methods in Computer Vision [Springer, Cham, Switzerland, 2021] but it does not include all the theoretical results, nor the detailed related proofs. A more complete and detailed analysis of the numerical experiments is also provided. The theoretical analysis of the numerical algorithm (introduced in section 3 and which is a result in itself) will be the subject of a separate article in preparation.

The purpose of this work is to use data from the Atmospheric Infrared Sounder (AIRS) and the Moderate Resolution Imaging Spectroradiometer (MODIS) to refine our knowledge of post-launch AIRS point spread functions (PSFs), including suspected changes over the mission. We develop methodology, by deriving mathematical optimization formulation based on variational principles and Sobolev gradient descent, for reconstruction of AIRS spatial response functions. We use the data over the ocean, collected for the duration of a day, to reconstruct a single PSF. We examine the repeatability of our reconstructions by computing PSFs based on data collected during two consecutive days, and also investigating the change in the reconstructions by comparing the reconstructed PSF based on data collected in the beginning and the middle of the mission. We also quantify uncertainties in our reconstruction results.

The multiscale hierarchical decomposition method (MHDM) proposed in [E. Tadmor, S. Nezzar, and L. Vese, Multiscale Model. Simul., 2 (2004), pp. 554--579] has been proven very appropriate for denoising images with features at different scales and for scale separation. Extensions of it to image deblurring or to time-dependent settings [E. Tadmor and P. Athavale, Inverse Probl. Imaging, 35 (2009), pp. 693--710] have also been considered, showing convergence properties and more applications. The recent paper [K. Modin, A. Nachman, and L. Rondi, Adv. Math., 346 (2019), pp. 1009--1066] fills in further qualitative results even for nonlinear problems and introduces a tighter version of MHDM with better convergence properties. The contribution of the present work is as follows. First, we derive novel error estimates for MHDM and its tighter version. Second, we provide rules for early stopping of the algorithms in the case of perturbed data, while still ensuring stable approximations of the true image. Last but not least, we propose a refined version of the tighter MHDM, which allows recovering structured images and promotes different features of the components, as compared to the entire image. The theoretical results are validated by numerous numerical experiments for image denoising and deblurring, which also assess the analyzed methods in terms of rate of convergence, stopping rule, and quality of restoration.

Multi-angle Imaging Spectro-Radiometer (MISR) instrument provides the multi-angle images of aerosols and clouds. There are a multitude of challenges for accurate stereo imaging of clouds and aerosols including the high variation of radiative properties of aerosols and clouds within an image. In this work, we adapt an image model to separate two specific types of clouds frequently appearing in MISR images. Specifically, we separate cirrus and cumulus clouds in the two-dimensional MISR single-channel images. We characterize these two cloud types according to their spatial variations and optical brightness. Cirrus clouds appear smooth and optically thin, while cumulus clouds present high optical oscillations and appear brighter. We adapt the additive piecewise-smooth (APS) model for this cloud separation task. We describe the differences between our results and the results of the previous joint work of the second author on cloud separation.

This work is devoted to image restoration (denoising and deblurring) by variational models. As in our prior work [Inverse Probl. Imaging, 3 (2009), pp. 43--68], the image $\tilde f$ to be restored is assumed to be the sum of a cartoon component $u$ (a function of bounded variation) and a texture component $v$ (an oscillatory function in a Sobolev space with negative degree of differentiability). In order to separate noise from texture in a blurred noisy textured image, we need to collect some information that helps distinguish noise, especially Gaussian noise, from texture. We know that homogeneous Sobolev spaces of negative differentiability help capture oscillations in images very well; however, these spaces do not directly provide clear distinction between texture and noise, which is also highly oscillatory, especially when the blurring effect is noticeable. Here, we propose a new method for distinguishing noise from texture by considering a family of Sobolev norms corresponding to noise and texture. It turns out that the two Sobolev norm profiles for texture and noise are different, and this enables us to better separate noise from texture during the deblurring process.

We believe that by adapting architectures to fit the requirements of a given application domain, we can significantly improve the efficiency of computation. To validate the idea for our application domain, we evaluate a wide spectrum of commodity computing platforms to quantify the potential benefits of heterogeneity and customization for the domain-specific applications. In particular, we choose medical imaging as the application domain for investigation, and study the application performance and energy efficiency across a diverse set of commodity hardware platforms, such as general-purpose multi-core CPUs, massive parallel many-core GPUs, low-power mobile CPUs and fine-grain customizable FPGAs. This study leads to a number of interesting observations that can be used to guide further development of domain-specific architectures.

We propose a new variational model for image restoration using BV and Sobolev spaces. It is well known that homogeneous Sobolev spaces of negative differentiability can capture oscillatory information very well, however, just one Sobolev space hardly recognizes any difference between texture and noise. By a means of learning a series of Sobolev norms of pure texture and pure noise that will provide us with one dimensional profiles describing different behaviors of texture and noise, we will be able to make a distinction between texture and noise, and use these measurements in restoring a better image. We want to point out that our model is specifically designed to deal with noisy blurred images. Parameter insensitivity is one of the advantages of using a series of Sobolev spaces.

We propose here a class of restoration algorithms for color images, based upon the Mumford-Shah (MS) model and nonlocal image information. The Ambrosio-Tortorelli and Shah elliptic approximations are defined to work in a small local neighborhood, which are sufficient to denoise smooth regions with sharp boundaries. However, texture is nonlocal in nature and requires semilocal/non-local information for efficient image denoising and restoration. Inspired from recent works (nonlocal means of Buades, Coll, Morel, and nonlocal total variation of Gilboa, Osher), we extend the local Ambrosio-Tortorelli and Shah approximations to MS functional (MS) to novel nonlocal formulations, for better restoration of fine structures and texture. We present several applications of the proposed nonlocal MS regularizers in image processing such as color image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, and color filter array demosaicing. In all the applications, the proposed nonlocal regularizers produce superior results over the local ones, especially in image inpainting with large missing regions. We also prove several characterizations of minimizers based upon dual norm formulations.

It has been speculated that the human motion system combines noisy measurements with prior expectations in an optimal, or rational, manner. The basic goal of our work is to discover experimentally which prior distribution is used. More specifically, we seek to infer the functional form of the motion prior from the performance of human subjects on motion estimation tasks. We restricted ourselves to priors which combine three terms for motion slowness, first-order smoothness, and second-order smoothness. We focused on two functional forms for prior distributions: L2-norm and L1-norm regularization corresponding to the Gaussian and Laplace distributions respectively. In our first experimental session we estimate the weights of the three terms for each functional form to maximize the fit to human performance. We then measured human performance for motion tasks and found that we obtained better fit for the L1-norm (Laplace) than for the L2-norm (Gaussian). We note that the L1-norm is also a better fit to the statistics of motion in natural environments. In addition, we found large weights for the second-order smoothness term, indicating the importance of high-order smoothness compared to slowness and lower-order smoothness. To validate our results further, we used the best fit models using the L1-norm to predict human performance in a second session with different experimental setups. Our results showed excellent agreement between human performance and model prediction -- ranging from 3\% to 8\% for five human subjects over ten experimental conditions -- and give further support that the human visual system uses an L1-norm (Laplace) prior.

Can images be decomposed into the sum of a geometric part and a textural part? In a theoretical breakthrough, [Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. Providence, RI: American Mathematical Society, 2001] proposed variational models that force the geometric part into the space of functions with bounded variation, and the textural part into a space of oscillatory distributions. Meyer's models are simple minimization problems extending the famous total variation model. However, their numerical solution has proved challenging. It is the object of a literature rich in variants and numerical attempts. This paper starts with the linear model, which reduces to a low-pass/high-pass filter pair. A simple conversion of the linear filter pair into a nonlinear filter pair involving the total variation is introduced. This new-proposed nonlinear filter pair retains both the essential features of Meyer's models and the simplicity and rapidity of the linear model. It depends upon only one transparent parameter: the texture scale, measured in pixel mesh. Comparative experiments show a better and faster separation of cartoon from texture. One application is illustrated: edge detection.

This paper presents a new image segmentation framework which employs a shape prior in the form of an edge strength function to introduce a higher-level influence on the segmentation process. We formulate segmentation as the minimization of three coupled functionals, respectively, defining three processes: prior-guided segmentation, shape feature extraction and local deformation estimation. Particularly, the shape feature extraction process is in charge of estimating an edge strength function from the evolving object region. The local deformation estimation process uses this function to determine a meaningful correspondence between a given prior and the evolving object region, and the deformation map estimated in return supervises the segmentation by enforcing the evolving object boundary towards the prior shape.

We wish to recover an original image u from several blurry-noisy versions fk, called frames. We assume a more severe degradation model, in which the image u has been blurred by a noisy (stochastic) point spread function. We consider the problem of restoring the degraded image in a variational framework. Since the recovery of u from one single frame f is a highly ill-posed problem, we formulate two minimization problems based on the multiframe approach proposed for image super-resolution by Marquina-Osher [2008]. Several experimental results for image restoration are shown, illustrating that the proposed models give visually satisfactory results.

The implicit framework of the level-set method has several advantages when tracking propagating fronts. Indeed, the evolving contour is embedded in a higher dimensional level-set function and its evolution can be phrased in terms of a Eulerian formulation. The ability of this intrinsic method to handle topological changes (merging and breaking) makes it useful in a wide range of applications (fluid mechanics, computer vision) and particularly in image segmentation, the main subject of this paper. Nevertheless, in some applications, this topological flexibility turns out to be undesirable: for instance, when the shape to be detected has a known topology, or when the resulting shape must be homeomorphic to the initial one. The necessity of designing topology-preserving processes arises in medical imaging, for example, in the human cortex reconstruction. It is known that the human cortex has a spherical topology so throughout the reconstruction process this topological feature must be preserved. Therefore, we propose in this paper a segmentation model based on an implicit level-set formulation and on the geodesic active contours, in which a topological constraint is enforced.

In [8], the authors proposed the large deformation log-unbiased diffeomorphic nonlinear image registration model which has been successfully used to obtain theoretically and intuitively correct deformation maps. In this paper, we extend this idea to simultaneously registering and tracking deforming objects in a sequence of two or more images. We generalize a level set based Chan-Vese multiphase segmentation model to consider Jacobian fields while segmenting regions of growth and shrinkage in deformations. Deforming objects are thus classified based on magnitude of homogeneous deformation. Numerical experiments demonstrating our results include a pair of two-dimensional synthetic images and pairs of two-dimensional and three-dimensional serial MRI images.

This paper presents a shape representation and a variational framework for the construction of diffeomorphisms that establish "meaningful"correspondences between images, in that they preserve the local geometry of singularities such as region boundaries. At the same time, the shape representation allows enforcing shape information locally in determining such region boundaries. Our representation is based on a kernel descriptor that characterizes local shape. This shape descriptor is robust to noise and forms a scale-space in which an appropriate scale can be chosen depending on the size of features of interest in the scene. In order to preserve local shape during the matching procedure, we introduce a novel constraint to traditional energybased approaches to estimate diffeomorphic deformations, and enforce it in a variational framework.

An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of the paper is then in the combination of these three previously developed components: image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.

We cast the problem of inferring the 3D shape of a scene from a collection of defocused images in the framework of anisotropic diffusion. We propose an algorithm that can estimate the shape of a scene by inferring the diffusion coefficient of a heat equation. The method is optimal, as we pose it as the minimization of a certain cost functional based on the input images, and fast. Furthermore, we also extend our algorithm to the case of multiple images, and derive a 3D scene segmentation algorithm that can work in the presence of pictorial camouflage.

An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two subimages. The novel contribution of this paper is then in the combination of these three previously developed components, image decomposition with in-painting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. The novelty in the approach is to perform filling-in in a domain different from the original given image space. Examples on real images show the advantages of this proposed approach.

In this paper, we present a new framework for object matching between two images. This method could handle multiple pairs of overlapping and non-overlapping shapes, open curves, and landmarks. When implemented in 3-D, the same framework could be used to warp 3-D objects with minimal modification. Our approach is to use the level set formulation to represent the objects to be matched. Using this representation, the problem becomes an energy minimization problem. Cost functions for warping overlapping, non-overlapping, open curves, and landmarks are proposed. Euler-Lagrange equations are applied and gradient descent is used to solve the corresponding partial differential equations. Moreover, a general framework for linking the level set approach and the infinite dimensional group actions is discussed.

We propose a new model for image restoration and decomposition, based on the total variation minimization of Rudin-Osher-Fatemi (1992), and on some new techniques by Y. Meyer (2002) for oscillatory functions. An initial image f is decomposed into a cartoon part u and a texture or noise part v. The u component is modeled by a function of bounded variation, while the v component by an oscillatory function, with bounded H/sup -1/ norm. After some transformation, the resulting PDE is of fourth order. The proposed model continues the ideas and techniques previously introduced by the authors in L Vese et al., (2002). Image decomposition and denoising numerical results will be shown by the proposed new fourth order nonlinear partial differential equation.

An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of this paper is then in the combination of these three previously developed components, image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.

We propose a new model for active contours to detect objects in a given image, based on techniques of curve evolution, Mumford-Shah (1989) functional for segmentation and level sets. Our model can detect objects whose boundaries are not necessarily defined by the gradient. We minimize an energy which can be seen as a particular case of the minimal partition problem. In the level set formulation, the problem becomes a "mean-curvature flow"-like evolving the active contour, which will stop on the desired boundary. However, the stopping term does not depend on the gradient of the image, as in the classical active contour models, but is instead related to a particular segmentation of the image. We give a numerical algorithm using finite differences. Finally, we present various experimental results and in particular some examples for which the classical snakes methods based on the gradient are not applicable. Also, the initial curve can be anywhere in the image, and interior contours are automatically detected.