Thermal multispectral imagery is imperative for a plethora of environmental applications. Unfortunately, there are no publicly-available datasets of thermal multi-spectral images with a high spatial resolution that would enable the development of algorithms and systems in this field. However, image-to-image (I2I) translation could be used to artificially synthesize such data by transforming largely-available datasets of other visual modalities. In most cases, pairs of content-wise-aligned input-target images are not available, making it harder to train and converge to a satisfying solution. Nevertheless, some data domains, and particularly the thermal domain, have unique properties that tie the input to the output that could help mitigate those weaknesses. We propose PETIT-GAN, a physically enhanced thermal image-translating generative adversarial network to transform between different thermal modalities - a step toward synthesizing a complete thermal multispectral dataset. Our novel approach embeds physically modeled prior information in an UI2I translation to produce outputs with greater fidelity to the target modality. We further show that our solution outperforms the current state-of-the-art architectures at thermal UI2I translation by approximately 50% with respect to the standard perceptual metrics, and enjoys a more robust training procedure. The code and data used for the development and analysis of our method are publicly available and can be accessed through our project’s website: https://bermanz.github.io/PETIT

IR cameras are widely used for temperature measurements in various applications, including agriculture, medicine, and security. Low-cost IR cameras have the immense potential to replace expensive radiometric cameras in these applications; however, low-cost microbolometer-based IR cameras are prone to spatially variant nonuniformity and to drift in temperature measurements, which limit their usability in practical scenarios. To address these limitations, we propose a novel approach for simultaneous temperature estimation and nonuniformity correction (NUC) from multiple frames captured by low-cost microbolometer-based IR cameras. We leverage the camera’s physical image-acquisition model and incorporate it into a deep-learning architecture termed kernel prediction network (KPN), which enables us to combine multiple frames despite imperfect registration between them. We also propose a novel offset block that incorporates the ambient temperature into the model and enables us to estimate the offset of the camera, which is a key factor in temperature estimation. Our findings demonstrate that the number of frames has a significant impact on the accuracy of the temperature estimation and NUC. Moreover, introduction of the offset block results in significantly improved performance compared to vanilla KPN. The method was tested on real data collected by a low-cost IR camera mounted on an unmanned aerial vehicle, showing only a small average error of$0.27-0.54^{\circ } C$relative to costly scientific-grade radiometric cameras. Real data collected horizontally resulted in similar errors of$0.48-0.68^{\circ } C$. Our method provides an accurate and efficient solution for simultaneous temperature estimation and NUC, which has important implications for a wide range of practical applications.

Solving the eigenvalue problem for differential operators is a common problem in many scientific fields. Classical numerical methods rely on intricate domain discretization and yield nonanalytic or nonsmooth approximations. We introduce a novel neural network-based solver for the eigenvalue problem of differential self-adjoint operators, where the eigenpairs are learned in an unsupervised end-to-end fashion. We propose several training procedures for solving increasingly challenging tasks toward the general eigenvalue problem. The proposed solver is capable of finding the M smallest eigenpairs for a general differential operator. We demonstrate the method on the Laplacian operator, which is of particular interest in image processing, computer vision, and shape analysis among many other applications. In addition, we solve the Legendre differential equation. Our proposed method simultaneously solves several eigenpairs and can be easily used on free-form domains. We exemplify it on L-shape and circular cut domains. A significant contribution of this work is an analysis of the numerical error of this method. In particular an upper bound for the (unknown) solution error is given in terms of the (measured) truncation error of the partial differential equation and the network structure.

We introduce a novel neural network-based PDEs solver for forward and inverse problems. The solver is grid free, mesh free, and shape free, and the solution is approximated by a neural network. We employ an unsupervised approach such that the input to the network is a point set in an arbitrary domain, and the output is the set of the corresponding function values. The network is trained to minimize deviations of the learned function from the PDE solution and satisfy the boundary conditions. The resulting solution in turn is an explicit, smooth, differentiable function with a known analytical form. We solve the forward problem (observations given the underlying model's parameters), semi-inverse problem (model's parameters given the observations in the whole domain), and full tomography inverse problem (model's parameters given the observations on the boundary) by solving the forward and semi-inverse problems at the same time. The optimized loss function consists of few elements: fidelity term of $L_2$ norm that enforces the PDE in the weak sense, an $L_\infty$ norm term that enforces pointwise fidelity and thus promotes a strong solution, and boundary and initial conditions constraints. It further accommodates regularizers for the solution and/or the model's parameters of the differential operator. This setting is flexible in the sense that regularizers can be tailored to specific problems. We demonstrate our method on several free shape two dimensional (2D) second order systems with application to electrical impedance tomography (EIT) and diffusion equation. Unlike other numerical methods such as finite differences and finite elements, the derivatives of the desired function can be analytically calculated to any order. This framework enables, in principle, the solution of high order and high dimensional nonlinear PDEs.

Image segmentation is often considered a deterministic process with a single ground truth. Nevertheless, in practice, and in particular, when medical imaging analysis is considered, the extraction of regions of interest (ROIs) is ill-posed and the concept of `most probable' segmentation is model-dependent. In this paper, a measure for segmentation uncertainty in the form of segmentation error margins is introduced. This measure provides a goodness quantity and allows a `fully informed' comparison between extracted boundaries of related ROIs as well as more meaningful statistical analysis. The tool we present is based on a novel technique for segmentation sampling in the Fourier domain and Markov Chain Monte Carlo (MCMC). The method was applied to cortical and sub-cortical structure segmentation in MRI. Since the accuracy of segmentation error margins cannot be validated, we use receiver operating characteristic (ROC) curves to support the proposed method. Precision and recall scores with respect to expert annotation suggest this method as a promising tool for a variety of medical imaging applications including user-interactive segmentation, patient follow-up, and cross-sectional analysis.

We introduce an (equi-)affine invariant diffusion geometry by which surfaces that go through squeeze and shear transformations can still be properly analyzed. The definition of an affine invariant metric enables us to construct an invariant Laplacian from which local and global geometric structures are extracted. Applications of the proposed framework demonstrate its power in generalizing and enriching the existing set of tools for shape analysis.

This paper addresses the problem of correspondence establishment in binocular stereo vision. We suggest a novel spatially continuous approach for stereo matching based on the variational framework. The proposed method suggests a unique regularization term based on Mumford-Shah functional for discontinuity preserving, combined with a new energy functional for occlusion handling. The evaluation process is based on concurrent minimization of two coupled energy functionals, one for domain segmentation (occluded versus visible) and the other for disparity evaluation. In addition to a dense disparity map, our method also provides an estimation for the half-occlusion domain and a discontinuity function allocating the disparity/depth boundaries. Two new constraints are introduced improving the revealed discontinuity map. The experimental tests include a wide range of real data sets from the Middlebury stereo database. The results demonstrate the capability of our method in calculating an accurate disparity function with sharp discontinuities and occlusion map recovery. Significant improvements are shown compared to a recently published variational stereo approach. A comparison on the Middlebury stereo benchmark with subpixel accuracies shows that our method is currently among the top-ranked stereo matching algorithms.

The Laplacian raised to fractional powers can be used to generate scale spaces as was shown in recent literature by Duits, Felsberg, Florack, and Platel [$\alpha$ scale spaces on a bounded domain, in Scale Space Methods in Computer Vision, L. D. Griffin and M. Lillholm, eds., Lecture Notes in Comput. Sci. 2695, Springer, Berlin, Heidelberg, 2003, pp. 494–510] and Duits, Florack, de Graaf, and ter Haar Romeny [J. Math. Imaging Vision, 20 (2004), pp. 267–298]. In this paper, we study the anisotropic diffusion processes by defining new generators that are fractional powers of an anisotropic scale space generator. This is done in a general framework that allows us to explain the relation between a differential operator that generates the flow and the generators that are constructed from its fractional powers. We then generalize this to any other function of the operator. We discuss important issues involved in the numerical implementation of this framework and present several examples of fractional versions of the Perona–Malik and Beltrami flows along with their properties.

We introduce a novel variational method for the extraction of objects with either bilateral or rotational symmetry in the presence of perspective distortion. Information on the symmetry axis of the object and the distorting transformation is obtained as a by--product of the segmentation process. The key idea is the use of a flip or a rotation of the image to segment as if it were another view of the object. We call this generated image the symmetrical counterpart image. We show that the symmetrical counterpart image and the source image are related by planar projective homography. This homography is determined by the unknown planar projective transformation that distorts the object symmetry. The proposed segmentation method uses a level-set-based curve evolution technique. The extraction of the object boundaries is based on the symmetry constraint and the image data. The symmetrical counterpart of the evolving level-set function provides a dynamic shape prior. It supports the segmentation by resolving possible ambiguities due to noise, clutter, occlusions, and assimilation with the background. The homography that aligns the symmetrical counterpart to the source level-set is recovered via a registration process carried out concurrently with the segmentation. Promising segmentation results of various images of approximately symmetrical objects are shown.

We present a novel formulation of non-Abelian invariant feature detection. By choosing suitable measuring functions, we show that the measuring space and the corresponding feature space are equivariant with respect to the SL(2, Ropf) Lie transformation group. This group is non-Abelian and may be decomposed via the Iwasawa decomposition into meaningful transformations on images. We calculate the induced representations of this group on the measuring space. Then, via these representations we construct a set of three PDEs determining an invariant function of the features. We show that this set of equations is solved by the discriminant of a binary form of order n. Hence, the discriminant plays the role of an invariant feature detector with respect to this transformation group.

A novel system, called the oscillator system, consisting of order of p3functions (signals) on the finite field Fp, with p an odd prime, is described and studied. The new functions are proved to satisfy good autocorrelation, cross-correlation, and low peak-to- average power ratio properties. Moreover, the oscillator system is closed under the operation of discrete Fourier transform. Applications of the oscillator system for discrete radar and digital communication theory are explained. Finally, an explicit algorithm to construct the oscillator system is presented.

This paper addresses the problem of correspondence establishment in binocular stereo vision. We suggest a novel variational approach that considers both the discontinuities and occlusions. It deals with color images as well as gray levels. The proposed method divides the image domain into the visible and occluded regions where each region is handled differently. The depth discontinuities in the visible domain are preserved by use of the total variation term in conjunction with the Mumford-Shah framework. In addition to the dense disparity and the occlusion maps, our method also provides a discontinuity function revealing the location of the boundaries in the disparity map. We evaluate our method on data sets from Middlebury site showing superior performance in comparison to the state of the art variational technique.

We present regularization by invariant denoising/smoothing of Diffusion Tensor MRI (DTI). Our solution to the problem emerges from a pure geometric point of view. The image domain and the image's values are combined together and described as a (mathematical) fiber bundle. The space of all possible DT images is the space of sections of this fiber bundle. DT image is a map that attaches a three-dimensional symmetric and positive-definite (SPD) matrix to each volume element. We treat the more general space Pnof n-dimensional SPD matrices and introduce a natural GL(n)-invariant metric via the underlying algebraic structure. A metric over sections of the fiber bundle is induced then in terms of the natural metric on Pn. This turns P3tensors, and in general Pntensors, into a Riemannian symmetric spaces. By means of the Beltrami framework we define a GL(n)-invariant functional over the space of sections. Then, by calculus of variations we derive the invariant equations of motion. We show that by choosing the Iwasawa coordinates the analytical calculations as well as the numerical implementation become simple. These coordinates evolve with respect to the geometry of the section via the induced metric. The numerical implementation of these flows via standard finite difference schemes is straightforward. The result is a full GL(n) invariant algorithm which is at least as fast and efficient as the Log-Euclidean method. Finally, we demonstrate this framework on real DTI data.

The Nomarski differential interference contrast (DIC) microscopy is of widespread use for observing live biological specimens. In fertility clinics the DIC microscope is used for evaluating human embryo cells. An image formation model for DIC imaging is needed for reconstruction and quantification of the visualized specimens. This calls for a complicated analysis of the interaction of light waves with biological matter. Most works express the solution via the first Born approximation, yet a theoretical bound is known that limits the validity of such approximation to very small objects. We show in this work that the theoretical bound is not directly relevant to microscopic imaging and is far too limiting. We derive a more realistic bound and show that it may justify in many cases the use of the Born approximation in biological cell microscopic imaging. It also provides limits on the validity of the Born expansion that several works violate.

We consider the problem of restoring a multichannel image corrupted by blur and impulsive noise (e.g., salt-and-pepper noise). Using the variational framework, we consider the L1 fidelity term and several possible regularizers. In particular, we use generalizations of the Mumford-Shah (MS) functional to color images and gamma-convergence approximations to unify deblurring and denoising. Experimental comparisons show that the MS stabilizer yields better results with respect to Beltrami and total variation regularizers. Color edge detection is a beneficial by-product of our methods.

We introduce a short-time kernel for the Beltrami image enhancing flow. The flow is implemented by "convolving" the image with a space dependent kernel in a similar fashion to the solution of the heat equation by a convolution with a Gaussian kernel. The kernel is appropriate for smoothing regular (flat) 2-D images, for smoothing images painted on manifolds, and for simultaneously smoothing images and the manifolds they are painted on. The kernel combines the geometry of the image and that of the manifold into one metric tensor, thus enabling a natural unified approach for the manipulation of both. Additionally, the derivation of the kernel gives a better geometrical understanding of the Beltrami flow and shows that the bilateral filter is a Euclidean approximation of it. On a practical level, the use of the kernel allows arbitrarily large time steps as opposed to the existing explicit numerical schemes for the Beltrami flow. In addition, the kernel works with equal ease on regular 2-D images and on images painted on parametric or triangulated manifolds. We demonstrate the denoising properties of the kernel by applying it to various types of images and manifolds.

The problem of dense optical flow computation is addressed from a variational viewpoint. A new geometric framework is introduced. It unifies previous art and yields new efficient methods. Along with the framework a new alignment criterion suggests itself. It is shown that the alignment between the gradients of the optical flow components and between the latter and the intensity gradients is an important measure of the flow’s quality. Adding this criterion as a requirement in the optimization process improves the resulting flow. This is demonstrated in synthetic and real sequences.

Shape symmetry is an important cue for image understanding. In the absence of more detailed prior shape information, segmentation can be significantly facilitated by symmetry. However, when symmetry is distorted by perspectivity, the detection of symmetry becomes non-trivial, thus complicating symmetry-aided segmentation. We present an original approach for segmentation of symmetrical objects accommodating perspective distortion. The key idea is the use of the replicative form induced by the symmetry for challenging segmentation tasks. This is accomplished by dynamic extraction of the object boundaries, based on the image gradients, gray levels or colors, concurrently with registration of the image symmetrical counterpart (e.g. reflection) to itself. The symmetrical counterpart of the evolving object contour supports the segmentation by resolving possible ambiguities due to noise, clutter, distortion, shadows, occlusions and assimilation with the background. The symmetry constraint is integrated in a comprehensive level-set functional for segmentation that determines the evolution of the delineating contour. The proposed framework is exemplified on various images of skewsymmetrical objects and its superiority over state of the art variational segmentation techniques is demonstrated.

Image restoration and segmentation are both classical problems, that are known to be difficult and have attracted major research efforts. This paper shows that the two problems are tightly coupled and can be successfully solved together. Mutual support of image restoration and segmentation processes within a joint variational framework is theoretically motivated, and validated by successful experimental results. The proposed variational method integrates semi-blind image deconvolution (parametric blur-kernel), and Mumford-Shah segmentation. The functional is formulated using the T-convergence approximation and is iteratively optimized via the alternate minimization method. While the major novelty of this work is in the unified treatment of the semi-blind restoration and segmentation problems, the important special case of known blur is also considered and promising results are obtained.

This paper is concerned with finding the best partial differential equation-based denoising process, out of a set of possible ones. We focus on either finding the proper weight of the fidelity term in the energy minimization formulation or on determining the optimal stopping time of a nonlinear diffusion process. A necessary condition for achieving maximal SNR is stated, based on the covariance of the noise and the residual part. We provide two practical alternatives for estimating this condition by observing that the filtering of the image and the noise can be approximated by a decoupling technique, with respect to the weight or time parameters. Our automatic algorithm obtains quite accurate results on a variety of synthetic and natural images, including piecewise smooth and textured ones. We assume that the statistics of the noise were previously estimated. No a priori knowledge regarding the characteristics of the clean image is required. A theoretical analysis is carried out, where several SNR performance bounds are established for the optimal strategy and for a widely used method, wherein the variance of the residual part equals the variance of the noise.

Denoising algorithms based on gradient dependent regularizers, such as nonlinear diffusion processes and total variation denoising, modify images towards piecewise constant functions. Although edge sharpness and location is well preserved, important information, encoded in image features like textures or certain details, is often compromised in the process of denoising. We propose a mechanism that better preserves fine scale features in such denoising processes. A basic pyramidal structure-texture decomposition of images is presented and analyzed. A first level of this pyramid is used to isolate the noise and the relevant texture components in order to compute spatially varying constraints based on local variance measures. A variational formulation with a spatially varying fidelity term controls the extent of denoising over image regions. Our results show visual improvement as well as an increase in the signal-to-noise ratio over scalar fidelity term processes. This type of processing can be used for a variety of tasks in partial differential equation-based image processing and computer vision, and is stable and meaningful from a mathematical viewpoint.

We address the issue of textured image segmentation in the context of the Gabor feature space of images. Gabor filters tuned to a set of orientations, scales and frequencies are applied to the images to create the Gabor feature space. A two-dimensional Riemannian manifold of local features is extracted via the Beltrami framework. The metric of this surface provides a good indicator of texture changes and is used, therefore, in a Beltrami-based diffusion mechanism and in a geodesic active contours algorithm for texture segmentation. The performance of the proposed algorithm is compared with that of the edgeless active contours algorithm applied for texture segmentation. Moreover, an integrated approach, extending the geodesic and edgeless active contours approaches to texture segmentation, is presented. We show that combining boundary and region information yields more robust and accurate texture segmentation results.

Object detection and segmentation can be facilitated by the availability of a reference object. However, accounting for possible transformations between the different object views, as part of the segmentation process, remains a challenge. Recent works address this problem by using comprehensive training data. Other approaches are applicable only to limited object classes or can only accommodate similarity transformations. We suggest a novel variational approach to prior-based segmentation, which accounts for planar projective transformation, using a single reference object. The prior shape is registered concurrently with the segmentation process, without point correspondence. The algorithm detects the object of interest and correctly extracts its boundaries. The homography between the two object views is accurately recovered as well. Extending the Chan-Vese level set framework, we propose a region-based segmentation functional that includes explicit representation of the projective homography between the prior shape and the shape to segment. The formulation is derived from two-view geometry. Segmentation of a variety of objects is demonstrated and the recovered transformation is verified.

Diffusion Tensor Magnetic Resonance Imaging (DT-MRI) is a non inva- sive method for brain neuronal fibers delineation. Here we show a mod- ification for DT-MRI that allows delineation of neuronal fibers which are infiltrated by edema. We use the Muliple Tensor Variational (MTV) framework which replaces the diffusion model of DT-MRI with a mul- tiple component model and fits it to the signal attenuation with a vari- ational regularization mechanism. In order to reduce free water con- tamination we estimate the free water compartment volume fraction in each voxel, remove it, and then calculate the anisotropy of the remaining compartment. The variational framework was applied on data collected with conventional clinical parameters, containing only six diffusion di- rections. By using the variational framework we were able to overcome the highly ill posed fitting. The results show that we were able to find fibers that were not found by DT-MRI.

The linear and nonlinear scale spaces, generated by the inherently real-valued diffusion equation, are generalized to complex diffusion processes, by incorporating the free Schrödinger equation. A fundamental solution for the linear case of the complex diffusion equation is developed. Analysis of its behavior shows that the generalized diffusion process combines properties of both forward and inverse diffusion. We prove that the imaginary part is a smoothed second derivative, scaled by time, when the complex diffusion coefficient approaches the real axis. Based on this observation, we develop two examples of nonlinear complex processes, useful in image processing: a regularized shock filter for image enhancement and a ramp preserving denoising process.

In many medical computer vision tasks the relevant data is attached to a specific tissue such as the colon or the cortex. This situation calls for regularization techniques which are defined over surfaces. We introduce in this paper the Beltrami flow over implicit manifolds. This new regularization technique overcomes the over-smoothing of the L/sub 2/ flow and the staircasing effects of the L/sub 1/ flow, that were recently suggested via the harmonic map methods. The key of our approach is first to clarify the link between the intrinsic Polyakov action and the implicit harmonic energy functional and then use the geometrical understanding of the Beltrami flow to generalize it to images on implicitly defined non flat surfaces. It is shown that once again the Beltrami flow interpolates between the L/sub 2/ and L/sub 1/ flows on non flat surfaces. The implementation scheme of this flow is presented and various experimental results obtained on a set of various real images illustrate the performances of the approach as well as the differences with the harmonic map flows. This extension of the Beltrami flow to the case of non flat surfaces opens new perspectives in the regularization of noisy data defined on manifolds.

Shape-from-shading (SFS) is a fundamental problem in computer vision. The vast majority of research in this field have assumed orthography as its projection model. This paper reexamines the basis of SFS, the image irradiance equation, under an assumption of perspective projection. The paper also shows that the perspective image irradiance equation depends merely on the natural logarithm of the depth function (and not on the depth function itself), and as such it is invariant to scale changes of the depth function. We then suggest a simple reconstruction algorithm based on the perspective formula, and compare it to existing orthographic SFS algorithms. This simple algorithm obtained lower error rates than legacy SFS algorithms, and equated with and sometimes surpassed state-of-the-art algorithms. These findings lend support to the assumption that transition to a more realistic set of assumptions improves reconstruction significantly.

The widely used denoising algorithms based on nonlinear diffusion, such as Perona-Malik and total variation denoising, modify images toward piecewise constant functions. Though edge sharpness and location is well preserved, important information, encoded in image features like textures or small details, is often lost in the process. We suggest a simple way to better preserve textures, small details, or global information. This is done by adding a spatially varying fidelity term that controls the amount of denoising in any region of the image. This form is very simple, can be used for a variety of tasks in PDE-based image processing and computer vision, and is stable and meaningful from a mathematical point of view.

In this paper, we study, in this paper, the problem of denoising images/data which are defined are nonflat surfaces. This problem arises often in many medical imaging tasks. The Beltrami flow which was defined in an explicit-intrinsic manner is generalized here to nonflat surfaces and is defined in an implicit way. We formulate the flow in a variational way which is generalized to a scalar field defined over an n-dimensional manifold. The implementation scheme of this flow is presented and various experimental results obtained on a set of real images illustrate the performances of the approach as well as the differences between various flows of interests.

Signal and image enhancement is considered in the context of a new type of diffusion process that simultaneously enhances, sharpens, and denoises images. The nonlinear diffusion coefficient is locally adjusted according to image features such as edges, textures, and moments. As such, it can switch the diffusion process from a forward to a backward (inverse) mode according to a given set of criteria. This results in a forward-and-backward (FAB) adaptive diffusion process that enhances features while locally denoising smoother segments of the signal or image. The proposed method, using the FAB process, is applied in a super-resolution scheme. The FAB method is further generalized for color processing via the Beltrami flow, by adaptively modifying the structure tensor that controls the nonlinear diffusion process. The proposed structure tensor is neither positive definite nor negative, and switches between these states according to image features. This results in a forward-and-backward diffusion flow where different regions of the image are either forward or backward diffused according to the local geometry within a neighborhood.

Two nonlinear diffusion processes with time-dependent diffusion coefficients are presented. Both processes converge to nontrivial solutions, eliminating the need to impose an arbitrary diffusion stopping time, otherwise required in the implementation of most nonlinear diffusion processes. The two schemes employ nonlinear cooling mechanisms that preserve edges. One scheme is intended for general denoising, whereas the other is targeted for enhancement or segmentation of images. Simulation results indicate that the proposed schemes provide a stable and efficient tool for processing of noisy images.

Signal and image enhancement in the presence of noise is considered in the context of the scale-space approach. A modified dynamic process, based on the action of a nonlinear diffusion equation, is presented. The diffusion coefficient is adjusted according to the local gradient, intensity and other image properties, and as such also reverses its sign, i.e. switches from a forward to a backward (inverse) diffusion process according to a given criterion. This results in enhancement of transients and singularities in the one-dimensional case, and of edges in images, while locally denoising smoother segments of the signal or image. Regularization of the ill-posed inverse diffusion problem is discussed. Examples of both one-dimensional signals and images are presented.

The Beltrami geometrical framework for scale-space flows is generalized to nontrivial color space geometries and implemented in analysis and processing of color images. We demonstrate how various models of color perception, interpreted as geometries of the color space, result in different enhanced processing schemes.

Algorithms for resolution enhancement are needed in various applications of image processing and communication such as compression, and HDTV. We develop a geometrical algorithm, based on diffusion processes which are used both for smoothing of colored images and enhancement of the colored images. The latter is accomplished by "solving" an inverse diffusion problem which is ill-posed. In order to stabilize the flow, as well as to enhance important features (e.g. edges) at the expense of less important image domains, we use a modified Beltrami diffusion equation. Results indicate that it is possible to impose the backwards flow, in spite of the instabilities, and thereby enhance the image up to a certain level of resolution which depends on the nature of the image.

In image analysis, processing and understanding, it is highly desirable to process the image and feature domains by methods that are specific to these domains. We show how the geometrical framework for scale-space flows is most convenient for this purpose, and demonstrate, as an example, how one can switch continuously between different processing flows of images and color domains. The parameter that interpolates between the norms is the luminance strength, taken here as a local function of the image embedding space. The resulting spatial and/or luminance preserving flow can be used for conditional denoising, enhancement and segmentation. This example demonstrates that the proposed framework can incorporate context or task dependent data, furnished by either the human user or by an active vision subsystem, in a coherent and convenient way.

We introduce a new geometrical framework based on which natural flows for image scale space and enhancement are presented. We consider intensity images as surfaces in the (x, I) space. The image is, thereby, a two-dimensional (2-D) surface in three-dimensional (3-D) space for gray-level images, and 2-D surfaces in five dimensions for color images. The new formulation unifies many classical schemes and algorithms via a simple scaling of the intensity contrast, and results in new and efficient schemes. Extensions to multidimensional signals become natural and lead to powerful denoising and scale space algorithms.

A general geometrical framework for image processing is presented. We consider intensity images as surfaces in the (x, I) space. The image is thereby a two dimensional surface in three dimensional space for gray level images. The new formulation unifies many classical schemes, algorithms, and measures via choices of parameters in "master" geometrical measure. More important, it is a simple and efficient tool for the design of natural schemes for image enhancement, segmentation, and scale space. Here we give the basic motivation and apply the scheme to enhance images. We present the concept of an image as a surface in dimensions higher than the three dimensional intuitive space. This will help us handle movies, color, and volumetric medical images.

We introduce a new geometrical framework for image processing. This framework finds a seamless link between the TV-L/sub 1/ and the L/sub 2/ norms that are often used in image processing, based on the geometry of the image and its interpretation as a surface. It unifies most of the current "scale space" models for images by a simple selection of one parameter, yet more important, it enables one to introduce new methods to deal with images in a simple and natural way. A functional called "Polyakov (1981) action", borrowed from high energy physics, is shown to be useful for image enhancement in color, texture, volumetric medical data, movies, and more.