Abstract We present a new algorithm for optimizing geometric energies and computing positively oriented simplicial mappings. Our major improvements over the state‐of‐the‐art are: (i) introduction of new energies for repairing inverted and collapsed simplices; (ii) adaptive partitioning of vertices into coordinate blocks with the blended local‐global strategy for more efficient optimization and (iii) introduction of the displacement norm for improving convergence criteria and for controlling block partitioning. Together these improvements form the basis for the Adaptive Block Coordinate Descent (ABCD) algorithm aimed at robust geometric optimization. ABCD achieves state‐of‐the‐art results in distortion minimization, even under hard positional constraints and highly distorted invalid initializations that contain thousands of collapsed and inverted elements. Starting with an invalid non‐injective initial map, ABCD behaves as a modified block coordinate descent up to the point where the current mapping is cleared of invalid simplices. Then, the algorithm converges rapidly into the chosen iterative solver. Our method is very general, fast‐converging and easily parallelizable. We show over a wide range of 2D and 3D problems that our algorithm is more robust than existing techniques for locally injective mapping.

The challenging inverse problem of blind deblurring has been investigated thoroughly for natural images. Existing algorithms exploit edge-type structures, or similarity to smaller patches within the image, to estimate the correct blurring kernel. However, these methods do not perform well enough on natural stochastic textures (NST), which are mostly random and in general are not characterized by distinct edges and contours. In NST even small kernels cause severe degradation to images. Restoration poses therefore an outstanding challenge. In this work, we refine an existing method by implementing an anisotropic fractal model to estimate the blur kernel's power spectral density. The final kernel is then estimated via an adaptation of a phase retrieval algorithm, originally proposed for sparse signals. We further incorporate additional constraints that are specific to blur filters, to yield even better results. The latter are compared with results obtained by recently published blind deblurring methods.

Embedding textured images in manifolds reveals latent information regarding texture structure and allows useful analysis of these high dimensional images in a low dimensional space. We present a framework for analysis and synthesis of natural stochastic textures (NST) which constitute an important subset of textures that are modelled as realizations of random processes. The randomness of NST differentiates them from other types of images and requires a dedicated method for analysis and synthesis. We demonstrate several applications of this framework. The first is synthesis of new types of NST. The second is NST analysis, reaffirming our previous findings regarding the fundamental properties of NST, and showing that they emerge naturally in the latent parameter space. Finally, we show the advantage of producing a manifold representation with intrinsic geometry.

Change detection is of paramount importance in medical imaging, serving as a non-invasive quantifiable powerful tool in diagnosis and in assessment of the outcome of treatment of tumors. We present a new quantitative method for detecting changes in volumetric medical data and in clustering of anatomical structures, based on assessment of volumetric distortions that are required in order to deform a test three-dimensional medical dataset segment onto its previously-acquired reference, or a given prototype in the case clustering. Unlike the voxel-based classical techniques of shape comparison, our algorithm operates on tetrahedral meshes and can, therefore be applied on both closed, simply-connected, surfaces and in volumetric domains with more sophisticated boundaries.

Natural stochastic textures (NSTs), characterized by their fine details, are prone to corruption by artifacts, introduced during the image acquisition process by the combined effect of blur and noise. While many successful algorithms exist for image restoration and enhancement, the restoration of natural textures and textured images based on suitable statistical models has yet to be further improved. We examine the statistical properties of NST using three image databases. We show that the Gaussian distribution is suitable for many NST, while other natural textures can be properly represented by a model that separates the image into two layers; one of these layers contains the structural elements of smooth areas and edges, while the other contains the statistically Gaussian textural details. Based on these statistical properties, an algorithm for the denoising of natural images containing NST is proposed, using patch-based fractional Brownian motion model and regularization by means of anisotropic diffusion. It is illustrated that this algorithm successfully recovers both missing textural details and structural attributes that characterize natural images. The algorithm is compared with classical as well as the state-of-the-art denoising algorithms.

Curvature is a fundamental component in differential geometry. It is used extensively in signal, image and shape processing, as a feature and in segmentation flows and regularization processes. In this paper we extend the notion of curvature in two ways. First, we present the Menger curvature which goes beyond classical curves and Riemannian manifolds to general metric spaces and is rigorously defined on a variety of discrete settings. We further extend the curvature to become a non-local entity using an adaptive, non-local integration measure, allowing curvature to be computed in a robust manner. Examples on natural and textural images highlight potential applications of these new concepts.

Denoising of Natural Stochastic Colored-Textures (color NST) is of special interest in image processing. Existing algorithms produce over-smoothed images with sharp edges, and do not restore the fine textural color details. We analyze the structure of color NST images and propose a simple model. This model is Gaussian and has a low number of parameters that can be estimated efficiently. A maximum-a-posteriori (MAP) scheme is proposed for patch-wise denoising of color NST. The denoised images exhibit better restored textural details compared to existing algorithms. A boosting algorithm is proposed for denoising of complex images containing both cartoon-type and textural image components.

Statistics of natural images has become an important subject of research in recent years. The highly kurtotic, non-Gaussian, statistics known to be characteristic of many natural images are exploited in various image processing tasks. In this paper, we focus on natural stochastic textures (NST) and substantiate our finding that NST have Gaussian statistics. Using the well-known statistical self-similarity property of natural images, exhibited even more profoundly in NST, we exploit a Gaussian self-similar process known as the fractional Brownian motion, to derive a fBm-PDE-based singleimage superresolution scheme for textured images. Using the same process as a prior, we also apply it in denoising of NST.

Enhancement of Natural Stochastic Textures (NST) is of special interest in image processing. These textures are considered to be realizations of random processes, and exhibit different statistical properties than those characteristic of cartoontype natural images or of structured textures. NSTs are interesting due to their fine details, which pose a challenge to enhancement algorithms. Existing algorithms, based on models of natural images, do not produce satisfactory results on NST. The long-range dependence (LRD) property of NST is explored and substantiated in this paper. An optimization scheme, implemented in the frequency domain, is then proposed. This scheme exploits the LRD as a property of the frequency magnitude, while a desired phase, extracted from the degraded image, is imposed. The scheme is general and can be further used in other image processing tasks. Zero-phase filters are of special interest in this context.

Texture enhancement presents an ongoing challenge, in spite of the considerable progress made in recent years. Whereas most of the effort has been devoted so far to enhancement of regular textures, stochastic textures that are encountered in most natural images, still pose an outstanding problem. The purpose of enhancement of stochastic textures is to recover details, which were lost during the acquisition of the image. In this paper, a texture model, based on fractional Brownian motion (fBm), is proposed. The model is global and does not entail using image patches. The fBm is a self-similar stochastic process. Self-similarity is known to characterize a large class of natural textures. The fBm-based model is evaluated and a single-image regularized superresolution algorithm is derived. The proposed algorithm is useful for enhancement of a wide range of textures. Its performance is compared with single-image superresolution methods and its advantages are highlighted.

The recently-proposed family of Telegraph-Diffusion (TeD) operators is analyzed in the context of image enhancement. Stability of such schemes in terms of energy convergence is investigated and compared with that of the widely-used diffusion-based schemes, thereby resolving the Perona-Malik paradox. The results indicating stability are further generalized in the context of the Forward-and-Backward (FAB) TeD enhancement operator. An approximated version of the enhancement operator that offers an increased stability is proposed and examined. This scheme is implemented in both stable and unstable regimes of the original FAB-TeD operator and shown to yield results similar to those obtained by the latter, without losing stability. The theoretical conclusions regarding stability of the approximated FAB-TeD are reinforced by simulations, exhibiting enhanced images with sharpened edges and yet very limited compromise on the quality of other image details.

Single-image superresolution has become a widely-studied subject in image processing in recent years. Although considerable effort has been devoted in this context to contour enhancement, much less has been done to improve the textures of a degraded image. In this study, we present a novel algorithm which utilizes the power-law spectra of approximated 1/f processes, fitting a model of degraded natural textures to recover the information lost by blurring. A mosaic of realizations of the approximated 1/f processes is first imposed on the degraded texture, and a deblurring process is then applied. The entire process is iterated until convergence. This algorithm exploits the self-similarity, characteristic of textures of natural images, and recovers the missing high-resolution information without using prior information of any specific image.

We address the challenging open problem of blindly separating time/position varying mixtures, and attempt to separate the sources from such mixtures without having prior information about the sources or the mixing system. Unlike studies concerning instantaneous or convolutive mixtures, we assume that the mixing system (medium) is varying in time/position. Attempts to solve this problem have mostly utilized, so far, online algorithms based on tracking the mixing system by methods previously developed for the instantaneous or convolutive mixtures. In contrast with these attempts, we develop a unified approach in the form of staged sparse component analysis (SSCA). Accordingly, we assume that the sources are either sparse or can be "sparsified." In the first stage, we estimate the filters of the mixing system, based on the scatter plot of the sparse mixtures' data, using a proper clustering and curve/surface fitting. In the second stage, the mixing system is inverted, yielding the estimated sources. We use the SSCA approach for solving three types of mixtures: time/position varying instantaneous mixtures, single-path mixtures, and multipath mixtures. Real-life scenarios and simulated mixtures are used to demonstrate the performance of our approach.

The conflicting demands for simultaneous low-pass and high-pass processing, required in image denoising and enhancement, still present an outstanding challenge, although a great deal of progress has been made by means of adaptive diffusion-type algorithms. To further advance such processing methods and algorithms, we introduce a family of second-order (in time) partial differential equations. These equations describe the motion of a thin elastic sheet in a damping environment. They are also derived by a variational approach in the context of image processing. The new operator enables better edge preservation in denoising applications by offering an adaptive lowpass filter, which preserves high-frequency components in the pass-band better than the adaptive diffusion filter, while offering slower error propagation across edges. We explore the action of this powerful operator in the context of image processing and exploit for this purpose the wealth of knowledge accumulated in physics and mathematics about the action and behavior of this operator. The resulting methods are further generalized for color and/or texture image processing, by embedding images in multidimensional manifolds. A specific application of the proposed new approach to superresolution is outlined.

We address the problem of separating position-varying mixtures of images. Such mixtures occur, for example, when a reflection is superimposed on a desired image by placing a semi-reflector in the optical pathway. Non-uniform light sources and/or a curved semi-reflector create a mixing system which is a function of the spatial coordinates. We use a staged sparse component analysis (SSCA) approach, in which the images are sparsified as a preprocessing stage. The mixing system is estimated in the first stage and the inverse problem is solved, in the second stage. We deal with three major types of position-varying mixing systems that encompass interesting applications. We implement our approach and illustrate its applications in separation of images.

Diffusion-type algorithms have been integrated successfully into the toolbox of image processing. We introduce a new more flexible and powerful family of parabolic-hyperbolic partial differential equations (PDEs) that somewhat resembles the structure of the parabolic diffusion equation, but incorporates the second order derivative in time. It is instructive intuitively to consider in this context the dynamics of image processing as the deformation of an `elastic sheet'. Indeed, our parabolic-hyperbolic PDE models elastic deformation. This analogy between a well-known physical system and process on one hand, and the dynamics of an image processing scheme on the other hand, contributes interesting and important insight about images and their processing. We explore and demonstrate the capabilities and advantages afforded by the application of the proposed family of equations in image enhancement, and highlight the importance of incorporating the second derivative in time. The problem of computational complexity is addressed, and efficient numeric schemes are also presented.

The framework of blind separation of mixed images is extended to the case of mixing by a nonuniform medium. As an example, we provide a solution to blind separation of optically-distorted reflections superimposed on images taken through a curved semi-reflector. We use two images taken from slightly different locations, to remove the reflections. We consider the problem in the context of sparse blind source separation, where the sources are spatially-varying filtered prior to being mixed. The Scale Invariant Feature Transform (SIFT) is used to sparsify the images and estimate the mixing system. The sources are then estimated, using a variational approach. It uses a regularization operator, which is edge-preserving and color-channel coupled.

There is a need to restore color images that suffer from distance-dependent degradation during acquisition. This occurs, for example, when imaging through scattering media. There, signal attenuation worsens with the distance of an object from the camera. A 'naive' restoration may attempt to restore the image by amplifying the signal in each pixel according to the distance of its corresponding object. This, however, would amplify the noise in a nonuniform manner. Moreover, standard space-invariant de-noising over-blurs close by objects (which have low noise), or insufficiently smoothes distant objects (which are very noisy). We present a variational method to overcome this problem. It uses a regularization operator which is distance dependent, in addition to being edge-preserving and color-channel coupled. Minimizing this functional results in a scheme of reconstruction-while-denoising. It preserves important features, such as the texture of close by objects and edges of distant ones. A restoration algorithm is presented for reconstructing color images taken through haze. The algorithm also restores the path radiance, which is equivalent to the distance map. We demonstrate the approach experimentally.

Image denoising and enhancement problems have many physical analogues that highlight new approaches to novel solutions. One such solution, based on viewing the image as elastic sheet, is presented. A processing scheme for grayscale images is outlined and further considered in the context of color images. Preliminary analysis and simulations on noisy images indicate that multidimensional manifold representation of combined space-color information incorporates the advantages of separate color channel representations. Experimental analysis reveals elastic sheet method to be a powerful and robust denoising tool, which preserves most meaningful details.

The complex diffusion process, recently introduced in image processing and computer vision by combining the linear diffusion equation and the `free-particle' Schrodinger equation, is further generalized by incorporating the Schrodinger potential. We show that this generalized complex diffusion equation is inherently endowed with processing properties Suitable for dealing with textures in a naturally coupled manner. The schrödinger potential is self adopting to the specific properties of an image at hand in that it implements an image specific wavelet shrinkage algorithm. Results indicate that the generalized complex diffusion processing scheme not only preserves textures better than demonstrated by previously reported results, but can even enhance textures by adjusting the coefficient that determines that magnitude of the potential, relative to the potentialless complex diffusion. This is in a way analogous to the enhancement of edges by the Forward-and-Backward diffusion process.

Cortical ensembles of neurons perform a large class of tasks which require real-time processing of complex temporal inputs. These tasks involve identifying complex structures and relationships in massive quantities of low precision, ambiguous noisy data. To get some insights into neural mechanisms responsible for the performance of these tasks in a relatively large ensemble of cortical cells, we use a natural closed-loop computational framework. By incorporating functions of readout, reward and feedback, we apply such a framework of neural ensembles to classification and recognition tasks of real-time temporal data. This approach is inspired by neurobiological findings from ex-vivo multicellular electrical recordings and injection of dopamine to the neural culture. Analysis of this functional system provides a new insight into the structure of the spatio-temporal firing patterns, coined "Cliques", that are in a way the neurophysiological carriers of higher-level task-dependent functions and perceptual elements. Such cliques are generated by neural ensembles that interface with the external environment by means of complementary networks that serve the function of readout. Cliques are thus functions of both, network's state and computational task. To observe the cliques, a readout network must be incorporated into the computational system in a closed-loop manner. The emergence of cliques is mediated by spatio-temporal dynamical liquid-currents, through a selective process of Co-Evolutionary learning of two neural ensembles. Liquid-currents from presented inputs to cliques are the dynamic realizations by means of the neural ensembles. The definition of liquid-current intensity leads to a conclusion that the emergence of cliques occurs in cortical ensembles of cells regardless of the realizations at the cellular level and of the intrinsic dimensions of the problem.

This paper is concerned with finding the best partial differential equation-based denoising process, out of a set of possible ones. We focus on either finding the proper weight of the fidelity term in the energy minimization formulation or on determining the optimal stopping time of a nonlinear diffusion process. A necessary condition for achieving maximal SNR is stated, based on the covariance of the noise and the residual part. We provide two practical alternatives for estimating this condition by observing that the filtering of the image and the noise can be approximated by a decoupling technique, with respect to the weight or time parameters. Our automatic algorithm obtains quite accurate results on a variety of synthetic and natural images, including piecewise smooth and textured ones. We assume that the statistics of the noise were previously estimated. No a priori knowledge regarding the characteristics of the clean image is required. A theoretical analysis is carried out, where several SNR performance bounds are established for the optimal strategy and for a widely used method, wherein the variance of the residual part equals the variance of the noise.

Denoising algorithms based on gradient dependent regularizers, such as nonlinear diffusion processes and total variation denoising, modify images towards piecewise constant functions. Although edge sharpness and location is well preserved, important information, encoded in image features like textures or certain details, is often compromised in the process of denoising. We propose a mechanism that better preserves fine scale features in such denoising processes. A basic pyramidal structure-texture decomposition of images is presented and analyzed. A first level of this pyramid is used to isolate the noise and the relevant texture components in order to compute spatially varying constraints based on local variance measures. A variational formulation with a spatially varying fidelity term controls the extent of denoising over image regions. Our results show visual improvement as well as an increase in the signal-to-noise ratio over scalar fidelity term processes. This type of processing can be used for a variety of tasks in partial differential equation-based image processing and computer vision, and is stable and meaningful from a mathematical viewpoint.

We address the issue of textured image segmentation in the context of the Gabor feature space of images. Gabor filters tuned to a set of orientations, scales and frequencies are applied to the images to create the Gabor feature space. A two-dimensional Riemannian manifold of local features is extracted via the Beltrami framework. The metric of this surface provides a good indicator of texture changes and is used, therefore, in a Beltrami-based diffusion mechanism and in a geodesic active contours algorithm for texture segmentation. The performance of the proposed algorithm is compared with that of the edgeless active contours algorithm applied for texture segmentation. Moreover, an integrated approach, extending the geodesic and edgeless active contours approaches to texture segmentation, is presented. We show that combining boundary and region information yields more robust and accurate texture segmentation results.

We pose the problem of tissue classification in MRI as a blind source separation (BSS) problem and solve it by means of sparse component analysis (SCA). Assuming that most MR images can be sparsely represented, we consider their optimal sparse representation. Sparse components define a physically-meaningful feature space for classification. We demonstrate our approach on simulated and real multi-contrast MRI data. The proposed framework is general in that it is applicable to other modalities of medical imaging as well, whenever the linear mixing model is applicable.

The relative Newton algorithm, previously proposed for quasi-maximum likelihood blind source separation and blind deconvolution of one-dimensional signals is generalized for blind deconvolution of images. Smooth approximation of the absolute value is used as the nonlinear term for sparse sources. In addition, we propose a method of sparsification, which allows blind deconvolution of arbitrary sources, and show how to find optimal sparsifying transformations by supervised learning.

We present an efficient Newton-like algorithm for quasimaximum likelihood (QML) blind deconvolution of images. This algorithm exploits the sparse structure of the Hessian. An optimal distribution-shaping approach by means of sparsification allows one to use simple and convenient sparsity prior for processing of a wide range of natural images. Simulation results demonstrate the efficiency of the proposed method.

The linear and nonlinear scale spaces, generated by the inherently real-valued diffusion equation, are generalized to complex diffusion processes, by incorporating the free Schrödinger equation. A fundamental solution for the linear case of the complex diffusion equation is developed. Analysis of its behavior shows that the generalized diffusion process combines properties of both forward and inverse diffusion. We prove that the imaginary part is a smoothed second derivative, scaled by time, when the complex diffusion coefficient approaches the real axis. Based on this observation, we develop two examples of nonlinear complex processes, useful in image processing: a regularized shock filter for image enhancement and a ramp preserving denoising process.

We address the problem of blind source separation (BSS) of superimposed images and, in particular, consider the recovery of a scene recorded through a semireflective medium (e.g. glass windshield) from its mixture with a virtual reflected image. We extend the sparse ICA (SPICA) approach to BSS and apply it to the separation of the desired image from the superimposed images, without having any a priori knowledge about its structure and/or statistics. Advances in the SPICA approach are discussed. Simulations and experimental results illustrate the efficiency of the proposed approach, and of its specific implementation in a simple algorithm of a low computational cost. The approach and the algorithm are generic in that they can be adapted and applied to a wide range of BSS problems involving one-dimensional signals or images.

In this paper, we consider the problem of recovery of a scene recorded through a semireflective medium from its mixture with a virtual reflected image using the blind source separation (BSS) framework. We extend the sparse ICA (SPICA) approach and apply it to the separation of the desired image from the superimposed images, without having any a priory knowledge about its structure and/or statistics. Advances in the SPICA approach are discussed. Simulations and experimental results illustrate the efficiency of the proposed approach, and of its specific implementation in a simple algorithm of a low computational cost. The approach and the algorithm are generic and can be adapted and applied to a wide range of BSS problems involving one-dimensional signals or images.

The widely used denoising algorithms based on nonlinear diffusion, such as Perona-Malik and total variation denoising, modify images toward piecewise constant functions. Though edge sharpness and location is well preserved, important information, encoded in image features like textures or small details, is often lost in the process. We suggest a simple way to better preserve textures, small details, or global information. This is done by adding a spatially varying fidelity term that controls the amount of denoising in any region of the image. This form is very simple, can be used for a variety of tasks in PDE-based image processing and computer vision, and is stable and meaningful from a mathematical point of view.

It was previously shown that sparse representations can improve and simplify the estimation of an unknown mixing matrix of a set of images and thereby improve the quality of separation of source images. Here we propose a multiscale approach to the problem of blind separation of images from a set of their mixtures. We take advantage of the properties of multiscale transforms such as wavelet packets and decompose signals and images according to sets of local features. The resulting partial representations on a tree of data structure depict various degrees of sparsity. We show how the separation error is affected by the sparsity of the decomposition coefficients, and by the misfit between the prior, formulated in accordance with the probabilistic model of the coefficients distribution, and the actual distribution of the coefficients. Our error estimator, based on the Taylor expansion of the quasi log-likelihood function, is used in selection of the best subsets of coefficients, utilized in turn for further separation. The performance of the proposed method is assessed by separation of noise-free and noisy data. Experiments with simulated and real signals and images demonstrate significant improvement of separation quality over previously reported results.

Signal and image enhancement is considered in the context of a new type of diffusion process that simultaneously enhances, sharpens, and denoises images. The nonlinear diffusion coefficient is locally adjusted according to image features such as edges, textures, and moments. As such, it can switch the diffusion process from a forward to a backward (inverse) mode according to a given set of criteria. This results in a forward-and-backward (FAB) adaptive diffusion process that enhances features while locally denoising smoother segments of the signal or image. The proposed method, using the FAB process, is applied in a super-resolution scheme. The FAB method is further generalized for color processing via the Beltrami flow, by adaptively modifying the structure tensor that controls the nonlinear diffusion process. The proposed structure tensor is neither positive definite nor negative, and switches between these states according to image features. This results in a forward-and-backward diffusion flow where different regions of the image are either forward or backward diffused according to the local geometry within a neighborhood.

Two nonlinear diffusion processes with time-dependent diffusion coefficients are presented. Both processes converge to nontrivial solutions, eliminating the need to impose an arbitrary diffusion stopping time, otherwise required in the implementation of most nonlinear diffusion processes. The two schemes employ nonlinear cooling mechanisms that preserve edges. One scheme is intended for general denoising, whereas the other is targeted for enhancement or segmentation of images. Simulation results indicate that the proposed schemes provide a stable and efficient tool for processing of noisy images.

A classical technique for reconstruction of emission tomography (ET) images from measured projections is based on the maximum likelihood (ML) estimation, achieved with the expectation maximization (EM) algorithm. We incorporate the wavelet transform (WT) and total variation (TV) based penalties into the ML framework, and compare performance of the EM algorithm and the previously proposed conjugate barrier (CB) algorithm. Using the WT- and TV-based penalties allows one to embed regularization procedures into the iterative process. In the case of the WT-based penalty, we impose a subset of wavelet coefficients with a desired resolution on the objective function. It appears that the CB algorithm outperforms substantially the EM algorithm in penalized reconstruction. The properties of the optimization algorithms along with WTand TV-based regularization are demonstrated on image reconstructions of a synthetic brain phantom, and the quality of reconstruction is compared with standard methods.

The blind source separation problem is concerned with extraction of the underlying source signals from a set of their linear mixtures, where the mixing matrix is unknown. It was discovered recently, that exploiting the sparsity of sources in their representation according to some signal dictionary, dramatically improves the quality of separation. It is especially useful in image processing problems, wherein signals possess strong spatial sparsity. We use multiscale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. Experiments with 1D signals and images demonstrate significant improvement of separation quality.

We consider a problem of blind source separation from a set of instan(cid:173) taneous linear mixtures, where the mixing matrix is unknown. It was discovered recently, that exploiting the sparsity of sources in an appro(cid:173) priate representation according to some signal dictionary, dramatically improves the quality of separation. In this work we use the property of multi scale transforms, such as wavelet or wavelet packets, to decompose signals into sets of local features with various degrees of sparsity. We use this intrinsic property for selecting the best (most sparse) subsets of features for further separation. The performance of the algorithm is ver(cid:173) ified on noise-free and noisy data. Experiments with simulated signals, musical sounds and images demonstrate significant improvement of sep(cid:173) aration quality over previously reported results.

Signal and image enhancement in the presence of noise is considered in the context of the scale-space approach. A modified dynamic process, based on the action of a nonlinear diffusion equation, is presented. The diffusion coefficient is adjusted according to the local gradient, intensity and other image properties, and as such also reverses its sign, i.e. switches from a forward to a backward (inverse) diffusion process according to a given criterion. This results in enhancement of transients and singularities in the one-dimensional case, and of edges in images, while locally denoising smoother segments of the signal or image. Regularization of the ill-posed inverse diffusion problem is discussed. Examples of both one-dimensional signals and images are presented.

The Beltrami geometrical framework for scale-space flows is generalized to nontrivial color space geometries and implemented in analysis and processing of color images. We demonstrate how various models of color perception, interpreted as geometries of the color space, result in different enhanced processing schemes.

Algorithms for resolution enhancement are needed in various applications of image processing and communication such as compression, and HDTV. We develop a geometrical algorithm, based on diffusion processes which are used both for smoothing of colored images and enhancement of the colored images. The latter is accomplished by "solving" an inverse diffusion problem which is ill-posed. In order to stabilize the flow, as well as to enhance important features (e.g. edges) at the expense of less important image domains, we use a modified Beltrami diffusion equation. Results indicate that it is possible to impose the backwards flow, in spite of the instabilities, and thereby enhance the image up to a certain level of resolution which depends on the nature of the image.

The multi-window Gabor-type scheme is generalized to the multidimensional case with special emphasis on images. Sampling rate p/q along each dimension, where p and q are relatively prime integers, is considered. The problem of multidimensional matrix operations required in dual window computations gives rise to the stacking method that merges dimensions from hyperdimensional to a two-dimensional conventional matrix. Algorithms are generalized to Gabor-type representation of images and multidimensional signals that are sampled over different grids, such as the hexagonal grids.

In image analysis, processing and understanding, it is highly desirable to process the image and feature domains by methods that are specific to these domains. We show how the geometrical framework for scale-space flows is most convenient for this purpose, and demonstrate, as an example, how one can switch continuously between different processing flows of images and color domains. The parameter that interpolates between the norms is the luminance strength, taken here as a local function of the image embedding space. The resulting spatial and/or luminance preserving flow can be used for conditional denoising, enhancement and segmentation. This example demonstrates that the proposed framework can incorporate context or task dependent data, furnished by either the human user or by an active vision subsystem, in a coherent and convenient way.

A method for wavelet image representation, coding and compression, using nonseparable two-dimensional (2D) multiwavelet transform, is presented. The application of this approach results in 2D wavelet functions and filter-banks that are more suitable for coding of two-dimensional signals, such as images, than the traditional separable transform. We deal mainly with the design of multiwavelets. The nonseparable approach permits the design of 2D multiwavelets with preferred orientations. We also present the nonlinear constraints on the polyphase matrix components which must be satisfied while imposing the requirement of vanishing moments.

A new method of farthest point strategy (FPS) for progressive image acquisition-an acquisition process that enables an approximation of the whole image at each sampling stage-is presented. Its main advantage is in retaining its uniformity with the increased density, providing efficient means for sparse image sampling and display. In contrast to previously presented stochastic approaches, the FPS guarantees the uniformity in a deterministic min-max sense. Within this uniformity criterion, the sampling points are irregularly spaced, exhibiting anti-aliasing properties comparable to those characteristic of the best available method (Poisson disk). A straightforward modification of the FPS yields an image-dependent adaptive sampling scheme. An efficient O(N log N) algorithm for both versions is introduced, and several applications of the FPS are discussed.

Two-dimensional compactly supported, symmetric, orthogonal wavelets and filter-banks are presented. Two cases are discussed, filters with two-fold symmetry (centrosymmetric), and filters with four-fold symmetry that are symmetric (or anti-symmetric) about the vertical and horizontal axes. We show that imposing the requirement of symmetry (linear phase) in the case of factorable wavelets and filter-banks, imposes a simple constraint on each of its polynomial degree-1 factors. We thus obtain a simple and complete method of constructing orthogonal factorable filter-banks with linear phase. This method is exemplified by design in the case of four-band separable sampling. An interesting result is obtained, similar to the one well known in the case of 1D, orthogonal factorable wavelets can not be both continuous and have four-fold symmetry.

The departments of electrical engineering and computer science at Technion offer a broad spectrum of courses in digital and optical signals and imaging systems, and a research environment most suitable for specialization in image system engineering. We describe a few of the courses and discuss some issues related to their teaching.

Images are decomposed by the visual systems into several spatially oriented frequency channels. Such a decomposition presented in the context of multiresolution analysis, provides a powerful tool for image processing. This study is devoted to multiresolution representation by partial information of the wavelet transform. A theorem defining the conditions necessary for the zero crossing representation to be stable and complete, is presented. This theorem is based on the properties of the multiresolution theory, using orthonormal wavelet functions. An iterative fast reconstruction algorithm, which is easy to implement, is presented and illustrated by examples.

The discrete (finite) Gabor scheme is generalized by incorporating multi-windows. Two approaches are presented for the analysis of the generalized scheme: the signal domain approach and Zak transform domain approach. These approaches are based on representing the frame operator as a matrix-valued function, and are far less demanding from a computational complexity viewpoint than a straightforward matrix algebra in various operations such as the computation of the dual frame. Issues related to undersampling, critical sampling and oversampling are considered. Examples illustrating the advantages of the multi-window scheme over the single-window scheme are presented.

Two-dimensional (2-D) gradient estimators are some of the most useful tools in image processing. A computational procedure for the extension of one-dimensional (1-D) gradient estimators to two dimensions (2-D) is presented. The procedure is equivalent to the surface fitting method. It is, however, simpler in design, as the design is 1-D rather than 2-D. Higher order derivative estimators can also be constructed by the same procedure.

The representation of a random process by a set of uncorrelated random variables is examined. The main result indicates that a basis decorrelates a random process if and only if it satisfies an integral equation similar to the type satisfied by the Karhunen-Loeve expansion, but by relaxing the requirement of orthogonality of the representation functions.

This paper is concerned with image representation by data distributed nonuniformly and in particular with a representation scheme suitable for "area-of-interest" imaging. The class of signals under consideration, whose information density varies with position, can be represented according to a nonuniform sampling scheme. Position-varying projection operators are presented as simple low-pass filtering operations in a Fourier-like domain. Sequential projections are used for pyramidal representation of nonuniformly sampled images. It is shown that irregular random sampling, prevents, under certain mild restrictions, aliasing effects.

A novel approach to signal representation by partial Fourier phase is given. It is shown that an amplitude-dependent bit allocation method preserves most of the signal's structures. Improved reconstruction algorithms are illustrated using imprecise phase and some of the signal's statistical characteristics. A recent complementary study provides a classification algorithm based on partial phase information, which is invariant to 2D similarity deformations (scale, rotation and shift).

A new method of "farthest point strategy" (FPS) for progressive image acquisition is presented. Its main advantage is in retaining its uniformity with the increased density, providing an efficient means for sparse image sampling and display. In contrast to previously presented stochastic approaches, the FPS guarantees the uniformity in a deterministic min-max sense. Within this uniformity criterion, the sampling points are irregularly spaced, exhibiting superior antialiasing properties. A straightforward modification of the FPS yields an image-dependent adaptive sampling scheme. An efficient, O(N log(N)), algorithm for both versions is introduced, and several applications of the FPS are discussed.

A hidden Markov model, which describes the evolution of a (binary) edge-image along the resolution axis, is presented. The model integrates two layers: A hidden layer consists of sources having the ability of "breeding" along the resolution axis according to a Markovian rule. A second layer consists of a Gibbs random field which is defined by all the sources. The available image is a realization of this field. After fitting such a model to a given pyramid, it is possible to estimate the super-resolution images by synthesizing additional levels of the process which created the pyramid. The hidden Markov model is found to be a useful tool, allowing us to incorporate selected properties in the process of evolution along the resolution axis, while simultaneously providing an interpretation of this process. The properties incorporated into the model significantly influence the super-resolution image.

Traditional colour image enhancement techniques, have ignored the design of the human visual system and have treated colour images as three independent grayscale images. Contrary to this the authors first discuss an opponent colour model of the early visual system and demonstrate it's use in spatial enhancement of both image brightness and chroma. The opponent transform is calculated such that cone redundancies are reduced and information compressed, similar to the way it is performed in the visual system. This results in an achromatic and two opponent chromatic channels. Laplacian based edge enhancement is then performed independently on each channel. The spatial enhancement of chroma highlights small colour detail and enhances colour edges thus increasing image detail in areas where achromatic enhancement has no effect. Due to spatial limitations of the visual system, the best enhanced image is obtained when both achromatic and chromatic edge enhancement are used together.

A complete and unique parametrization is given for all compactly supported, orthogonal, two dimensional wavelet functions for the case of the quincunx decimation matrix. Once we have a complete parametrization we search for optimal wavelet functions. Here we concentrate on the task of image representation and compression. We suggest three different criterions for measuring optimality. The first is tailored for each specific image, the second measures optimality in a more statistical sense and the third uses the general quality of vanishing moments. We compare between the three criterions which seem at first very different in nature, and show that they give very similar results.

Properties of basis functions which constitute a finite scheme of discrete Gabor representation are investigated. The approach is based on the concept of frames and utilizes the Piecewise Finite Zak Transform (PFZT). The frame operator associated with the Gabor-type frame is examined by representing it as a matrix-values function in the PFZT domain. The frame property of the Gabor representation functions are examined in relation to the properties of the matrix-valued function. The frame bounds are calculated by means of the eignevalues of the matrix-valued function, and the dual frame, which is used in calculation of the expansion coefficients, is expressed by means of the inverse matrix. DFT-based algorithms for computation of the expansion coefficients, and for the reconstruction of signals from these coefficients are generalized for the case of oversampling of the Gabor space. It is illustrated by an example that a better reconstruction is obtained in from the same number of coefficients in the case of oversampling.

A method for calculating the coefficients of the Gabor expansion in the context of oversampling is presented. The method is based on the concept of frames and utilizes the Zak transform. The Zak transform further highlights the meaning and importance of frames in the context of oversampling and other aspects of signal representation by means of nonorthogonal bases.>

A method for image analysis, based on the formalism of functional analysis is proposed. This approach, which is employed for pattern recognition in the combined position-orientation space, is motivated by a variety of neurophysiological findings that emphasize the importance of the orientation feature in visual image processing and analysis. The approach is applied to the so-called Glass patterns, which are special cases of images characterized uniquely by their orientational feature of local correlations between pairs of corresponding dots. The approach can be of interest in the analysis of optical flow.>

A method for the analysis of deformed patterns is presented and analyzed. The image is transformed into a new set of coordinates in which the deformation has a particular simple form. A number of deformations are considered. The practical implementation of the method is discussed. Similar aspects of biological vision are also considered.>

A mathematical approach to image analysis and synthesis in the combined frequency-position space is presented. The formalism is based on the Zak (1967) transform, which provides one of the most fundamental tools for studies of nonstationary processes and images as such. Algorithms suitable for computation of Gabor (1946) expansion coefficients are presented and compared with regard to stability and computational complexity.>

A method for the recognition of deformed patterns is presented. It is is shown that for planar deformations generated by Lie transformation groups, there exist kernels such that the associated integral transforms are invariant under the deformations. Many examples of deformations and the corresponding kernels are given. It is possible to extend the results to higher dimensions.>

A mathematical approach to image analysis and synthesis in the combined frequency- position space is presented. The formalism is based on the Weil map - Zak transform, which provides one of the most fundamental tools for studies of nonstationary processes and images as such. Algorithms suitable for computation of Gabor expansion coefficients are presented.

The Gram determinant technique is applied to signal representation by non-orthogonal bases. A special case of image representation in biological and machine vision using Gabor elementary functions (GEFs) is considered. It is shown that, in general, the Gram determinant is a better approach to computation of the expansion coefficients than the one using bi-orthonormal auxiliary functions. An optimal representation by finite sets of coefficients is attained without a significant computational effort and the resultant reconstruction error converges monotonically with the addition of basis’ components to the reconstruction set. The Gram approach appears to be in a better accord with biological findings, regarding information processing along the visual pathway, compared to the conventional bi-orthogonal scheme.

Both biological and man-made (usually computer-based) vision systems have to acquire visual data and process it in real time at a formidable bit rate. To cope with this fundamental problem, biological systems of higher species employ mechanisms of data reduction already at the level of visual data acquisition.

A methodology is presented for analysis of wire patterns reduced from scenes containing cuboid objects. The 2D wire-pattern graph is partitioned into subgraphs representing individual cuboids. These are in turn mapped onto the log-complex plane to determine by a simple procedure the 3D orientation of each object. The transform from the polar to the log-complex plane is based on a model of the retinotopic mapping which takes place at the level of the visual cortex, which achieves size-and-rotation invariance.>

The basic biological principles of nonuniform image representation in the combined frequency-position space are utilized in an approach to image representation and processing. A scheme of image decomposition into optimal localized frequency operators, the so-called Gabor elementary functions which minimize the uncertainty inherent in image representation, is accordingly proposed. Applications to size-invariance and object-separable pattern analysis, localized texture discrimination, and representation by partial information are discussed, illustrating the advantages of the Gabor transform.>

A scheme suitable for visual information representation in a combined frequency-position space is investigated through image decomposition into a finite set of two-dimensional Gabor elementary functions (GEF). The scheme is generalized to account for the position-dependent Gabor-sampling rate, oversampling, logarithmic frequency scaling and phase-quantization characteristic of the visual system. Comparison of reconstructed signal highlights the advantages of the generalized Gabor scheme in coding typical bandlimited images. It is shown that there exists a tradeoff between the number of frequency components used per position and the number of such clusters (sampling rate) utilized along the spatial coordinate.>

Neural encoders translate information on the time-varying intensity of stimuli into sequences of membrane depolarization spikes. Their output can be considered the realization of a stochastic point process, the overall encoder behaviour being characterized through ensemble-averaged responses to identical stimuli and environmental conditions. A new mathematical model for the coding process is presented and analyzed. The model is an integrate and fire-at-threshold scheme, the stochastic features of its response resulting from random fluctuations in the firing threshold. As a consequence of feedback self-inhibition and threshold control, which is assumed to account for adaptive neutral responses, the model output is a self-exciting point process. An approximate description of the averaged encoder response is obtained by considering an ensemble of identical coding units as a whole, instead of concentrating on output sample-path evolution. This approach overcomes the difficulty inherent in analysing the global behaviour of self-exciting point processes. A conceptual decoding scheme implementing a coding unit in a feedback configuration is also introduced and discussed.

Extensive processing and reorganization of information are required as retinally generated signals flow centrally. Insight into some principles of the reorganization and transformations that occur in the early stages of the visual pathway, prior to the higher stages of pattern recognition, is provided through an analysis of receptive fields and cell counts. The rate of information flow is reduced approximately 100-fold by a retinal position-dependent (inhomogeneous) spatial sampling scheme. This principle of specialization, or nonuniform processing, is further elaborated in the retino-cortical mapping. In the central fovea, where the retinal spatial sampling rate is the highest and processing function is uniform, there are about 4000 striate cortical neurons receiving information from each cone. This number, which provides a measure of function multiplicity, drops to about 200 over the range of the near periphery. In the peripheral field beyond eight degrees, where information is compressed at the retina, the functional multiplicity stays approximately constant at 200. As it seems that no two cells perform exactly the same operation, it is concluded that the striate cortex performs many simultaneous functional mappings.

IN RECENT YEARS, experimental and theoretical progress has rekindled interest in the study of organizational and computational principles underlying neural systems. Neural systems are both complex and intriguing, and the relation of neural and sensory information processing to communications, control, and behavior in living organisms continues to pose fundamental research issues in a variety of disciplines. This Special Issue on Neural and Sensory Information Processing is intended to provide a timely sampling of current views and progress in these studies.