Solving the eigenvalue problem for differential operators is a common problem in many scientific fields. Classical numerical methods rely on intricate domain discretization and yield nonanalytic or nonsmooth approximations. We introduce a novel neural network-based solver for the eigenvalue problem of differential self-adjoint operators, where the eigenpairs are learned in an unsupervised end-to-end fashion. We propose several training procedures for solving increasingly challenging tasks toward the general eigenvalue problem. The proposed solver is capable of finding the M smallest eigenpairs for a general differential operator. We demonstrate the method on the Laplacian operator, which is of particular interest in image processing, computer vision, and shape analysis among many other applications. In addition, we solve the Legendre differential equation. Our proposed method simultaneously solves several eigenpairs and can be easily used on free-form domains. We exemplify it on L-shape and circular cut domains. A significant contribution of this work is an analysis of the numerical error of this method. In particular an upper bound for the (unknown) solution error is given in terms of the (measured) truncation error of the partial differential equation and the network structure.

We introduce a novel neural network-based PDEs solver for forward and inverse problems. The solver is grid free, mesh free, and shape free, and the solution is approximated by a neural network. We employ an unsupervised approach such that the input to the network is a point set in an arbitrary domain, and the output is the set of the corresponding function values. The network is trained to minimize deviations of the learned function from the PDE solution and satisfy the boundary conditions. The resulting solution in turn is an explicit, smooth, differentiable function with a known analytical form. We solve the forward problem (observations given the underlying model's parameters), semi-inverse problem (model's parameters given the observations in the whole domain), and full tomography inverse problem (model's parameters given the observations on the boundary) by solving the forward and semi-inverse problems at the same time. The optimized loss function consists of few elements: fidelity term of $L_2$ norm that enforces the PDE in the weak sense, an $L_\infty$ norm term that enforces pointwise fidelity and thus promotes a strong solution, and boundary and initial conditions constraints. It further accommodates regularizers for the solution and/or the model's parameters of the differential operator. This setting is flexible in the sense that regularizers can be tailored to specific problems. We demonstrate our method on several free shape two dimensional (2D) second order systems with application to electrical impedance tomography (EIT) and diffusion equation. Unlike other numerical methods such as finite differences and finite elements, the derivatives of the desired function can be analytically calculated to any order. This framework enables, in principle, the solution of high order and high dimensional nonlinear PDEs.

Sparse representation theory has been increasingly used in signal processing and machine learning. In this paper we introduce a hierarchical sparse modeling approach which integrates information from the image patch level to derive a mid-level invariant image and pattern representation. The proposed framework is based on a hierarchical architecture of dictionary learning for sparse coding in a cortical (log-polar) space, combined with a novel pooling operator which incorporates the Rapid transform and max pooling to attain rotation and scale invariance. The invariant sparse representation of patterns here presented - can be used in different object recognition tasks. Promising results are obtained for three applications - 2D shapes classification, texture recognition and object detection.

Sparse representation theory has been increasingly used in the fields of signal processing and machine learning. The standard sparse models are not invariant to spatial transformations such as image rotations, and the representation is very sensitive even under small such distortions. Most studies addressing this problem proposed algorithms which either use transformed data as part of the training set, or are invariant or robust only under minor transformations. In this paper we suggest a framework which extracts sparse features invariant under significant rotations and scalings. The algorithm is based on a hierarchical architecture of dictionary learning for sparse coding in a cortical (log-polar) space. The proposed model is tested in supervised classification applications and proved to be robust under transformed data.

Many problems in image processing are addressed via the minimization of a cost functional. The most prominently used optimization technique is gradient-descent, often used due to its simplicity and applicability where other techniques, e.g., those coming from discrete optimization, cannot be applied. Yet, gradient-descent suffers from slow convergence, and often to just local minima which highly depend on the initialization and the condition number of the functional Hessian. Newton-type methods, on the other hand, are known to have a faster, quadratic convergence. In its classical form, the Newton method relies on the $L^2$-type norm to define the descent direction. In this paper, we generalize and reformulate this very important optimization method by introducing Newton-type methods based on more general norms. Such norms are introduced both in the descent computation (Newton step) and in the corresponding stabilizing trust-region. This generalization opens up new possibilities in the extraction of the Newton step, including benefits such as mathematical stability and the incorporation of smoothness constraints. We first present the derivation of the modified Newton step in the calculus of variation framework needed for image processing. Then, we demonstrate the method with two common objective functionals: variational image deblurring and geometric active contours for image segmentation. We show that in addition to the fast convergence, norms adapted to the problem at hand yield different and superior results.

Many problems in image processing are solved via the minimization of a cost functional. The most widely used optimization technique is the gradient descent, often used due to its simplicity and applicability where other optimization techniques, e.g., those coming from discrete optimization, can not be used. Yet, gradient descent suffers from a slow convergence, and often to just local minima which highly depends on the condition number of the functional Hessian. Newton- type methods, on the other hand, are known to have a rapid (quadratic) convergence. In its classical form, the Newton method relies on the L2-type norm to define the descent direction. In this paper, we generalize and reformulate this very important optimization method by introducing a novel Newton method based on general norms. This generalization opens up new possibilities in the extraction of the Newton step, including benefits such as mathematical stability and smoothness constraints. We first present the derivation of the modified Newton step in the calculus of variation framework. Then we demonstrate the method with two common objective functionals: variational image deblurring and geodesic active contours. We show that in addition to the fast convergence, different selections norm yield different and superior results.

The problem of motion estimation and restoration of objects in a blurred video sequence is addressed in this paper. Fast movement of the objects, together with the aperture time of the camera, result in a motion-blurred image. The direct velocity estimation from this blurred video is inaccurate. On the other hand, an accurate estimation of the velocity of the moving objects is critical for restoration of motion-blurred video. Therefore, restoration needs accurate motion estimation and vice versa, and a joint process is called for. To address this problem we derive a novel model of the blurring process and propose a Mumford-Shah type of variational framework, acting on consecutive frames, for joint object deblurring and velocity estimation. The proposed procedure distinguishes between the moving object and the background and is accurate also close to the boundary of the moving object. Experimental results both on simulated and real data show the importance of this joint estimation and its superior performance when compared to the independent estimation of motion and restoration.

We consider the problem of restoring a multichannel image corrupted by blur and impulsive noise (e.g., salt-and-pepper noise). Using the variational framework, we consider the L1 fidelity term and several possible regularizers. In particular, we use generalizations of the Mumford-Shah (MS) functional to color images and gamma-convergence approximations to unify deblurring and denoising. Experimental comparisons show that the MS stabilizer yields better results with respect to Beltrami and total variation regularizers. Color edge detection is a beneficial by-product of our methods.

Image restoration and segmentation are both classical problems, that are known to be difficult and have attracted major research efforts. This paper shows that the two problems are tightly coupled and can be successfully solved together. Mutual support of image restoration and segmentation processes within a joint variational framework is theoretically motivated, and validated by successful experimental results. The proposed variational method integrates semi-blind image deconvolution (parametric blur-kernel), and Mumford-Shah segmentation. The functional is formulated using the T-convergence approximation and is iteratively optimized via the alternate minimization method. While the major novelty of this work is in the unified treatment of the semi-blind restoration and segmentation problems, the important special case of known blur is also considered and promising results are obtained.