Minimization of boundary curvature is a classic regularization technique for image segmentation in the presence of noisy image data. Techniques for minimizing curvature have historically been derived from gradient descent methods which could be trapped by a local minimum and, therefore, required a good initialization. Recently, combinatorial optimization techniques have overcome this barrier by providing solutions that can achieve a global optimum. However, curvature regularization methods can fail when the true object has high curvature. In these circumstances, existing methods depend on a data term to overcome the high curvature of the object. Unfortunately, the data term may be ambiguous in some images, which causes these methods also to fail. To overcome these problems, we propose a contrast driven elastica model (including curvature), which can accommodate high curvature objects and an ambiguous data model. We demonstrate that we can accurately segment extremely challenging synthetic and real images with ambiguous data discrimination, poor boundary contrast, and sharp corners. We provide a quantitative evaluation of our segmentation approach when applied to a standard image segmentation data set.

Patient-specific blood flow modeling combining imaging data and computational fluid dynamics can aid in the assessment of coronary artery disease. Accurate coronary segmentation and realistic physiologic modeling of boundary conditions are important steps to ensure a high diagnostic performance. Segmentation of the coronary arteries can be constructed by a combination of automated algorithms with human review and editing. However, blood pressure and flow are not impacted equally by different local sections of the coronary artery tree. Focusing human review and editing towards regions that will most affect the subsequent simulations can significantly accelerate the review process. We define geometric sensitivity as the standard deviation in hemodynamics-derived metrics due to uncertainty in lumen segmentation. We develop a machine learning framework for estimating the geometric sensitivity in real time. Features used include geometric and clinical variables, and reduced-order models. We develop an anisotropic kernel regression method for assessment of lumen narrowing score, which is used as a feature in the machine learning algorithm. A multi-resolution sensitivity algorithm is introduced to hierarchically refine regions of high sensitivity so that we can quantify sensitivities to a desired spatial resolution. We show that the mean absolute error of the machine learning algorithm compared to 3D simulations is less than 0.01. We further demonstrate that sensitivity is not predicted simply by anatomic reduction but also encodes information about hemodynamics which in turn depends on downstream boundary conditions. This sensitivity approach can be extended to other systems such as cerebral flow, electro-mechanical simulations, etc.

This paper describes a method of finding thin, elongated structures in images and volumes. We use shortest paths to minimize very general functionals of higher-order curve properties, such as curvature and torsion. Our globally optimal method uses line graphs and its runtime is polynomial in the size of the discretization, often in the order of seconds on a single computer. To our knowledge, we are the first to perform experiments in three dimensions with curvature and torsion regularization. The largest graphs we process have almost one hundred billion arcs. Experiments on medical images and in multi-view reconstruction show the significance and practical usefulness of regularization based on curvature while torsion is still only tractable for small-scale problems.

Existing methods for surface matching are limited by the tradeoff between precision and computational efficiency. Here, we present an improved algorithm for dense vertex-to-vertex correspondence that uses direct matching of features defined on a surface and improves it by using spectral correspondence as a regularization. This algorithm has the speed of both feature matching and spectral matching while exhibiting greatly improved precision (distance errors of 1.4 percent). The method, FOCUSR, incorporates implicitly such additional features to calculate the correspondence and relies on the smoothness of the lowest-frequency harmonics of a graph Laplacian to spatially regularize the features. In its simplest form, FOCUSR is an improved spectral correspondence method that nonrigidly deforms spectral embeddings. We provide here a full realization of spectral correspondence where virtually any feature can be used as an additional information using weights on graph edges, but also on graph nodes and as extra embedded coordinates. As an example, the full power of FOCUSR is demonstrated in a real-case scenario with the challenging task of brain surface matching across several individuals. Our results show that combining features and regularizing them in a spectral embedding greatly improves the matching precision (to a submillimeter level) while performing at much greater speed than existing methods.

Algorithms based on total variation (TV) minimization are prevalent in image processing. They play a key role in a variety of applications such as image denoising, compressive sensing, and inverse problems in general. In this work, we extend the TV dual framework that includes Chambolle's and Gilboa and Osher's projection algorithms for TV minimization. We use a flexible graph data representation that allows us to generalize the constraint on the projection variable. We show how this new formulation of the TV problem may be solved by means of fast parallel proximal algorithms. In denoising and deblurring examples, the proposed approach is shown not only to perform better than recent TV-based approaches, but also to perform well on arbitrary graphs instead of regular grids. The proposed method consequently applies to a variety of other inverse problems including image fusion and mesh filtering.

The amount of calibration data needed to produce images of adequate quality can prevent auto-calibrating parallel imaging reconstruction methods like generalized autocalibrating partially parallel acquisitions (GRAPPA) from achieving a high total acceleration factor. To improve the quality of calibration when the number of auto-calibration signal (ACS) lines is restricted, we propose a sparsity-promoting regularized calibration method that finds a GRAPPA kernel consistent with the ACS fit equations that yields jointly sparse reconstructed coil channel images. Several experiments evaluate the performance of the proposed method relative to unregularized and existing regularized calibration methods for both low-quality and underdetermined fits from the ACS lines. These experiments demonstrate that the proposed method, like other regularization methods, is capable of mitigating noise amplification, and in addition, the proposed method is particularly effective at minimizing coherent aliasing artifacts caused by poor kernel calibration in real data. Using the proposed method, we can increase the total achievable acceleration while reducing degradation of the reconstructed image better than existing regularized calibration methods.

problem using Random Walker (RW) segmentation as the core segmentation algorithm, rather than the traditional MRF approach adopted in the literature so far. Our formulation is similar to previous approaches in the sense that it also permits Cosegmentation constraints (which impose consistency between the extracted objects from ≥ 2 images) using a nonparametric model. However, several previous nonparametric cosegmentation methods have the serious limitation that they require adding one auxiliary node (or variable) for every pair of pixels that are similar (which effectively limits such methods to describing only those objects that have high entropy appearance models). In contrast, our proposed model completely eliminates this restrictive dependence -the resulting improvements are quite significant. Our model further allows an optimization scheme exploiting quasiconvexity for model-based segmentation with no dependence on the scale of the segmented foreground. Finally, we show that the optimization can be expressed in terms of linear algebra operations on sparse matrices which are easily mapped to GPU architecture. We provide a highly specialized CUDA library for Cosegmentation exploiting this special structure, and report experimental results showing these advantages.

Algorithms based on the minimization of the Total Variation are prevalent in computer vision. They are used in a variety of applications such as image denoising, compressive sensing and inverse problems in general. In this work, we extend the TV dual framework that includes Chambolle's and Gilboa Osher's projection algorithms for TV minimization in a flexible graph data representation by generalizing the constraint on the projection variable. We show how this new formulation of the TV problem may be solved by means of a fast parallel proximal algorithm, which performs better than the classical TV approach for denoising, and is also applicable to inverse problems such as image deblurring.

Both compressed sensing (CS) and parallel imaging effectively reconstruct magnetic resonance images from undersampled data. Combining both methods enables imaging with greater undersampling than accomplished previously. This paper investigates the choice of a suitable sampling pattern to accommodate both CS and parallel imaging. A combined method named SpRING is described and extended to handle random undersampling, and both GRAPPA and SpRING are evaluated for uniform and random undersampling using both simulated and real data. For the simulated data, when the undersampling factor is large, SpRING performs better with random undersampling. However, random undersampling is not as beneficial to SpRING for real data with approximate sparsity.

Drawing a box around an intended segmentation target has become both a popular user interface and a common output for learning-driven detection algorithms. Despite the ubiquity of using a box to define a segmentation target, it is unclear in the literature whether a box is sufficient to define a unique segmentation or whether segmentation from a box is ill-posed without higher-level (semantic) knowledge of the intended target. We examine this issue by conducting a study of 14 subjects who are asked to segment a boxed target in a set of 50 real images for which they have no semantic attachment. We find that the subjects do indeed perceive and trace almost the same segmentations as each other, despite the inhomogeneity of the image intensities, irregular shapes of the segmentation targets and weakness of the target boundaries. Since the subjects produce the same segmentation, we conclude that the problem is well-posed and then provide a new segmentation algorithm from a box which achieves results close to the perceived target.

In this work, we extend a common framework for graph-based image segmentation that includes the graph cuts, random walker, and shortest path optimization algorithms. Viewing an image as a weighted graph, these algorithms can be expressed by means of a common energy function with differing choices of a parameter q acting as an exponent on the differences between neighboring nodes. Introducing a new parameter p that fixes a power for the edge weights allows us to also include the optimal spanning forest algorithm for watershed in this same framework. We then propose a new family of segmentation algorithms that fixes p to produce an optimal spanning forest but varies the power q beyond the usual watershed algorithm, which we term the power watershed. In particular, when q=2, the power watershed leads to a multilabel, scale and contrast invariant, unique global optimum obtained in practice in quasi-linear time. Placing the watershed algorithm in this energy minimization framework also opens new possibilities for using unary terms in traditional watershed segmentation and using watershed to optimize more general models of use in applications beyond image segmentation.

Maximum flow (and minimum cut) algorithms have had a strong impact on computer vision. In particular, graph cut algorithms provide a mechanism for the discrete optimization of an energy functional which has been used in a variety of applications such as image segmentation, stereo, image stitching, and texture synthesis. Algorithms based on the classical formulation of max-flow defined on a graph are known to exhibit metrication artifacts in the solution. Therefore, a recent trend has been to instead employ a spatially continuous maximum flow (or the dual min-cut problem) in these same applications to produce solutions with no metrication errors. However, known fast continuous max-flow algorithms have no stopping criteria or have not been proved to converge. In this work, we revisit the continuous max-flow problem and show that the analogous discrete formulation is different from the classical max-flow problem. We then apply an appropriate combinatorial optimization technique to this combinatorial continuous max-flow (CCMF) problem to find a null-divergence solution that exhibits no metrication artifacts and may be solved exactly by a fast, efficient algorithm with provable convergence. Finally, by exhibiting the dual problem of our CCMF formulation, we clarify the fact, already proved by Nozawa in the continuous setting, that the max-flow and the total variation problems are not always equivalent.

Two challenges in computer vision are to accommodate noisy data and missing data. Many problems in computer vision, such as segmentation, filtering, stereo, reconstruction, inpainting and optical flow seek solutions that match the data while satisfying an additional regularization, such as total variation or boundary length. A regularization which has received less attention is to minimize the curvature of the solution. One reason why this regularization has received less attention is due to the difficulty in finding an optimal solution to this image model, since many existing methods are complicated, slow and/or provide a suboptimal solution. Following the recent progress of Schoenemann et al., we provide a simple formulation of curvature regularization which admits a fast optimization which gives globally optimal solutions in practice. We demonstrate the effectiveness of this method by applying this curvature regularization to image segmentation.

Many computer vision applications such as image filtering, segmentation and stereo-vision can be formulated as optimization problems. Whereas in previous decades continuous-domain, iterative procedures were common, recently discrete, convex, globally optimal methods have received a lot of attention. However not all problems in computer vision are convex, for instance L0norm optimization such as seen in compressive sensing. Recently, a novel discrete framework encompassing many known segmentation methods was proposed: power watershed. We are interested to explore the possibilities of this minimizer to solve other problems than segmentation, in particular with respect to unusual norms optimization. In this article we reformulate the problem of anisotropic diffusion as an L0optimization problem, and we show that power watersheds are able to optimize this energy quickly and effectively. This study paves the way for using the power watershed as a useful general-purpose minimizer in many different computer vision contexts.

Shortest paths have been used to segment object boundaries with both continuous and discrete image models. Although these techniques are well defined in 2D, the character of the path as an object boundary is not preserved in 3D. An object boundary in three dimensions is a 2D surface. However, many different extensions of the shortest path techniques to 3D have been previously proposed in which the 3D object is segmented via a collection of shortest paths rather than a minimal surface, leading to a solution which bears an uncertain relationship to the true minimal surface. Specifically, there is no guarantee that a minimal path between points on two closed contours will lie on the minimal surface joining these contours. We observe that an elegant solution to the computation of a minimal surface on a cellular complex (e.g., a 3D lattice) was given by Sullivan [47]. Sullivan showed that the discrete minimal surface connecting one or more closed contours may be found efficiently by solving a Minimum-cost Circulation Network Flow (MCNF) problem. In this work, we detail why a minimal surface properly extends a shortest path (in the context of a boundary) to three dimensions, present Sullivan's solution to this minimal surface problem via an MCNF calculation, and demonstrate the use of these minimal surfaces on the segmentation of image data.

Interactive image segmentation traditionally involves the use of algorithms such as graph cuts or random walker. Common concerns with using graph cuts are metrication artifacts (blockiness) and the shrinking bias (bias towards shorter boundaries). The random walker avoids these problems, but suffers from the proximity bias (sensitivity to location of pixels labeled by the user). In this work, we introduce a new family of segmentation algorithms that includes graph cuts and random walker as special cases. We explore image segmentation using continuous-valued Markov random fields (MRFs) with probability distributions following the p-norm of the difference between configurations of neighboring sites. For p=1 these MRFs may be interpreted as the standard binary MRF used by graph cuts, while for p=2 these MRFs may be viewed as Gaussian MRFs employed by the random walker algorithm. By allowing the probability distribution for neighboring sites to take any arbitrary p-norm (p ≥ 1), we pave the path for hybrid extensions of these algorithms. Experiments show that the use of a fractional p (1 <; p <; 2) can be used to resolve the aforementioned drawbacks of these algorithms.

In this work, we extend a common framework for seeded image segmentation that includes the graph cuts, random walker, and shortest path optimization algorithms. Viewing an image as a weighted graph, these algorithms can be expressed by means of a common energy function with differing choices of a parameter q acting as an exponent on the differences between neighboring nodes. Introducing a new parameter p that fixes a power for the edge weights allows us to also include the optimal spanning forest algorithm for watersheds in this same framework. We then propose a new family of segmentation algorithms that fixes p to produce an optimal spanning forest but varies the power q beyond the usual watershed algorithm, which we term power watersheds. Placing the watershed algorithm in this energy minimization framework also opens new possibilities for using unary terms in traditional watershed segmentation and using watersheds to optimize more general models of use in application beyond image segmentation.

The Mumford-Shah functional has had a major impact on a variety of image analysis problems, including image segmentation and filtering, and, despite being introduced over two decades ago, it is still in widespread use. Present day optimization of the Mumford-Shah functional is predominated by active contour methods. Until recently, these formulations necessitated optimization of the contour by evolving via gradient descent, which is known for its overdependence on initialization and the tendency to produce undesirable local minima. In order to reduce these problems, we reformulate the corresponding Mumford-Shah functional on an arbitrary graph and apply the techniques of combinatorial optimization to produce a fast, low-energy solution. In contrast to traditional optimization methods, use of these combinatorial techniques necessitates consideration of the reconstructed image outside of its usual boundary, requiring additionally the inclusion of regularization for generating these values. The energy of the solution provided by this graph formulation is compared with the energy of the solution computed via traditional gradient descent-based narrow-band level set methods. This comparison demonstrates that our graph formulation and optimization produces lower energy solutions than the traditional gradient descent based contour evolution methods in significantly less time. Finally, we demonstrate the usefulness of the graph formulation to apply the Mumford-Shah functional to new applications such as point clustering and filtering of nonuniformly sampled images.

Interactive segmentation is often performed on images that have been stored on disk (e.g., a medical image server) for some time prior to user interaction. We propose to use this time to perform an offline precomputation of the segmentation prior to user interaction that significantly decreases the amount of user time necessary to produce a segmentation. Knowing how to effectively precompute the segmentation prior to user interaction is difficult, since a user may choose to guide the segmentation algorithm to segment any object (or multiple objects) in the image. Consequently, precomputation performed prior to user interaction must be performed without any knowledge of the user interaction. Specifically, we show that one may precompute several eigenvectors of the weighted Laplacian matrix of a graph and use this information to produce a linear-time approximation of the Random Walker segmentation algorithm, even without knowing where the foreground/background seeds will be placed. Finally, we also show that this procedure may be interpreted as a seeded (interactive) Normalized Cuts algorithm.

We propose a scheme to introduce directionality in the random walker algorithm for image segmentation. In particular, we extend the optimization framework of this algorithm to combinatorial graphs with directed edges. Our scheme is interactive and requires the user to label a few pixels that are representative of a foreground object and of the background. These labeled pixels are used to learn intensity models for the object and the background, which allow us to automatically set the weights of the directed edges. These weights are chosen so that they bias the direction of the object boundary gradients to flow from regions that agree well with the learned object intensity model to regions that do not agree well. We use these weights to define an energy function that associates asymmetric quadratic penalties with the edges in the graph. We show that this energy function is convex, hence it has a unique minimizer. We propose a provably convergent iterative algorithm for minimizing this energy function. We also describe the construction of an equivalent electrical network with diodes and resistors that solves the same segmentation problem as our framework. Finally, our experiments on a database of 69 images show that the use of directional information does improve the segmenting power of the random Walker algorithm.

In this work, we present a common framework for seeded image segmentation algorithms that yields two of the leading methods as special cases - The Graph Cuts and the Random Walker algorithms. The formulation of this common framework naturally suggests a new, third, algorithm that we develop here. Specifically, the former algorithms may be shown to minimize a certain energy with respect to either an 𝓁1or an 𝓁2norm. Here, we explore the segmentation algorithm defined by an 𝓁∞norm, provide a method for the optimization and show that the resulting algorithm produces an accurate segmentation that demonstrates greater stability with respect to the number of seeds employed than either the Graph Cuts or Random Walker methods.

We present a new approach for the incorporation of shape information into a segmentation algorithm. Unlike previous approaches to the problem, our method requires no initialization, is non-iterative and finds a steady-state (i.e., global optimum) solution. In the present work, we are specifically focused on the segmentation of rectilinear shapes. The key idea is to use the fact that certain shape classes optimize the ratio of specific metrics, which can be expressed as graph Laplacian matrices applied to indicator vectors. We show that a relaxation of the binary formulation of this problem allows a global solution via generalized eigenvectors. The approach is tested on both synthetic examples and natural images.

Shortest path algorithms on weighted graphs have found widespread use in the computer vision literature. Although a shortest path may be found in a 3D weighted graph, the character of the path as an object boundary in 2D is not preserved in 3D. An object boundary in three dimensions is a (2D) surface. Therefore, a discrete minimal surface computation is necessary to extend shortest path approaches to 3D data in applications where the character of the path as a boundary is important. This minimal surface problem finds natural application in the extension of the intelligent scissors/ live wire segmentation algorithm to 3D. In this paper, the discrete minimal surface problem is both formulated and solved on a 3D graph. Specifically, we show that the problem may be formulated as a linear programming problem that is computed efficiently with generic solvers.

A novel method is proposed for performing multilabel, interactive image segmentation. Given a small number of pixels with user-defined (or predefined) labels, one can analytically and quickly determine the probability that a random walker starting at each unlabeled pixel will first reach one of the prelabeled pixels. By assigning each pixel to the label for which the greatest probability is calculated, a high-quality image segmentation may be obtained. Theoretical properties of this algorithm are developed along with the corresponding connections to discrete potential theory and electrical circuits. This algorithm is formulated in discrete space (i.e., on a graph) using combinatorial analogues of standard operators and principles from continuous potential theory, allowing it to be applied in arbitrary dimension on arbitrary graphs.

Spectral graph partitioning provides a powerful approach to image segmentation. We introduce an alternate idea that finds partitions with a small isoperimetric constant, requiring solution to a linear system rather than an eigenvector problem. This approach produces the high quality segmentations of spectral methods, but with improved speed and stability.

The recently introduced random walker segmentation algorithm by Grady and Funka-Lea (2004) has been shown to have desirable theoretical properties and to perform well on a wide variety of images in practice. However, this algorithm requires user-specified labels and produces a segmentation where each segment is connected to a labeled pixel. We show that incorporation of a nonparametric probability density model allows for an extended random walkers algorithm that can locate disconnected objects and does not require user-specified labels. Finally, we show that this formulation leads to a deep connection with the popular graph cuts method by Boykov et al. (2001) and Wu and Leahy (1993).

In the short time since publication of Boykov and Jolly's seminal paper [2001], graph cuts have become well established as a leading method in 2D and 3D semi-automated image segmentation. Although this approach is computationally feasible for many tasks, the memory overhead and supralinear time complexity of leading algorithms results in an excessive computational burden for high-resolution data. In this paper, we introduce a multilevel banded heuristic for computation of graph cuts that is motivated by the well-known narrow band algorithm in level set computation. We perform a number of numerical experiments to show that this heuristic drastically reduces both the running time and the memory consumption of graph cuts while producing nearly the same segmentation result as the conventional graph cuts. Additionally, we are able to characterize the type of segmentation target for which our multilevel banded heuristic yields different results from the conventional graph cuts. The proposed method has been applied to both 2D and 3D images with promising results.

The inhomogeneous Laplace (Poisson) equation with internal Dirichlet boundary conditions has recently appeared in several applications to image processing and analysis. Although these approaches have demonstrated quality results, the computational burden of solution demands an efficient solver. Design of an efficient multigrid solver is difficult for these problems due to unpredictable inhomogeneity in the equation coefficients and internal Dirichlet conditions with arbitrary location and value. We present a geometric multigrid approach to solving these systems designed around weighted prolongation/restriction operators and an appropriate system coarsening. This approach is compared against a modified incomplete Cholesky conjugate gradient solver for a range of image sizes. We note that this approach applies equally well to the anisotropic diffusion problem and offers an alternative method to the classic multigrid approach of Acton (1998).