In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

Methods that leverage neighbourhood structures in high-dimensional image spaces have recently attracted attention. These approaches extract information from a new image using its "neighbours" in the image space equipped with an application-specific distance. Finding the neighbourhood of a given image is challenging due to large dataset sizes and costly distance evaluations. Furthermore, automatic neighbourhood search for a new image is currently not possible when the distance is based on ground truth annotations. In this article we present a general and efficient solution to these problems. "neighbourhood approximation forests" (NAF) is a supervised learning algorithm that approximates the neighbourhood structure resulting from an arbitrary distance. As NAF uses only image intensities to infer neighbours it can also be applied to distances based on ground truth annotations. We demonstrate NAF in two scenarios: (i) choosing neighbours with respect to a deformation-based distance, and (ii) age prediction from brain MRI. The experiments show NAF's approximation quality, computational advantages and use in different contexts.

We present a preconditioning scheme for improving the efficiency of optimization of arbitrary difference measures in deformable registration problems. This is of particular interest for high-dimensional registration problems with statistical difference measures such as MI, and the demons method, since in these cases the range of applicable optimization methods is limited. The proposed scheme is simple and computationally efficient: It performs an approximate normalization of the point-wise vectors of the difference gradient to unit length. The major contribution of this work is a theoretical analysis which demonstrates the improvement of the condition by our approach, which is furthermore shown to be an approximation to the optimal case for the analyzed model. Our scheme improves the convergence speed while adding only negligible computational cost, thus resulting in shorter effective runtimes. The theoretical findings are confirmed by experiments on 3D brain data.

Variational level set methods are formulated as energy minimisation problems, which are often solved by gradient-based optimisation methods, such as gradient descent. Unfortunately, the gradient obtained by applying the calculus of variations is not suitable, because it is only an element of the function space L 2 making it prone to lead into wrong local minima. Consequently, some regularisation strategy - be it the restriction to signed distance functions or the choice of smooth function spaces - is necessary. In this paper we propose diffusion-based regularisation strategies and compare them to the recently proposed ones of Charpiat et al. and Sundaramoorthi et al. From this comparison we derive two general regularisation paradigms at the level of update equations and show that the diffusion-based paradigm enjoys both theoretical and practical advantages, such as an improved convergence rate, while being of the same computational complexity as the other paradigm.

We revisit the classic Horn and Schunck optical flow method, with focus on its interpretation as a Gauss-Newton (GN) minimisation of a non-linear energy. This is in contrast to the traditional derivation by linearisation of the brightness constancy assumption. The proposed interpretation provides a much simpler derivation and a better theoretical understanding of the method, and allows for its variations, by casting it in the least-squares optimisation framework. An important resulting implication is that ‐ in contrast to popular belief ‐ the incremental version of Horn and Schunck actually minimises a non-linear energy. We emphasise this finding by demonstrating the equivalence of incremental Horn and Schunck to several methods committed to minimising the nonlinearised energy. Furthermore, we analyse the effect of GN for motion estimation in comparison to methods based on gradient descent. Several examples demonstrate the practical applications of the proposed interpretation. We specify the class of difference measures with sparse Jacobians of the error term as the one which can be efficiently treated in the Horn and Schunck framework. For extension to arbitrary difference measures, we propose a modification based on the analysis of GN for motion estimation, and the concept of preconditioning. We further discuss a modification resulting in decoupled linear systems and the use of compositional updates.

We apply the concept of natural gradients to deformable registration. The motivation stems from the lack of physical interpretation for gradients of image-based difference measures. The main idea is to endow the space of deformations with a distance metric which reflects the variation of the difference measure between two deformations. This is in contrast to standard approaches which assume the Euclidean frame. The modification of the distance metric is realized by treating the deformations as a Riemannian manifold. In our case, the manifold is induced by the Riemannian metric tensor based on the approximation of the Fisher Information matrix, which takes into account the information about the chosen difference measure and the input images. Thus, the resulting natural gradient defined on this manifold inherently takes into account this information. The practical advantages of the proposed approach are the improvement of registration error and faster convergence for low-gradient regions. The proposed scheme is applicable to arbitrary difference measures and can be readily integrated into standard variational deformable registration methods with practically no computational overhead.

Alignment of angiographic 3D scans to 2D projections is an important issue for 3D depth perception and navigation during interventions. Currently, in a setting where only one 2D projection is available, methods employing a rigid transformation model present the state of the art for this problem. In this work, we introduce a method capable of deformably registering 3D vessel structures to a respective single projection of the scene. Our approach addresses the inherent ill-posedness of the problem by incorporating a priori knowledge about the vessel structures into the formulation. We minimize the distance between the 2D points and corresponding projected 3D points together with regularization terms encoding the properties of length preservation of vessel structures and smoothness of deformation. We demonstrate the performance and accuracy of the proposed method by quantitative tests on synthetic examples as well as real angiographic scenes.

Nonlinear registration is mostly performed after initialization by a global, linear transformation (in this work, we focus on similarity transformations), computed by a linear registration method. For the further processing of the results, it is mostly assumed that this preregistration step completely removes the respective linear transformation. However, we show that in deformable settings, this is not the case. As a consequence, a significant linear component is still existent in the deformation computed by the nonlinear registration algorithm. For construction of statistical shape models (SSM) from deformations, this is an unwanted property: SSMs should not contain similarity transformations, since these do not capture information about shape. We propose a method which performs an a posteriori extraction of a similarity transformation from a given nonlinear deformation field, and we use the processed fields as input for SSM construction. For computation of minimal displacements, a closed-form solution minimizing the squared Euclidean norm of the displacement field subject to similarity parameters is used. Experiments on real inter-subject data and on a synthetic example show that the theoretically justified removal of the similarity component by the proposed method has a large influence on the shape model and significantly improves the results.

We propose a novel numerical approach for solving the free-form deformable registration problem. The central idea is to utilize the well understood techniques from variational deformable registration problems. We demonstrate that it is possible to formulate the free-form deformable registration problem as the optimization of an energy functional as in the dense deformation case. This energy functional possesses image distance and regularization terms, which are both functions of the free-form deformation control points. We then setup a semi-backward (implicit) partial differential equation that optimizes the established energy functional. In addition to being mathematically justified, this approach provides both accuracy and speed. Our evaluation on synthetic, real, two dimensional, and three dimensional data demonstrates accuracy and computational effectiveness.