Unsupervised representation learning has become an important ingredient of today's deep learning systems. However, only a few methods exist that explain a learned vector embedding in the sense of providing information about which parts of an input are the most important for its representation. These methods generate the explanation for a given input after the model has been evaluated and tend to produce either inaccurate explanations or are slow, which limits their practical use. To address these limitations, we introduce the Neural Explanation Masks (NEM) framework, which turns a fixed representation model into a self-explaining model by augmenting it with a masking network. This network provides occlusion-based explanations in parallel to computing the representations during inference. We present an instance of this framework, the NEM-U (NEM using U-net structure) architecture, which leverages similarities between segmentation and occlusion-based masks. Our experiments show that NEM-U generates explanations faster and with lower complexity compared to the current state-of-the-art while maintaining high accuracy as measured by locality.

Diffeomorphic deformation is a popular choice in medical image registration. A fundamental property of diffeomorphisms is invertibility, implying that once the relation between two points A to B is found, then the relation B to A is given per definition. Consistency is a measure of a numerical algorithm's ability to mimic this invertibility, and achieving consistency has proven to be a challenge for many state-of-the-art algorithms. We present CDD (Collocation for Diffeomorphic Deformations), a numerical solution to diffeomorphic image registration, which solves for the Stationary Velocity Field (SVF) using an implicit A-stable collocation method. CDD guarantees the preservation of the diffeomorphic properties at all discrete points and is thereby consistent to machine precision. We compared CDD's collocation method with the following standard methods: Scaling and Squaring, Forward Euler, and Runge-Kutta 4, and found that CDD is up to 9 orders of magnitude more consistent. Finally, we evaluated CDD on a number of standard bench-mark data sets and compared the results with current state-of-the-art methods: SPM-DARTEL, Diffeomorphic Demons and SyN. We found that CDD outperforms state-of-the-art methods in consistency and delivers comparable or superior registration precision.

In this paper, we propose a multi-scale, multi-kernel shape, compactly supported kernel bundle framework for stationary velocity field-based image registration (Wendland kernel bundle stationary velocity field, wKB-SVF). We exploit the possibility of directly choosing kernels to construct a reproducing kernel Hilbert space (RKHS) instead of imposing it from a differential operator. The proposed framework allows us to minimize computational cost without sacrificing the theoretical foundations of SVF-based diffeomorphic registration. In order to recover deformations occurring at different scales, we use compactly supported Wendland kernels at multiple scales and orders to parameterize the velocity fields, and the framework allows simultaneous optimization over all scales. The performance of wKB-SVF is extensively compared to the 14 non-rigid registration algorithms presented in a recent comparison paper. On both MGH10 and CUMC12 datasets, the accuracy of wKB-SVF is improved when compared to other registration algorithms. In a disease-specific application for intra-subject registration, atrophy scores estimated using the proposed registration scheme separates the diagnostic groups of Alzheimer's and normal controls better than the state-of-the-art segmentation technique. Experimental results show that wKB-SVF is a robust, flexible registration framework that allows theoretically well-founded and computationally efficient multi-scale representation of deformations and is equally well-suited for both inter- and intra-subject image registration.

We study the space of first order models of smooth frame fields using the method of moving frames. By exploiting the Maurer-Cartan matrix of connection forms we develop geometrical embeddings for frame fields which lie on spherical, ellipsoidal and generalized helicoid surfaces. We design methods for optimizing connection forms in local neighborhoods and apply these to a statistical analysis of heart fiber geometry, using diffusion magnetic resonance imaging. This application of moving frames corroborates and extends recent characterizations of muscle fiber orientation in the heart wall, but also provides for a rich geometrical interpretation. In particular, we can now obtain direct local measurements of the variation of the helix and transverse angles, of fiber fanning and twisting, and of the curvatures of the heart wall in which these fibers lie.

Abstract We present a photon splatting technique which reduces noise and blur in the rendering of caustics. Blurring of illumination edges is an inherent problem in photon splatting, as each photon is unaware of its neighbours when being splatted. This means that the splat size is usually based on heuristics rather than knowledge of the local flux density. We use photon differentials to determine the size and shape of the splats such that we achieve adaptive anisotropic flux density estimation in photon splatting. As compared to previous work that uses photon differentials, we present the first method where no photons or beams or differentials need to be stored in a map. We also present improvements in the theory of photon differentials, which give more accurate results and a faster implementation. Our technique has good potential for GPU acceleration, and we limit the number of parameters requiring user adjustment to an overall smoothing parameter and the number of photons to be traced.

This paper presents a unifying approach for calculating a wide range of popular, but seemingly very different, similarity measures. Our domain is the registration of n-dimensional images sampled on a regular grid, and our approach is well suited for gradient-based optimization algorithms. Our approach is based on local intensity histograms and built upon the technique of Locally Orderless Images. Histograms by Locally Orderless Images are well posed and offer explicit control over the three inherent and unavoidable scales: the spatial resolution, intensity levels, and spatial extent of local histograms. Through Locally Orderless Images, we offer new insight into the relations between these scales. We demonstrate our unification by developing a Locally Orderless Registration algorithm for two quite different similarity measures, namely, Normalized Mutual Information and Sum of Squared Differences, and we compare these variations both theoretically and empirically. Finally, using our algorithm, we explain the empirically observed differences between two popular joint density estimation techniques used in registration: Parzen Windows and Generalized Partial Volume.

The estimation of covariance matrices is a crucial step in several statistical tasks. Especially when using few samples of a high dimensional representation of shapes, the standard maximum likelihood estimation (ML) of the covariance matrix can be far from the truth, is often rank deficient, and may lead to unreliable results. In this paper, we discuss regularization by prior knowledge using maximum a posteriori (MAP) estimates. We compare ML to MAP using a number of priors and to Tikhonov regularization. We evaluate the covariance estimates on both synthetic and real data, and we analyze the estimates' influence on a missing-data reconstruction task, where high resolution vertebra and cartilage models are reconstructed from incomplete and lower dimensional representations. Our results demonstrate that our methods outperform the traditional ML method and Tikhonov regularization.

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

Abstract Density estimation employed in multi‐pass global illumination algorithms give cause to a trade‐off problem between bias and noise. The problem is seen most evident as blurring of strong illumination features. In particular, this blurring erodes fine structures and sharp lines prominent in caustics. To address this problem, we introduce a photon mapping algorithm based on nonlinear anisotropic diffusion. Our algorithm adapts according to the structure of the photon map such that smoothing occurs along edges and structures and not across. In this way, we preserve important illumination features, while eliminating noise. We demonstrate the applicability of our algorithm through a series of tests. In the tests, we evaluate the visual and computational performance of our algorithm comparing it to existing popular algorithms.

Perl is a powerful language for writing portable programs on a wide range of platforms. In this article two stand-alone Perl toolkits, Prima and IPA, are presented, which provide graphical user interface and image processing functionality with features not found in any other Perl toolkits. Prima is a platform-independent Perl graphic toolkit with an object-oriented interface. Its features include an extensive set of Perl-coded interface elements, a wide range of image types and conversion routines, and a visual builder. IPA is an image processing toolkit based on Prima functionality, and it provides a set of common two-dimensional operators. Prima and IPA are used as a base for biological research, where the effects of various peptide-based preparations on neuron culture are studied. Copyright © 2005 John Wiley & Sons, Ltd.

Rotation of three-dimensional objects by a two-dimensional mouse is a typical task in computer-aided design, operation simulations, and desktop virtual reality. The most commonly used rotation technique is a virtual trackball surrounding the object and operated by the mouse pointer. This article reviews and provides a mathematical foundation for virtual trackballs. The first, but still popular, virtual trackball was described by Chen et al. We show that the virtual trackball by Chen et al. does not rotate the object along the intended great circular arc on the virtual trackball and we give a correction. Another popular virtual trackball is Shoemake's quaternion implementation, which we show to be a special case of the virtual trackball by Chen et al.. Shoemake extends the scope of the virtual trackball to the full screen. Unfortunately, Shoemake's virtual trackball is inhomogeneous and discontinuous with consequences for usability. Finally, we review Bell's virtual trackball and discuss studies of the usability of virtual trackballs.

Udgivelsesdato: June

The structure in digitized images resides within two scales, the inner and outer scale. The inner scale is defined by the sampling resolution, and the outer scale is given by the image size. However, some images contain almost no fine scale structure, and these may be down-sampled without essential loss of image detail. Likewise some images may be reduced in size by removing borders with no structure. Hence we define essential inner and outer scales. Such considerations are the essence of local size estimation: a textured patch in an image has an essential inner scale related to the structure of the primitive textons, and an essential outer scale given by the size of the patch. In this paper, several functionals are examined that automatically find both the essential inner and outer scales in local neighborhoods of an image. In this preliminary work we present a general formulation for local scale selection, that is shown to be a generalization of Lindeberg's (see International Journal of Computer Vision, vol.30, no.2, p.117-56, 1998) Blob-detector and its morphological equivalent, and we present promising results using locally orderless images.

This article investigates Renyi's (1976) generalized entropies under linear and nonlinear scale-space evolutions of images. Scale-spaces are useful computer vision concepts for both scale analysis and image restoration. We regard images as densities and prove monotony and smoothness properties for the generalized entropies. The scale-space extended generalized entropies are applied to global scale selection and size estimations. Finally, we introduce an entropy-based fingerprint description for textures.

The isophote curvature times the gradient magnitude to some power has been studied in the literature as a measure of cornerness in images, and stability in terms of sampling noise has been proposed by selecting corners in these measures at high scale and locating them at fine scale. We will examine the problem of tracking extrema of these measures in the linear scale-space and conclude that annihilations and creations generically occurs, so that corners in general cannot be tracked to arbitrarily fine/coarse scale. However, there are quantitatively differences, and the analysis indicates that isophote curvature times the gradient magnitude is best suited for binary images.

Viewing images as distributions of light quanta enables an information theoretic study of image structures on different scales. This article combines Shannon's entropy and Witkin and Koenderink's scale-space to establish a precise connection between the heat equation and the thermodynamic entropy in scale-space. Experimentally the entropy function is used to study global textures.