Deblurring is a fundamental inverse problem in bioimaging. It requires modeling the point spread function (PSF), which captures the optical distortions entailed by the image formation process. The PSF limits the spatial resolution attainable for a given microscope. However, recent applications require a higher resolution and have prompted the development of super-resolution techniques to achieve sub-pixel accuracy. This requirement restricts the class of suitable PSF models to analog ones. In addition, deblurring is computationally intensive, hence further requiring computationally efficient models. A custom candidate fitting both the requirements is the Gaussian model. However, this model cannot capture the rich tail structures found in both the theoretical and empirical PSFs. In this paper, we aim at improving the reconstruction accuracy beyond the Gaussian model, while preserving its computational efficiency. We introduce a new class of analog PSF models based on the Gaussian mixtures. The number of Gaussian kernels controls both the modeling accuracy and the computational efficiency of the model: the lower the number of kernels, the lower the accuracy and the higher the efficiency. To explore the accuracy-efficiency tradeoff, we propose a variational formulation of the PSF calibration problem, where a convex sparsity-inducing penalty on the number of Gaussian kernels allows trading accuracy for efficiency. We derive an efficient algorithm based on a fully split formulation of alternating split Bregman. We assess our framework on synthetic and real data, and demonstrate a better reconstruction accuracy in both geometry and photometry in point source localization-a fundamental inverse problem in fluorescence microscopy.

Kymographs are widely used to represent and analyse spatio-temporal dynamics of fluorescence markers along curvilinear biological compartments. These objects have a singular geometry, thus kymograph reconstruction is inherently an analog image processing task. However, the existing approaches are essentially digital: the kymograph photometry is sampled directly from the time-lapse images. As a result, such kymographs rely on raw image data that suffer from the degradations entailed by the image formation process and the spatio-temporal resolution of the imaging setup. In this work, we address these limitations and introduce a well-grounded Bayesian framework for the analog reconstruction of kymographs. To handle the movement of the object, we introduce an intrinsic description of kymographs using differential geometry: a kymograph is a photometry defined on a parameter space that is embedded in physical space by a time-varying map that follows the object geometry. We model the kymograph photometry as a Lévy innovation process, a flexible class of non-parametric signal priors. We account for the image formation process using the virtual microscope framework. We formulate a computationally tractable representation of the associated maximum a posteriori problem and solve it using a class of efficient and modular algorithms based on the alternating split Bregman. We assess the performance of our Bayesian framework on synthetic data and apply it to reconstruct the fluorescence dynamics along microtubules in vivo in the budding yeast S. cerevisiae. We demonstrate that our framework allows revealing patterns from single time-lapse data that are invisible on standard digital kymographs.

-norm) is unable to correctly represent non-smooth displacement fields, that can, for example, occur at sliding interfaces in the thorax and abdomen in image time-series during respiration. In this paper, isotropic Total Variation (TV) regularization is used to enable accurate registration near such interfaces. We further develop the TV-regularization for parametric displacement fields and provide an efficient numerical solution scheme using the Alternating Directions Method of Multipliers (ADMM). The proposed method was successfully applied to four clinical databases which capture breathing motion, including CT lung and MR liver images. It provided accurate registration results for the whole volume. A key strength of our proposed method is that it does not depend on organ masks that are conventionally required by many algorithms to avoid errors at sliding interfaces. Furthermore, our method is robust to parameter selection, allowing the use of the same parameters for all tested databases. The average target registration error (TRE) of our method is superior (10% to 40%) to other techniques in the literature. It provides precise motion quantification and sliding detection with sub-pixel accuracy on the publicly available breathing motion databases (mean TREs of 0.95 mm for DIR 4D CT, 0.96 mm for DIR COPDgene, 0.91 mm for POPI databases).

We introduce a generative probabilistic model for segmentation of brain lesions in multi-dimensional images that generalizes the EM segmenter, a common approach for modelling brain images using Gaussian mixtures and a probabilistic tissue atlas that employs expectation-maximization (EM), to estimate the label map for a new image. Our model augments the probabilistic atlas of the healthy tissues with a latent atlas of the lesion. We derive an estimation algorithm with closed-form EM update equations. The method extracts a latent atlas prior distribution and the lesion posterior distributions jointly from the image data. It delineates lesion areas individually in each channel, allowing for differences in lesion appearance across modalities, an important feature of many brain tumor imaging sequences. We also propose discriminative model extensions to map the output of the generative model to arbitrary labels with semantic and biological meaning, such as "tumor core" or "fluid-filled structure", but without a one-to-one correspondence to the hypo- or hyper-intense lesion areas identified by the generative model. We test the approach in two image sets: the publicly available BRATS set of glioma patient scans, and multimodal brain images of patients with acute and subacute ischemic stroke. We find the generative model that has been designed for tumor lesions to generalize well to stroke images, and the extended discriminative -discriminative model to be one of the top ranking methods in the BRATS evaluation.

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

In this paper, a novel Markov random field (MRF)-based approach is presented for segmenting medical images while simultaneously registering an atlas nonrigidly. In the literature, both segmentation and registration have been studied extensively. For applications that involve both, such as segmentation via atlas-based registration, earlier studies proposed addressing these problems iteratively by feeding the output of each to initialize the other. This scheme, however, cannot guarantee an optimal solution for the combined task at hand, since these two individual problems are then treated separately. In this paper, we formulate simultaneous registration and segmentation (SRS) as a maximum a-posteriori (MAP) problem. We decompose the resulting probabilities such that the MAP inference can be done using MRFs. An efficient hierarchical implementation is employed, allowing coarse-to-fine registration while estimating segmentation at pixel level. The method is evaluated on two clinical data sets: 1) mandibular bone segmentation in 3D CT and 2) corpus callosum segmentation in 2D midsaggital slices of brain MRI. A video tracking example is also given. Our implementation allows us to directly compare the proposed method with the individual segmentation/registration and the iterative approach using the exact same potential functions. In a leave-one-out evaluation, SRS demonstrated more accurate results in terms of dice overlap and surface distance metrics for both data sets. We also show quantitatively that the SRS method is less sensitive to the errors in the registration as opposed to the iterative approach.

Interaction with virtual deformable models is common in several haptic contexts, such as in medical training simulators. This paper presents a methodological procedure for the creation of such virtual models from their real-life counterparts. Both the surface geometry and the elastic parametrization of an object are reconstructed from position/force readings during an operator-assisted exploration of the object. A 3D mesh model is then generated from the surface contact points. The internal elastic modulus is found using the 3D finite element method. This modeling method is compared with two common 1D elastic models, namely Kelvin-Voigt and Hunt-Crossley. Results using three deformable homogeneous silicone samples show successful geometry reconstruction. 1D model parameterizations exhibit high variation dependent on geometry and contact location. In contrast, elastic modulus reconstruction yields a global model parameterization independent of geometry. Elastic moduli estimated in experiments correlated with their known values, and were shown to be reproducible among samples with different geometries.

Shape prediction from sparse observation is of increasing interest in minimally invasive surgery, in particular when the target is not directly visible on images. This can be caused by a limited field-of-view of the imaging device, missing contrast or an insufficient signal-to-noise ratio. In such situations, a statistical shape model can be employed to estimate the location of unseen parts of the organ of interest from the observation and identification of the visible parts. However, the quantification of the reliability of such a prediction can be crucial for patient safety. We present here a framework for the estimation of complete shapes and of the associated uncertainties. This paper formalizes and extends previous work in the area by taking into account and incorporating the major sources of uncertainties, in particular the estimation of pose together with shape parameters, as well as the identification of correspondences between the sparse observation and the model. We evaluate our methodology on a large database of 171 human femurs and synthetic experiments based on a liver model. The experiments show that informative and reliable confidence regions can be estimated by the proposed approach.

We present a setup for simple and fast experimental estimation of drag coefficients. Our system can accurately determine the parameters of fluid-object interaction for complicated geometry and boundary conditions in the realm of classical Stokes equations. Obtained results are compared with theoretical and numerical solutions. Good agreement with those references is achieved in both cases. An advantage of our method is the prompt and easy parameter retrieval that still maintains appropriate accuracy. Comparable detailed numerical estimations run on the order of hours or days.

In our current research we examine the application of visuo-haptic augmented reality setups in medical training. To this end, highly accurate calibration, system stability, and low latency are indispensable prerequisites. These are necessary to maintain user immersion and avoid breaks in presence which potentially diminish the training outcome. In this paper we describe the developed calibration methods for visuo-haptic integration, the hybrid tracking technique for stable alignment of the augmentation, and the distributed framework ensuring low latency and component synchronization. Finally, we outline an early prototype system based on the multimodal augmented reality framework. The latter allows colocated visuo-haptic interaction with real and virtual scene components in a simplified open surgery setting.

Correspondence establishment is a key step in statistical shape model building. There are several automated methods for solving this problem in 3D, but they usually can only handle objects with simple topology, like that of a sphere or a disc. We propose an extension to correspondence establishment over a population based on the optimization of the minimal description length function, allowing considering objects with arbitrary topology. Instead of using a fixed structure of kernel placement on a sphere for the systematic manipulation of point landmark positions, we rely on an adaptive, hierarchical organization of surface patches. This hierarchy can be built on surfaces of arbitrary topology and the resulting patches are used as a basis for a consistent, multi-scale modification of the surfaces' parameterization, based on point distribution models. The feasibility of the approach is demonstrated on synthetic models with different topologies.

Mass spring models are frequently used to simulate deformable objects because of their conceptual simplicity and computational speed. Unfortunately, the model parameters are not related to elastic material constitutive laws in an obvious way. Several methods to set optimal parameters have been proposed, but so far only with limited success. We analyze the parameter identification problem and show the difficulties, which have prevented previous work from reaching wide usage. Our main contribution is a new method to derive analytical expressions for the spring parameters from an isotropic linear elastic reference model. The method is described and expressions for several mesh topologies are derived. These include triangle, rectangle and tetrahedron meshes. The formulae are validated by comparing the static deformation of the MSM with reference deformations simulated with the finite element method.

The driving force of our research is the precise combination of real and - possibly indistinguishable - virtual objects in an interactive augmented reality environment. This requires real-time, multimodal simulation, as well as stable and accurate overlay of the computer-generated objects. This paper describes several methods to improve accuracy and stability of our hybrid augmented reality system. In a comparison of two approaches to hybrid head pose refinement, we show that Quasi-Newton method enables high performance optimization for image space error minimization. Moreover, a 3D landmark refinement step is proposed, which significantly improves quality and robustness of the overlay process. The enhanced system is demonstrated in an interactive AR environment, which provides accurate haptic feedback from real and virtual deformable objects. Finally, the effect of landmark occlusion on tracking stability during user interaction is also analyzed.

A central training objective of virtual reality based surgical simulation is the removal of pathologic tissue. This necessitates stable, real-time updates of the underlying mesh representation. Within the framework of a hysteroscopy simulator, we have developed a hybrid cutting approach for tetrahedral meshes. It combines the topological update by subdivision with adjustments of the existing topology. Moreover, the mechanical and the visual model are decoupled, thus allowing different resolutions for the underlying mesh representations. With our method, we can closely approximate an arbitrary, user-defined cut surface while avoiding the creation of small or badly shaped elements, thus strongly reducing stability problems in the subsequent deformation computation. The presented approach has been integrated into a virtual reality training system for hysteroscopic interventions. The performance of the algorithm is demonstrated by examples of intra-uterine tumor ablations.

An integral part of every augmented reality system is the calibration between camera and camera-mounted tracking markers. Accuracy and robustness of the AR overlay process is greatly influenced by the quality of this step. In order to meet the very high precision requirements of medical skill training applications, we have set up a calibration environment based on direct sensing of LED markers. A simulation framework has been developed to predict and study the achievable accuracy of the backprojection needed for the scene augmentation process. We demonstrate that the simulation is in good agreement with experimental results. Even if a slight improvement of the precision has been observed compared to well-known hand-eye calibration methods, the subpixel accuracy required by our application cannot be achieved even when using commercial tracking systems providing marker positions within very low error limits.

Assessing the displacement of bony implants is an important topic in arthroplasty, particularly in total hip replacement (THR). The observation of the migration is supposed to provide an insight into the fixation of the implant. Diagnostic standard radiographs of the pelvis are an advantageous data source for this purpose. The previous methods based on these images, however, lack of a thorough consideration of their projective nature. They do, hence, not reach the desired precision, which should lie in the submillimeter range to allow a detection of migration in the first one or two years after implantation. The aim of the work presented here was, therefore, a method for measuring the distance of the artificial hip socket to the bone with an error of less than 0.5 mm. The approach has been on the one hand to define the bone-cup distance measured in the radiograph so that the variability of the intrinsic and extrinsic parameters at exposure has a minimal impact. On the other, specialized matching techniques are applied in order to optimize the localization of the necessary bony landmarks and the cup in the X-ray image. The coordinates of the bony landmarks are determined by means of a template matching algorithm. The position of the implant is estimated by intensity-based registration using the cup's CAD-model. The method was validated theoretically, experimentally, and clinically. In the clinical radiographs, the standard deviation of the migration measurements resulted to be 0.28 mm when using only natural bony landmarks. The implantation of a bony marker was found to increase the precision to a standard deviation of 0.20 mm. The interobserver variability in the two cases was estimated to be 0.11 mm and 0.04 mm.

Medical image acquisition devices are becoming increasingly multidimensional, and predictions assume 5000 images per patient study within the next decade. Therefore, new paradigms for computerized medical image analysis and visualization are of fundamental importance to make possibly full use of the information buried in the enormous flood of image data. The aim of the presented project is the optimal cooperation between computer-based image analysis algorithms and human operators using new closed-loop segmentation systems for improved information flow. This paper describes an enhanced, visuo-haptic interaction tool we have developed for medical segmentation. Evaluation studies with the system, which confirm the value of adding haptic feedback, are also presented.

We present a new virtual reality-based interaction metaphor for semi-automatic segmentation of medical 3D volume data. The mouse-based, manual initialization of deformable surfaces in 3D represents a major bottleneck in interactive segmentation. In our multi-modal system we enhance this process with additional sensory feedback. A 3D haptic device is used to extract the centerline of a tubular structure. Based on the obtained path a cylinder with varying diameter is generated which in turn is used as the initial guess for a deformable surface.

Abstract We present a new virtual reality‐based interaction metaphor for semi‐automatic segmentation of medical 3D volume data. The mouse‐based, manual initialization of deformable surfaces in 3D represents a major bottleneck in interactive segmentation. In our multi‐modal system we enhance this process with additional sensory feedback. A 3D haptic device is used to extract the centreline of a tubular structure. Based on the obtained path a cylinder with varying diameter is generated, which in turn is used as the initial guess for a deformable surface. Copyright © 2002 John Wiley & Sons, Ltd.

Abstract A detailed finite element model of the human kidney for trauma research has been created directly from the National Library of Medicine Visible Human Female (VHF) Project data set. An image segmentation and organ reconstruction software package has been developed and employed to transform the 2D VHF images into a 3D polygonal representation. Non‐uniform rational B‐spline (NURBS) surfaces were then mapped to the polygonal surfaces, and were finally utilized to create a robust 3D hexahedral finite element mesh within a commercially available meshing software. The model employs a combined viscoelastic and hyperelastic material model to successfully simulate the behaviour of biological soft tissues. The finite element model was then validated for use in biomechanical research. Copyright © 2002 John Wiley & Sons, Ltd.

A procedure for the parameterization of surface meshes of objects with spherical topology is presented. The generation of such a parameterisation has been formulated and solved as a large constrained optimization problem by C. Brechbuhler (1995), but the convergence of this algorithm becomes unstable for object meshes consisting of several thousand vertices. We propose a new more stable algorithm to overcome this problem using multi-resolution meshes. A triangular mesh is mapped to a sphere by harmonic mapping. Next, a mesh hierarchy is constructed. The coarsest level is then optimized using a modification of the original procedure to map object surfaces to the unit sphere. The result is used as a starting point for the mapping of the next finer mesh, a process which is repeated until the final result is obtained. The new approach is compared to the original one and some parameterized object surfaces are presented.

Virtual reality surgery simulators are almost certainly the future of endoscopic surgery trainers. While many aspects of such systems, such as the visualization of the operating scene, have been brought to satisfactory levels of resemblance to real endoscopic surgery, the haptic feedback simulation is one of the major obstacles remaining. The aim of this work is to describe the main components necessary for a realistic haptic feedback in surgery, concentrating on the modelling of soft organic tissue, necessary for the simulation of soft tissue deformation. We also present a method to measure in-vivo the material parameters figuring in the developed elastomechanical models of living tissues.

This paper presents a new approach to the correction of intensity inhomogeneities in magnetic resonance imaging (MRI) that significantly improves intensity-based tissue segmentation. The distortion of the image brightness values by a low-frequency bias field impedes visual inspection and segmentation. The new correction method called parametric bias field correction (PABIC) is based on a simplified model of the imaging process, a parametric model of tissue class statistics, and a polynomial model of the inhomogeneity field. We assume that the image is composed of pixels assigned to a small number of categories with a priori known statistics. Further we assume that the image is corrupted by noise and a low-frequency inhomogeneity field. The estimation of the parametric bias field is formulated as a nonlinear energy minimization problem using an evolution strategy (ES). The resulting bias field is independent of the image region configurations and thus overcomes limitations of methods based on homomorphic filtering. Furthermore, PABIC can correct bias distortions much larger than the image contrast. Input parameters are the intensity statistics of the classes and the degree of the polynomial function. The polynomial approach combines bias correction with histogram adjustment, making it well suited for normalizing the intensity histogram of datasets from serial studies. We present simulations and a quantitative validation with phantom and test images. A large number of MR image data acquired with breast, surface, and head coils, both in two dimensions and three dimensions, have been processed and demonstrate the versatility and robustness of this new bias correction scheme.

This paper presents a new technique for the automatic model-based segmentation of three-dimensional (3-D) objects from volumetric image data. The development closely follows the seminal work of Taylor and Cootes on active shape models, but is based on a hierarchical parametric object description rather than a point distribution model. The segmentation system includes both the building of statistical models and the automatic segmentation of new image data sets via a restricted elastic deformation of shape models. Geometric models are derived from a sample set of image data which have been segmented by experts. The surfaces of these binary objects are converted into parametric surface representations, which are normalized to get an invariant object-centered coordinate system. Surface representations are expanded into series of spherical harmonics which provide parametric descriptions of object shapes. It is shown that invariant object surface parametrization provides a good approximation to automatically determine object homology in terms of sets of corresponding sets of surface points. Gray-level information near object boundaries is represented by 1-D intensity profiles normal to the surface. Considering automatic segmentation of brain structures as our driving application, our choice of coordinates for object alignment was the well-accepted stereotactic coordinate system. Major variation of object shapes around the mean shape, also referred to as shape eigenmodes, are calculated in shape parameter space rather than the feature space of point coordinates. Segmentation makes use of the object shape statistics by restricting possible elastic deformations into the range of the training shapes. The mean shapes are initialized in a new data set by specifying the landmarks of the stereotactic coordinate system. The model elastically deforms, driven by the displacement forces across the object's surface, which are generated by matching local intensity profiles. Elastic deformations are limited by setting bounds for the maximum variations in eigenmode space. The technique has been applied to automatically segment left and right hippocampus, thalamus, putamen, and globus pallidus from volumetric magnetic resonance scans taken from schizophrenia studies. The results have been validated by comparison of automatic segmentation with the results obtained by interactive expert segmentation.

Presents a novel, parameter-free technique for the segmentation and local description of line structures on multiple scales, both in 2D and in 3D. The algorithm is based on a nonlinear combination of linear filters and searches for elongated, symmetric line structures, while suppressing the response to edges. The filtering process creates one sharp maximum across the line-feature profile and across the scale-space. The multi-scale response reflects local contrast and is independent of the local width. The filter is steerable in both the orientation and scale domains, leading to an efficient, parameter-free implementation. A local description is obtained that describes the contrast, the position of the center-line, the width, the polarity, and the orientation of the line. Examples of images from different application domains demonstrate the generic nature of the line segmentation scheme. The 3D filtering is applied to magnetic resonance volume data in order to segment cerebral blood vessels.>

We present a new approach to segmentation of 3-D shapes that initializes and then optimizes a 3-D surface model given only the data and a very small number of 3-D seed points and corresponding surface normals. This is a valuable capability for medical, robotic and cartographic applications where such seed points can be naturally supplied. In effect, the surface model is clamped onto the object boundary in manner reminiscent of a Velcro being closed. We develop the method's mathematic framework and show preliminary results using volumetric medical data.>

We propose a snake-based approach that lets a user specify only the distant end points of the curve he wishes to delineate without having to supply an almost complete polygonal approximation. We achieve much better convergence properties than those of traditional snakes by using the image information around these end points to provide boundary conditions and by introducing an optimization schedule that allows the snake to take image information into account first only near its extremities and then, progressively, towards its center. These snakes could be used to alleviate the often repetitive task practitioners have to face when segmenting images by abolishing the need to sketch a feature of interest in its entirety, that is, to perform a painstaking, almost complete, manual segmentation.>

In this paper, we present a new snake-based method to delineate contours where the user has to specify only the two distant end points without having to supply an almost complete polygonal approximation. We achieve much better convergence properties than those of traditional snakes by propagating the image information along the curve from both end points towards its center. We use our new method to outline curved object boundaries and for interactive road delineation.