Many real-world vision problems suffer from inherent ambiguities. In clinical applications for example, it might not be clear from a CT scan alone which particular region is cancer tissue. Therefore a group of graders typically produces a set of diverse but plausible segmentations. We consider the task of learning a distribution over segmentations given an input. To this end we propose a generative segmentation model based on a combination of a U-Net with a conditional variational autoencoder that is capable of efficiently producing an unlimited number of plausible hypotheses. We show on a lung abnormalities segmentation task and on a Cityscapes segmentation task that our model reproduces the possible segmentation variants as well as the frequencies with which they occur, doing so significantly better than published approaches. These models could have a high impact in real-world applications, such as being used as clinical decision-making algorithms accounting for multiple plausible semantic segmentation hypotheses to provide possible diagnoses and recommend further actions to resolve the present ambiguities.

Dental radiography plays an important role in clinical diagnosis, treatment and surgery. In recent years, efforts have been made on developing computerized dental X-ray image analysis systems for clinical usages. A novel framework for objective evaluation of automatic dental radiography analysis algorithms has been established under the auspices of the IEEE International Symposium on Biomedical Imaging 2015 Bitewing Radiography Caries Detection Challenge and Cephalometric X-ray Image Analysis Challenge. In this article, we present the datasets, methods and results of the challenge and lay down the principles for future uses of this benchmark. The main contributions of the challenge include the creation of the dental anatomy data repository of bitewing radiographs, the creation of the anatomical abnormality classification data repository of cephalometric radiographs, and the definition of objective quantitative evaluation for comparison and ranking of the algorithms. With this benchmark, seven automatic methods for analysing cephalometric X-ray image and two automatic methods for detecting bitewing radiography caries have been compared, and detailed quantitative evaluation results are presented in this paper. Based on the quantitative evaluation results, we believe automatic dental radiography analysis is still a challenging and unsolved problem. The datasets and the evaluation software will be made available to the research community, further encouraging future developments in this field. (http://www-o.ntust.edu.tw/~cweiwang/ISBI2015/).

This paper presents an approach for motion-based anomaly detection, where a prototype pattern is detected and elastically registered against a test sample to detect anomalies in the test sample. The prototype model is learned from multiple sequences to define accepted variations. “Supertrajectories” based on hierarchical clustering of dense point trajectories serve as an efficient and robust representation of motion patterns. An efficient hashing approach provides transformation hypotheses that are refined by a spatiotemporal elastic registration. We propose a new method for elastic registration of 3D+time trajectory patterns that induces spatial elasticity from trajectory affinities. The method is evaluated on a new motion anomaly dataset of juggling patterns and performs well in detecting subtle anomalies. Moreover, we demonstrate the applicability to biological motion patterns.

With volumetric data from wide field fluorescence microscopy, many emerging questions in biological and biomedical research are being investigated. Data can be recorded with high temporal resolution while the specimen is only exposed to a low amount of photo toxicity. These advantages come at the cost of strong recording blur caused by the infinitely extended point spread function (PSF). For wide field microscopy, its magnitude only decays with the square of the distance to the focal point and consists of an airy bessel pattern which is intricate to describe in the spatial domain. However, the Fourier transform of the incoherent PSF (denoted as Optical Transfer Function (OTF)) is well localized and smooth. In this paper, we present a blind deconvolution method that improves results of state-of-the-art deconvolution methods on wide field data by exploiting the properties of the wide field OTF.

BACKGROUND: Absorption and refraction induced signal attenuation can seriously hinder the extraction of quantitative information from confocal microscopic data. This signal attenuation can be estimated and corrected by algorithms that use physical image formation models. Especially in thick heterogeneous samples, current single view based models are unable to solve the underdetermined problem of estimating the attenuation-free intensities. RESULTS: We present a variational approach to estimate both, the real intensities and the spatially variant attenuation from two views of the same sample from opposite sides. Assuming noise-free measurements throughout the whole volume and pure absorption, this would in theory allow a perfect reconstruction without further assumptions. To cope with real world data, our approach respects photon noise, estimates apparent bleaching between the two recordings, and constrains the attenuation field to be smooth and sparse to avoid spurious attenuation estimates in regions lacking valid measurements. CONCLUSIONS: We quantify the reconstruction quality on simulated data and compare it to the state-of-the art two-view approach and commonly used one-factor-per-slice approaches like the exponential decay model. Additionally we show its real-world applicability on model organisms from zoology (zebrafish) and botany (Arabidopsis). The results from these experiments show that the proposed approach improves the quantification of confocal microscopic data of thick specimen.

In many vision problems, rotation-invariant analysis is necessary or preferred. Popular solutions are mainly based on pose normalization or brute-force learning, neglecting the intrinsic properties of rotations. In this paper, we present a rotation invariant detection approach built on the equivariant filter framework, with a new model for learning the filtering behavior. The special properties of the harmonic basis, which is related to the irreducible representation of the rotation group, directly guarantees rotation invariance of the whole approach. The proposed kernel weighted mapping ensures high learning capability while respecting the invariance constraint. We demonstrate its performance on 2D object detection with in-plane rotations, and a 3D application on rotation-invariant landmark detection in microscopic volumetric data.

We present a method for densely computing local rotation invariant image descriptors in volumetric images. The descriptors are based on a transformation to the harmonic domain, which we compute very efficiently via differential operators. We show that this fast voxelwise computation is restricted to a family of basis functions that have certain differential relationships. Building upon this finding, we propose local descriptors based on the Gaussian Laguerre and spherical Gabor basis functions and show how the coefficients can be computed efficiently by recursive differentiation. We exemplarily demonstrate the effectiveness of such dense descriptors in a detection and classification task on biological 3D images. In a direct comparison to existing volumetric features, among them 3D SIFT, our descriptors reveal superior performance.

We propose an algorithm for 3-D multiview deblurring using spatially variant point spread functions (PSFs). The algorithm is applied to multiview reconstruction of volumetric microscopy images. It includes registration and estimation of the PSFs using irregularly placed point markers (beads). We formulate multiview deblurring as an energy minimization problem subject to L1-regularization. Optimization is based on the regularized Lucy-Richardson algorithm, which we extend to deal with our more general model. The model parameters are chosen in a profound way by optimizing them on a realistic training set. We quantitatively and qualitatively compare with existing methods and show that our method provides better signal-to-noise ratio and increases the resolution of the reconstructed images.

Rotation-invariant descriptions are required in many 3D volumetric image analysis tasks. The histogram-of-oriented-gradient (HOG) is widely used in 2D images and proves to be a very robust local description. This paper concentrates on how to use the HOG feature in 3D volumetric images when rotation-invariance is concerned. This is challenging because of the complexity of 3D rotations. We present a decent solution based on the spherical harmonics theory which is an effective tool for analysing 3D rotations, together with the spherical tensor operations which explore high order tensor information in spherical coordinates. The design is quite general and could be used for different applications. It achieves high scores on Princeton Shape Benchmark and SHREC 2009 Generic Shape Benchmark, and also produces promising results when applying on biological microscopy images.

Gradient vector flow snakes are a very common method in bio-medical image segmentation. The use of gradient vector flow herein brings some major advantages like a large capture range and a good adaption of the snakes in concave regions. In some cases though, the application of gradient vector flow can also have undesired effects, e.g. if only parts of an image are strongly blurred, the remaining weak gradients will be smoothed away. Also, large gradients resulting from small but bright image structures usually have strong impact on the overall result. To tackle this problem, we present an improvement of the gradient vector flow, using the mean shift procedure and show its advantages on the segmentation of 3D cell nuclei.

When using deformable models for the segmentation of biological data, the choice of the best weighting parameters for the internal and external forces is crucial. Especially when dealing with 3D fluorescence microscopic data and cells within dense tissue, object boundaries are sometimes not visible. In these cases, one weighting parameter set for the whole contour is not desirable. We are presenting a method for the dynamic adjustment of the weighting parameters, that is only depending on the underlying data and does not need any prior information. The method is especially apt to handle blurred, noisy, and deficient data, as it is often the case in biological microscopy.

In this paper, we present a novel approach for a trainable rotation invariant detection of complex structures in 3D microscopic multichannel data using a nonlinear filter approach. The basic idea of our approach is to compute local features in a window around each 3D position and map these features by means of a nonlinear mapping onto new local harmonic descriptors of the local window. These local harmonic descriptors are then combined in a linear way to form the output of the filter. The optimal combination of the computed local harmonic descriptors is determined in previous training step, and allows the filter to be adapted to an arbitrary structure depending on the problem at hand. Our approach is not limited to scalar-valued images and can also be used for vector-valued (multichannel) images such as gradient vector flow fields. We present realizations of a scalar-valued and a vector-valued multichannel filter. Our proposed algorithm was quantitatively evaluated on colorectal cancer cell lines (cells grown under controlled conditions), on which we successfully detected complex 3D mitotic structures. For a qualitative evaluation we tested our algorithms on human 3D tissue samples of colorectal cancer. We compare our results with a steerable filter approach as well as a morphology-based approach.

In this paper, polar and spherical Fourier analysis are defined as the decomposition of a function in terms of eigenfunctions of the Laplacian with the eigenfunctions being separable in the corresponding coordinates. The proposed transforms provide effective decompositions of an image into basic patterns with simple radial and angular structures. The theory is compactly presented with an emphasis on the analogy to the normal Fourier transform. The relation between the polar or spherical Fourier transform and the normal Fourier transform is explored. As examples of applications, rotation-invariant descriptors based on polar and spherical Fourier coefficients are tested on pattern classification problems.