In learning from label proportions (LLP), the instances are grouped into bags, and the task is to learn an instance classifier given relative class proportions in training bags. LLP is useful when obtaining individual instance labels is impossible or costly.In this work, we focus on the case of small bags, which allows to design an algorithm that explicitly considers all consistent instance label combinations. In particular, we propose an EM algorithm alternating between optimizing a general neural network instance classifier and incorporating bag-level annotations. Using two different image datasets, we experimentally compare this method with an approach based on normal approximation and two existing LLP methods. The results show that our approach converges faster to a comparable or better solution.

Automatic Non-rigid Histological Image Registration (ANHIR) challenge was organized to compare the performance of image registration algorithms on several kinds of microscopy histology images in a fair and independent manner. We have assembled 8 datasets, containing 355 images with 18 different stains, resulting in 481 image pairs to be registered. Registration accuracy was evaluated using manually placed landmarks. In total, 256 teams registered for the challenge, 10 submitted the results, and 6 participated in the workshop. Here, we present the results of 7 well-performing methods from the challenge together with 6 well-known existing methods. The best methods used coarse but robust initial alignment, followed by non-rigid registration, used multiresolution, and were carefully tuned for the data at hand. They outperformed off-the-shelf methods, mostly by being more robust. The best methods could successfully register over 98% of all landmarks and their mean landmark registration accuracy (TRE) was 0.44% of the image diagonal. The challenge remains open to submissions and all images are available for download.

Image registration is a common task for many biomedical analysis applications. The present work focuses on the benchmarking of registration methods on differently stained histological slides. This is a challenging task due to the differences in the appearance model, the repetitive texture of the details and the large image size, between other issues. Our benchmarking data is composed of 616 image pairs at two different scales - average image diagonal 2.4k and 5k pixels. We compare eleven fully automatic registration methods covering the widely used similarity measures (and optimization strategies with both linear and elastic transformation). For each method, the best parameter configuration is found and subsequently applied to all the image pairs. The performance of the algorithms is evaluated from several perspectives - the registrations (in)accuracy on manually annotated landmarks, the method robustness and its processing computation time.

We propose a novel approach to reconstructing curvilinear tree structures evolving over time, such as road networks in 2D aerial images or neural structures in 3D microscopy stacks acquired in vivo. To enforce temporal consistency, we simultaneously process all images in a sequence, as opposed to reconstructing structures of interest in each image independently. We formulate the problem as a Quadratic Mixed Integer Program and demonstrate the additional robustness that comes from using all available visual clues at once, instead of working frame by frame. Furthermore, when the linear structures undergo local changes over time, our approach automatically detects them.

We present an efficient matching method for generalized geometric graphs. Such graphs consist of vertices in space connected by curves and can represent many real world structures such as road networks in remote sensing, or vessel networks in medical imaging. Graph matching can be used for very fast and possibly multimodal registration of images of these structures. We formulate the matching problem as a single player game solved using Monte Carlo Tree Search, which automatically balances exploring new possible matches and extending existing matches. Our method can handle partial matches, topological differences, geometrical distortion, does not use appearance information and does not require an initial alignment. Moreover, our method is very efficient-it can match graphs with thousands of nodes, which is an order of magnitude better than the best competing method, and the matching only takes a few seconds.

Multiple myeloma is a disease primarily affecting bone marrow. This paper presents a novel method that can automatically detect infiltration of the bone marrow by multiple myeloma in diseased femurs (thigh bones) from low-dose CT images. This detection is done by evaluating two aspects of the CT images: bone marrow infiltrations (increased density values compared to regular fatty bone marrow) and scalloping (indentation of the inner margin of the cortical bone). First, bone marrow and cortical bone are automatically segmented from CT images. Afterwards, a probabilistic model of the bone marrow density is created in order to automatically detect islands of infiltrations. Finally, scalloping is detected by means of the quantification of the roughness of the boundary between the bone marrow and bony tissue. We have experimentally tested both infiltration and scalloping detection methods, obtaining a sensitivity of 74.9% and 69.2% and a specificity of 75% and 61% respectively.

Studies concerning gene expression patterns of Drosophila are of paramount importance in basic biological research because many genes are conserved across organisms providing information of fundamental activity. However, mapping a gene requires analyzing hundreds of objects that have been segmented previously. Hence, a reliable segmentation is a crucial step. Here, we introduce the concept of supertextons and propose a novel segmentation procedure for localized Drosophila ovaries. First, a pre-segmentation step is performed using superpixels; each superpixel that belongs to a single class is transform into a feature vector. Then, a dictionary is built by clustering representative feature vectors per class, such clusters are called supertextons. Finally, during the classification stage, new superpixels are assigned to certain classes using the k-NN classifier and the supertexton dictionary. This proposal has been applied to segmentation of cells in Drosophila oogenesis where the results have shown the effectiveness of our approach.

Many medical images contain graph-like geometrical structures such as blood vessels and neuronal networks. We present an algorithm for matching geometrical graphs, in order to quickly and robustly align such images. We use a sampling-based curve descriptor to prune dissimilar edges. The matching is modeled as a single-player game, growing the matching from a random initial correspondence. The coarse global solution is found using a Monte Carlo Tree Search and then refined locally. We show experimentally that our approach finds the correct matching in all tested datasets and is the fastest of all global methods.

We present a new approach for matching sets of branching curvilinear structures that form graphs embedded in R2 or R3 and may be subject to deformations. Unlike earlier methods, ours does not rely on local appearance similarity nor does require a good initial alignment. Furthermore, it can cope with non-linear deformations, topological differences, and partial graphs. To handle arbitrary non-linear deformations, we use Gaussian process regressions to represent the geometrical mapping relating the two graphs. In the absence of appearance information, we iteratively establish correspondences between points, update the mapping accordingly, and use it to estimate where to find the most likely correspondences that will be used in the next step. To make the computation tractable for large graphs, the set of new potential matches considered at each iteration is not selected at random as with many RANSAC-based algorithms. Instead, we introduce a so-called Active Testing Search strategy that performs a priority search to favor the most likely matches and speed-up the process. We demonstrate the effectiveness of our approach first on synthetic cases and then on angiography data, retinal fundus images, and microscopy image stacks acquired at very different resolutions.

We propose an approach to reconstructing tree structures that evolve over time in 2D images and 3D image stacks such as neuronal axons or plant branches. Instead of reconstructing structures in each image independently, we do so for all images simultaneously to take advantage of temporal-consistency constraints. We show that this problem can be formulated as a Quadratic Mixed Integer Program and solved efficiently. The outcome of our approach is a framework that provides substantial improvements in reconstructions over traditional single time-instance formulations. Furthermore, an added benefit of our approach is the ability to automatically detect places where significant changes have occurred over time, which is challenging when considering large amounts of data.

We present a new approach to matching graphs embedded in ℝ2or ℝ3. Unlike earlier methods, our approach does not rely on the similarity of local appearance features, does not require an initial alignment, can handle partial matches, and can cope with non-linear deformations and topological differences. To handle arbitrary non-linear deformations, we represent them as Gaussian Processes. In the absence of appearance information, we iteratively establish correspondences between graph nodes, update the structure accordingly, and use the current mapping estimate to find the most likely correspondences that will be used in the next iteration. This makes the computation tractable. We demonstrate the effectiveness of our approach first on synthetic cases and then on angiography data, retinal fundus images, and microscopy image stacks acquired at very different resolutions.

We address the problem of estimating the uncertainty of pixel based image registration algorithms, given just the two images to be registered, for cases when no ground truth data is available. Our novel method uses bootstrap resampling. It is very general, applicable to almost any registration method based on minimizing a pixel-based similarity criterion; we demonstrate it using the SSD, SAD, correlation, and mutual information criteria. We show experimentally that the bootstrap method provides better estimates of the registration accuracy than the state-of-the-art CramEr-Rao bound method. Additionally, we evaluate also a fast registration accuracy estimation (FRAE) method which is based on quadratic sensitivity analysis ideas and has a negligible computational overhead. FRAE mostly works better than the CramEr-Rao bound method but is outperformed by the bootstrap method.

Fast level set methods replace continuous PDEs by a discrete formulation, improving the execution times. The regularization in fast level set methods was so far handled indirectly via level set function smoothing. We propose to incorporate standard curvature based regularization into fast level set methods and address the problem of efficiently estimating local curvature of a discretized interface in 2D or 3D based on local partial volume. We present two algorithms for incremental partial volume evaluation: the first is recommended for moderate neighborhood sizes, the second has an excellent asymptotic complexity and can be useful for very large neighborhoods. The performance of the proposed methods is compared experimentally with previous approaches.

In this paper, a 2-D locally regularized strain estimation method for imaging deformation of soft biological tissues from radio-frequency (RF) ultrasound (US) data is introduced. Contrary to most 2-D techniques that model the compression-induced local displacement as a 2-D shift, our algorithm also considers a local scaling factor in the axial direction. This direction-dependent model of tissue motion and deformation is induced by the highly anisotropic resolution of RF US images. Optimal parameters are computed through the constrained maximization of a similarity criterion defined as the normalized correlation coefficient. Its value at the solution is then used as an indicator of estimation reliability, the probability of correct estimation increasing with the correlation value. In case of correlation loss, the estimation integrates an additional constraint, imposing local continuity within displacement and strain fields. Using local scaling factors and regularization increase the method's robustness with regard to decorrelation noise, resulting in a wider range of precise measurements. Results on simulated US data from a mechanically homogeneous medium subjected to successive uniaxial loadings demonstrate that our method is theoretically able to accurately estimate strains up to 17%. Experimental strain images of phantom and cut specimens of bovine liver clearly show the harder inclusions.

We present an algorithm for estimating entropy from high-dimensional data based on Kozachenko-Leonenko nearest neighbor estimator. The problem of finding all nearest neighbors is approximately solved using a best-bin first (BBF) bottom-up k-D tree traversal. Our main application is evaluating higher-order mutual information (MI) image similarity criteria that, unlike standard scalar MI, are directly usable for vector (e.g. color) images and can take into account neighborhood information. As during the optimization the MI criterion is often evaluated for very similar images, it is advantageous to update the k-D tree incrementally. We show that the resulting algorithm is fast and accurate enough to be practical for the image registration application

The goal of quantitative analysis of microarrays is to determine the strength of the hybridization for every element of the array (spot). Since new technology allows the detection of the signal at single molecule level, new methods for analysis are necessary. A detection error of 10% is considered acceptable. In this paper we discuss three approaches to single peak detection inside the spots of the arrays, and compare the results we obtain on simulated and real images. These approaches are: global thresholding, an adaptive filter combined with local thresholding. We proposed a third algorithm, a statistical estimation of the background combined with clustering, which produces comparable results to well known algorithms, without having to perform the manual adjustment of the parameters.

The forward electroencephalography (EEG) problem involves finding a potential V from the Poisson equation inverted Delta x (sigma inverted Delta V) f, in which f represents electrical sources in the brain, and sigma the conductivity of the head tissues. In the piecewise constant conductivity head model, this can be accomplished by the boundary element method (BEM) using a suitable integral formulation. Most previous work uses the same integral formulation, corresponding to a double-layer potential. In this paper we present a conceptual framework based on a well-known theorem (Theorem 1) that characterizes harmonic functions defined on the complement of a bounded smooth surface. This theorem says that such harmonic functions are completely defined by their values and those of their normal derivatives on this surface. It allows us to cast the previous BEM approaches in a unified setting and to develop two new approaches corresponding to different ways of exploiting the same theorem. Specifically, we first present a dual approach which involves a single-layer potential. Then, we propose a symmetric formulation, which combines single- and double-layer potentials, and which is new to the field of EEG, although it has been applied to other problems in electromagnetism. The three methods have been evaluated numerically using a spherical geometry with known analytical solution, and the symmetric formulation achieves a significantly higher accuracy than the alternative methods. Additionally, we present results with realistically shaped meshes. Beside providing a better understanding of the foundations of BEM methods, our approach appears to lead also to more efficient algorithms.

We propose a new spatio-temporal elastic registration algorithm for motion reconstruction from a series of images. The specific application is to estimate displacement fields from two-dimensional ultrasound sequences of the heart. The basic idea is to find a spatio-temporal deformation field that effectively compensates for the motion by minimizing a difference with respect to a reference frame. The key feature of our method is the use of a semi-local spatio-temporal parametric model for the deformation using splines, and the reformulation of the registration task as a global optimization problem. The scale of the spline model controls the smoothness of the displacement field. Our algorithm uses a multiresolution optimization strategy to obtain a higher speed and robustness. We evaluated the accuracy of our algorithm using a synthetic sequence generated with an ultrasound simulation package, together with a realistic cardiac motion model. We compared our new global multiframe approach with a previous method based on pairwise registration of consecutive frames to demonstrate the benefits of introducing temporal consistency. Finally, we applied the algorithm to the regional analysis of the left ventricle. Displacement and strain parameters were evaluated showing significant differences between the normal and pathological segments, thereby illustrating the clinical applicability of our method.

We present an algorithm for fast elastic multidimensional intensity-based image registration with a parametric model of the deformation. It is fully automatic in its default mode of operation. In the case of hard real-world problems, it is capable of accepting expert hints in the form of soft landmark constraints. Much fewer landmarks are needed and the results are far superior compared to pure landmark registration. Particular attention has been paid to the factors influencing the speed of this algorithm. The B-spline deformation model is shown to be computationally more efficient than other alternatives. The algorithm has been successfully used for several two-dimensional (2-D) and three-dimensional (3-D) registration tasks in the medical domain, involving MRI, SPECT, CT, and ultrasound image modalities. We also present experiments in a controlled environment, permitting an exact evaluation of the registration accuracy. Test deformations are generated automatically using a random hierarchical fractional wavelet-based generator.

We present an algorithm for nonlinear multidimensional registration. The correspondence function is represented in a spline space. We also use cubic splines to interpolate the volumetric data to be registered. We use a multiresolution strategy, which gives us robustness and additional speedup. We analyze the computational complexity of the algorithm and its performance using different multidimensional optimization methods. Finally, we present an application of the algorithm for registering technetium ethylene cysteine diethylester (ECD) SPECT images for realignment of corresponding Xe inhalation SPECT images.

Echo-planar imaging (EPI) is a fast nuclear magnetic resonance imaging (MRI) method. Unfortunately, local magnetic field inhomogeneities induced mainly by the subject's presence cause significant geometrical distortion, predominantly along the phase-encoding direction, which must be undone to allow for meaningful further processing. So far, this aspect has been too often neglected. In this paper, we suggest a new approach using an algorithm specifically developed for the automatic registration of distorted EPI images with corresponding anatomically correct MRI images. We model the deformation field with splines, which gives us a great deal of flexibility, while comprising the affine transform as a special case. The registration criterion is least squares. Interestingly, the complexity of its evaluation does not depend on the resolution of the control grid. The spline model gives us good accuracy thanks to its high approximation order. The short support of splines leads to a fast algorithm. A multiresolution approach yields robustness and additional speedup. The algorithm was tested on real as well as synthetic data, and the results were compared with a manual method. A wavelet-based Sobolev-type random deformation generator was developed for testing purposes. A blind test indicates that the proposed automatic method is faster, more reliable, and more precise than the manual one.

Due to magnetic field inhomogeneities, EPI images are geometrically distorted, predominantly along the phase-encoding direction. Currently, the distortion is either ignored or compensated manually using a warping function defined through a set of landmarks. We propose an automatic method to unwarp the geometric distortion of EPI images by registering them with corresponding undistorted anatomical MRI images. We show that cubic splines are optimal interpolating functions for both landmark interpolation and approximation. We will consequently use the same warping space in our algorithm which replaces landmarks by an image difference criterion. B-splines are used as generating functions, which leads to a fast and accurate computation. Multiresolution gives robustness and additional speedup. The algorithm performance was evaluated using both real and synthetic data and was found superior to the manual method.