The characterization of phenotypes in cells or organisms from microscopy data largely depends on differences in the spatial distribution of image intensity. Multiple methods exist for quantifying the intensity distribution - or image texture - across objects in natural images. However, many of these texture extraction methods do not directly adapt to 3D microscopy data. Here, we present Spherical Texture extraction, which measures the variance in intensity per angular wavelength by calculating the Spherical Harmonics or Fourier power spectrum of a spherical or circular projection of the angular mean intensity of the object. This method provides a 20-value characterization that quantifies the scale of features in the spherical projection of the intensity distribution, giving a different signal if the intensity is, for example, clustered in parts of the volume or spread across the entire volume. We apply this method to different systems and demonstrate its ability to describe various biological problems through feature extraction. The Spherical Texture extraction characterizes biologically defined gene expression patterns in Drosophila melanogaster embryos, giving a quantitative read-out for pattern formation. Our method can also quantify morphological differences in Caenorhabditis elegans germline nuclei, which lack a predefined pattern. We show that the classification of germline nuclei using their Spherical Texture outperforms a convolutional neural net when training data is limited. Additionally, we use a similar pipeline on 2D cell migration data to extract the polarization direction and quantify the alignment of fluorescent markers to the migration direction. We implemented the Spherical Texture method as a plugin in ilastik to provide a parameter-free and data-agnostic application to any segmented 3D or 2D dataset. Additionally, this technique can also be applied through a Python package to provide extra feature extraction for any object classification pipeline or downstream analysis.

We propose a theoretical framework that generalizes simple and fast algorithms for hierarchical agglomerative clustering to weighted graphs with both attractive and repulsive interactions between the nodes. This framework defines GASP, a Generalized Algorithm for Signed graph Partitioning11Code available at: https://github.com/abailoni/GASP, and allows us to explore many combinations of different linkage criteria and cannotlink constraints. We prove the equivalence of existing clustering methods to some of those combinations and introduce new algorithms for combinations that have not been studied before. We study both theoretical and empirical properties of these combinations and prove that some of these define an ultrametric on the graph. We conduct a systematic comparison of various instantiations of GASP on a large variety of both synthetic and existing signed clustering problems, in terms of accuracy but also efficiency and robustness to noise. Lastly, we show that some of the algorithms included in our framework, when combined with the predictions from a CNN model, result in a simple bottom-up instance segmentation pipeline. Going all the way from pixels to final segments with a simple procedure, we achieve state-of-the-art accuracy on the CREMI 2016 EM segmentation benchmark without requiring domain-specific superpixels.

Most state-of-the-art instance segmentation methods have to be trained on densely annotated images. While difficult in general, this requirement is especially daunting for biomedical images, where domain expertise is often required for annotation and no large public data collections are available for pre-training. We propose to address the dense annotation bottleneck by introducing a proposal-free segmentation approach based on non-spatial embeddings, which exploits the structure of the learned embedding space to extract individual instances in a differentiable way. The segmentation loss can then be applied directly to instances and the overall pipeline can be trained in a fully-or weakly supervised manner. We consider the challenging case of positive-unlabeled supervision, where a novel self-supervised consistency loss is introduced for the unlabeled parts of the training data. We evaluate the proposed method on 2D and 3D segmentation problems in different microscopy modalities as well as on the Cityscapes and CVPPP instance segmentation benchmarks, achieving state-of-the-art results on the latter.

Image partitioning, or segmentation without semantics, is the task of decomposing an image into distinct segments, or equivalently to detect closed contours. Most prior work either requires seeds, one per segment; or a threshold; or formulates the task as multicut / correlation clustering, an NP-hard problem. Here, we propose an efficient algorithm for graph partitioning, the "Mutex Watershed". Unlike seeded watershed, the algorithm can accommodate not only attractive but also repulsive cues, allowing it to find a previously unspecified number of segments without the need for explicit seeds or a tunable threshold. We also prove that this simple algorithm solves to global optimality an objective function that is intimately related to the multicut / correlation clustering integer linear programming formulation. The algorithm is deterministic, very simple to implement, and has empirically linearithmic complexity. When presented with short-range attractive and long-range repulsive cues from a deep neural network, the Mutex Watershed gives the best results currently known for the competitive ISBI 2012 EM segmentation benchmark.

We present a novel approach to the problem of neuron segmentation in image volumes acquired by an electron microscopy. Existing methods, such as agglomerative or correlation clustering, rely solely on boundary evidence and have problems where such an evidence is lacking (e.g., incomplete staining) or ambiguous (e.g., co-located cell and mitochondria membranes). We investigate if these difficulties can be overcome by means of sparse region appearance cues that differentiate between pre- and postsynaptic neuron segments in mammalian neural tissue. We combine these cues with the traditional boundary evidence in the asymmetric multiway cut (AMWC) model, which simultaneously solves the partitioning and the semantic region labeling problems. We show that AMWC problems over superpixel graphs can be solved to global optimality with a cutting plane approach, and that the introduction of semantic class priors leads to significantly better segmentations.