Designing volume visualizations showing various structures of interest is critical to the exploratory analysis of volumetric data. The last few years have witnessed dramatic advances in the use of convolutional neural networks for identification of objects in large image collections. Whereas such machine learning methods have shown superior performance in a number of applications, their direct use in volume visualization has not yet been explored. In this paper, we present a deep-learning-assisted volume visualization to depict complex structures, which are otherwise challenging for conventional approaches. A significant challenge in designing volume visualizations based on the high-dimensional deep features lies in efficiently handling the immense amount of information that deep-learning methods provide. In this paper, we present a new technique that uses spectral methods to facilitate user interactions with high-dimensional features. We also present a new deep-learning-assisted technique for hierarchically exploring a volumetric dataset. We have validated our approach on two electron microscopy volumes and one magnetic resonance imaging dataset.

Analyzing live-cell images is particularly challenging because cells simultaneously move and undergo systematic changes. Visually inspecting live-cell images therefore involves simultaneously tracking individual cells and detecting relevant spatio-temporal changes. The high cognitive burden of such a complex task makes this kind of analysis inefficient and error prone. In this paper, we describe a deep-learning-assisted visualization based on automatically derived high-level features to identify target cell changes in live-cell images. Applying a novel user-mediated color assignment scheme that maps abstract features into corresponding colors, we create color-based visual annotations that facilitate visual reasoning and analysis of complex time-varying live-cell image datasets.

Modern imaging technologies enable the study of microstructural features, which require capturing the finest details in high-resolution gigapixel images. Nevertheless, the resolution disparity between gigapixel images and megapixel displays presents a challenge to effective visual analysis because subtle texture differences are hardly perceivable at coarser resolutions. In this paper, we present a hierarchical segmentation technique based on the joint distribution of intensity and noise-resistant local binary patterns to differentiate subtle microstructural textures across various scales. The coarse-to-fine segmentation procedure subdivides each parent segment into texturally-distinct child segments at progressively higher resolutions. The hierarchical structure of segments allows creating intermediate segmentation results interactively. Based on the intermediate results, we highlight regions with texture differences using distinct colors, which provide salient visual hints to users despite the current viewing resolution. Our new technique has been validated on large microscopy images and shows promising results.

BACKGROUND: The goal of this survey paper is to overview cellular measurements using optical microscopy imaging followed by automated image segmentation. The cellular measurements of primary interest are taken from mammalian cells and their components. They are denoted as two- or three-dimensional (2D or 3D) image objects of biological interest. In our applications, such cellular measurements are important for understanding cell phenomena, such as cell counts, cell-scaffold interactions, cell colony growth rates, or cell pluripotency stability, as well as for establishing quality metrics for stem cell therapies. In this context, this survey paper is focused on automated segmentation as a software-based measurement leading to quantitative cellular measurements. METHODS: We define the scope of this survey and a classification schema first. Next, all found and manually filteredpublications are classified according to the main categories: (1) objects of interests (or objects to be segmented), (2) imaging modalities, (3) digital data axes, (4) segmentation algorithms, (5) segmentation evaluations, (6) computational hardware platforms used for segmentation acceleration, and (7) object (cellular) measurements. Finally, all classified papers are converted programmatically into a set of hyperlinked web pages with occurrence and co-occurrence statistics of assigned categories. RESULTS: The survey paper presents to a reader: (a) the state-of-the-art overview of published papers about automated segmentation applied to optical microscopy imaging of mammalian cells, (b) a classification of segmentation aspects in the context of cell optical imaging, (c) histogram and co-occurrence summary statistics about cellular measurements, segmentations, segmented objects, segmentation evaluations, and the use of computational platforms for accelerating segmentation execution, and (d) open research problems to pursue. CONCLUSIONS: The novel contributions of this survey paper are: (1) a new type of classification of cellular measurements and automated segmentation, (2) statistics about the published literature, and (3) a web hyperlinked interface to classification statistics of the surveyed papers at https://isg.nist.gov/deepzoomweb/resources/survey/index.html.