Scale-aware image filters are useful in graphics applications, but applying such filters to 3D images has not been well studied and introduces 3D-specific issues. This paper proposes a fast scale-aware 3D image filter that recursively applies an average-based joint filter that alternates its input and guidance images at each recursion. The joint filter transforms each voxel domain via the arc length of the guidance image manifold, and the $$L^{1}$$ Gaussian convolutions are performed efficiently and accurately on the transformed and decomposed sub-domains. Our filter employs a novel 3D half-box filter to avoid corner-missing artifacts, incorporates voxel pitches into the formulations, fixes the convolution order to reduce non-uniform convergence, and introduces a novel convolution kernel to obtain consistent results that avoid distorted edges. The new filter was numerically examined and compared to conventional filters in terms of accuracy, speed, convergence rate, and visual quality, and was found to achieve fast computations while providing high-quality results.

BACKGROUND: To effectively detect and investigate various cell-related diseases, it is essential to understand cell behaviour. The ability to detection mitotic cells is a fundamental step in diagnosing cell-related diseases. Convolutional neural networks (CNNs) have been successfully applied to object detection tasks, however, when applied to mitotic cell detection, most existing methods generate high false-positive rates due to the complex characteristics that differentiate normal cells from mitotic cells. Cell size and orientation variations in each stage make detecting mitotic cells difficult in 2D approaches. Therefore, effective extraction of the spatial and temporal features from mitotic data is an important and challenging task. The computational time required for detection is another major concern for mitotic detection in 4D microscopic images. RESULTS: In this paper, we propose a backbone feature extraction network named full scale connected recurrent deep layer aggregation (RDLA++) for anchor-free mitotic detection. We utilize a 2.5D method that includes 3D spatial information extracted from several 2D images from neighbouring slices that form a multi-stream input. CONCLUSIONS: Our proposed technique addresses the scale variation problem and can efficiently extract spatial and temporal features from 4D microscopic images, resulting in improved detection accuracy and reduced computation time compared with those of other state-of-the-art methods.

Mitosis detection is one of the challenging steps in biomedical imaging research, which can be used to observe the cell behavior. Most of the already existing methods that are applied in detecting mitosis usually contain many nonmitotic events (normal cell and background) in the result (false positives, FPs). In order to address such a problem, in this study, we propose to apply 2.5-dimensional (2.5D) networks called CasDetNet_CLSTM, which can accurately detect mitotic events in 4D microscopic images. This CasDetNet_CLSTM involves a 2.5D faster region-based convolutional neural network (Faster R-CNN) as the first network, and a convolutional long short-term memory (CLSTM) network as the second network. The first network is used to select candidate cells using the information from nearby slices, whereas the second network uses temporal information to eliminate FPs and refine the result of the first network. Our experiment shows that the precision and recall of our networks yield better results than those of other state-of-the-art methods.

Abstract This paper proposes a fast and accurate computational framework for scale-aware image filters. Our framework is based on accurately approximating $$L^{1}$$ L 1 Gaussian convolution with respect to a transformed pixel domain representing geodesic distance on a guidance image manifold in order to recover salient edges in a manner faithful to scale-space theory while removing small image structures. Our framework possesses linear computational complexity with high approximation precision. We examined it numerically in terms of speed, accuracy, and quality compared with conventional methods.

CNN classifiers using microscopy images to pick up induced pluripotent stem (iPS) cells from a lot of non-iPS cells ware proposed to improves efficiency of human iPS cell production. By selecting iPS-forming cells at early culture stage, it should be more promising to reduce a considerable amount of work. This paper proposes a CNN reprogramming stage classifier trained by respective types (grown into high-, medium, and low-growth status of iPS-like cells) of time-lapse images taken from the completed cell culturing process. These trained CNNs are used to classify a time-lapse image taken in cell culturing as one stage (a number of consecutive frames) of a respective set of trained time-lapse images. The differences between the actual stage with respective classified stages are than calculated to decide which stage of the completed cell this time-step image taken in cell culturing is near to, then to predict the iPS cell growth status that the culturing cells finally grow into. The method achieved the 85% accuracy to predict the growth status of culturing cells two days later, thus can be applicable for early selection suggestion of good iPS-forming cells.

Recurrent Neural Network (RNN) operates on sequential data using a recurrent hidden state dependent on previous data. Even proved successful in many fields, RNNs suffer from vanishing and exploding gradient problems. Gating approach of LSTM and GRU solves these problems but lose prediction accuracies if short-term changes occur in time-lapse microscopy images taken in cell culture. This study uses trend training (feed forward and pre-training) based RNN to improve the prediction accuracy for human iPS cell reprogramming and formation in future culture. The feed forward RNN uses the prediction of each time-step in the next recursive batch training for the next prediction recursively to give recurrent RNN internal states approaching optimal. Pre-training gives optimal initial RNN parameters selected from RNN models pre-trained by sets of time-lapse images taken from a completed cell culture process. An example shows the predicted accuracies and iPS cell colony shapes by feed forward RNN are apparently better than those without the feed forward training, the accuracies and iPS cell colony shapes furthermore applying the pre-trained models are better than only using the feed forward training. Thirty experimental results show the proposed method achieved 0.752 (SD 0.137) accuracy in future 72 time-steps (1.5 days).

Mitotic event detection is a fundamental step in investigating of cell behaviors. The event can be used to analyze various diseases, but most mitotic event detections performed previously focused only on two-dimensional (2D) images with time information. Owing to the complex background (normal cells) and mitotic event orientations, the 2D detection methods yield many false positive and false negative results. To solve this problem, we proposed a 2.5 dimensional (2.5D) cascaded end-to-end network combined with 3D anchors for accurate detection of mitotic events in 4D microscopic images. Our proposed network uses a convolutional long short-term memory to handle issues relating to time sequence; this helps to improve the detection accuracy (reduction of false positives). Furthermore, it uses 3D anchors to capture volume information used to address the orientation problem (reduction of false negatives). The experimental results show that the proposed method can achieve higher precision and recall compared with state-of-the-art methods.

This paper proposes an accurate 3D segmentation method for visualization and quantitative analysis of differentiation activities of mouse embryonic stem (ES) cells using time-lapse confocal fluorescence microscopy images. One of critical tasks in ES cell segmentation arises due to that ES cell nuclei are often close to each other. Several segmentation methods by convexities and concavities on cell or nucleus contours to detect possible touching cells or nuclei were proposed. Comparing to image processing methods, these methods are more accurate in some conditions, however, still cannot detect touching nuclei without concavities on nucleus contours. Our method uses the nucleus size and convex, concave, strait and extrusion features on nucleus contour to touch a boundary between touching cell nuclei in 2D slices and interslices. Experimental results show our method can well detect touching boundaries of 2D and 3D nucleus for confocal microscopy images of mouse ES cells in an early stage of differentiating into neural progenitor cells. Based on the accurate ES cell segmentation, cell activities (velocities and shape changes) during differentiation can be accurately visualized and quantitatively analyzed.

The paper proposes a robust plane fitting method for highly corrupted 3D point sets with significant outliers. Our method integrates a sampling strategy of robust regressions with median approximation of the covariation matrix. We numerically examined our method with popular conventional approaches in terms of speed and accuracy, and achieved higher accuracy for various distributions of outliers.

This paper explores the feasibility of automatic 3D segmentation, visualization and quantitative analysis for differentiation activities of mouse embryonic stem cells using time-lapse confocal fluorescence microscopy images. Technical approaches include bilateral filtering, mean-shift segmentation, adaptive thresholding, watershed segmentation, connected component labeling, and video tracking. Our method processes simultaneously two image channels, one for cytoplasm and the other for nuclei. The nucleus images are used to segment 2D and then 3D nuclei and to track each nucleus and calculate velocities of the 3D nucleus. The cytoplasm images are used to help nucleus segmentation and calculate the S/V (surface to volume) ratio of cytoplasm surrounding a nucleus. Volume rendering on the time-lapse fluorescence images generates time-series 3D images for visualizing the dynamic changes of cell velocity and S/V ratios. Using our prototype system, cells with different amount of EGFP fluorescent protein possesses different differentiation activity (velocity and S/V ratio) can be visualized and quantitatively analyzed.

We present an automatic image processing and visualization method to quantitatively analyze kinematics, proliferation and attachment of mouse embryonic stem (mES) cells using time-series confocal time-lapse fluorescence microscopy images. An automatic method is presented to determine the 3D boundary of each cell nucleus in each cell colony. The cells and colonies are then tracked among the time-series images to determine the kinematics, proliferation and attachment of the cells and colonies. The cells and colonies are visualized through a 3D interface, and the kinematics, proliferation and attachment are illustrated in tree structures. The information of cell kinematics, proliferation and attachment indicates how the culturing conditions and cell positions affect the kinematics, proliferation and attachment. The implementation results show that the automatic method can successfully analyze the cell kinematics, proliferation and attachment, thereby yield a potential tool for helping mES cell culture.

In this paper, we propose a novel framework of analyzing the internal micro-structure of steel materials. Our framework consists of destructive imaging and geometric computing techniques. Imaging is based on a high-precision sectioning tool, and optical imaging with sub-micrometer resolution. We adapt geometry processing methods for segmented multi-material volumes to obtain the structural features. We use our framework to image inclusions and cracks of ball-bearing steels. Our analysis indicates an interesting relationship between the concavity of inclusion shapes and crack initiation.

This paper proposes a fast and accurate approximation algorithm to convolve a L1Gaussian function with images. Our new algorithm is based on splitting a pixel domain into representative regions where we can efficiently perform discrete convolutions. Our algorithm is applicable to non-uniform pixels with linear computational complexity. We examine it numerically in terms of speed, precision, and quality. We also introduce a novel edge-aware filter by using our algorithm.

This paper presents a novel three dimensional (3D) flower modeling technique that utilizes an X-ray computed tomography (CT) system and real-world flowers. Although a CT system provides volume data that captures the internal structures of flowers, it is difficult to accurately segment them into regions of particular organs and model them as smooth surfaces because a flower consists of thin organs that contact one another. We thus introduce a semi-automatic modeling technique that is based on a new active contour model with energy functionals designed for flower CT. Our key idea is to approximate flower components by two important primitives, a shaft and a sheet. Based on our active contour model, we also provide novel user interfaces and a numerical scheme to fit these primitives so as to reconstruct realistic thin flower organs efficiently. To demonstrate the feasibility of our technique, we provide various flower models reconstructed from CT volumes.

Abstract In this paper, we propose a novel contour‐based volume image segmentation technique. Our technique is based on an implicit surface reconstruction strategy, whereby a signed scalar field is generated from user‐specified contours. The key idea is to compute the scalar field in a joint spatial‐range domain (i.e., bilateral domain) and resample its values on an image manifold. We introduce a new formulation of Hermite radial basis function (HRBF) interpolation to obtain the scalar field in the bilateral domain. In contrast to previous implicit methods, bilateral HRBF (B‐HRBF) generates a segmentation boundary that passes through all contours, fits high‐contrast image edges if they exist, and has a smooth shape in blurred areas of images. We also propose an acceleration scheme for computing B‐HRBF to support a real‐time and intuitive segmentation interface. In our experiments, we achieved high‐quality segmentation results for regions of interest with high‐contrast edges and blurred boundaries.

Abstract Medical volume images contain ambiguous and low‐contrast boundaries around which existing fully‐ or semiautomatic segmentation algorithms often cause errors. In this paper, we propose a novel system for intuitively and efficiently refining medical volume segmentation by modifying multiple curved contours. Starting with segmentation data obtained using any existing algorithm, the user places a three‐dimensional curved cross‐section and contours of the foreground region by drawing a cut stroke, and then modifies the contours referring to the cross‐section. The modified contours are used as constraints for deforming a boundary surface that envelops the foreground region, and the region is updated by that deformed boundary. Our surface deformation algorithm seamlessly integrates detail‐preserving and curvature‐diffusing methods to keep important detail boundary features intact while obtaining smooth surfaces around unimportant boundary regions. Our system supports topological manipulations as well as contour shape modifications. We illustrate the feasibility of our system by providing examples of its application to the extraction of bones, muscles, kidneys with blood vessels, and bowels.

Abstract In spite of high computational complexity, the bilateral filter and its modifications and extensions have recently become very popular image and shape processing tools. In this paper, we propose a fast and accurate approximation of the bilateral filter. Our approach combines a dimension elevation trick with a Fast Gauss Transform. First we represent the bilateral filter as a convolution in a high dimensional space. Then the convolution is efficiently approximated by using space partitioning and Gaussian function expansions. Advantages of our approach include linear computational complexity, user‐specified precision, and an ability to process high dimensional and non‐uniformly sampled data. We demonstrate capabilities of the approach by considering its applications to the image and volume denoising and high‐dynamic‐range tone mapping problems.

We propose a novel approach to 3D image registration of intracellular volumes. The approach extends a standard image registration framework to the curved cell geometry. An intracellular volume is mapped onto another intracellular domain by using two pairs of point set surfaces approximating their nuclear and plasma membranes. The mapping function consists of the affine transformation, tetrahedral barycentric interpolation, and least-squares formulation of radial basis functions for extracted cell geometry features. An interactive volume registration system is also developed based on our approach. We demonstrate that our approach is capable of creating cell models containing multiple organelles from observed data of living cells.

This paper deals with the problem of intracellular image segmentation. Our goal is to propose an algorithm selection framework that has the potential to be general enough to be used for a variety of intracellular image segmentation tasks. With this framework, an optimal algorithm suited to each segmentation task can be selected automatically by our proposed evaluation criteria derived from region similarity of image features and boundary shape. Furthermore, using our framework, we can rank different algorithms, as well as define each algorithm's parameters. We tested our prototype framework on confocal microscope images and showed that application of these criteria gave highly accurate segmentation results without missing any biologically important image characteristics.

The paper proposes a novel 3D image registration approach to calculating intracellular volumes. The approach extends a standard image registration framework to the curved cell geometry. An intracellular volume is mapped onto another intracellular domain by using two pairs of point set surfaces approximating their nuclear and plasma membranes. The approach is implemented to an interactive volume registration system. We demonstrate that our approach can create models of cells with multiple organelles from data collected from living cells.

Abstract Animations of characters with flexible bodies such as jellyfish, snails, and, hearts are difficult to design using traditional skeleton‐based approaches. A standard approach is keyframing, but adjusting the shape of the flexible body for each key frame is tedious. In addition, the character cannot dynamically adjust its motion to respond to the environment or user input. This paper introduces a new procedural deformation framework (ProcDef) for designing and driving animations of such flexible objects. Our approach is to synthesize global motions procedurally by integrating local deformations. ProcDef provides an efficient design scheme for local deformation patterns; the user can control the orientation and magnitude of local deformations as well as the propagation of deformation signals by specifying line charts and volumetric fields. We also present a fast and robust deformation algorithm based on shape‐matching dynamics and show some example animations to illustrate the feasibility of our framework.

A new geometry-based finite difference method for a fast and reliable detection of perceptually salient curvature extrema on surfaces approximated by dense triangle meshes is proposed. The foundations of the method are two simple curvature and curvature derivative formulas overlooked in modern differential geometry textbooks and seemingly new observation about inversion-invariant local surface-based differential forms.