• We propose a weakly-supervised segmentation method for IHC staining pathological cell images, which only uses positive cell counting labels. • It transforms image level counting results into superpixel level classification results as coarse segmentation masks via the designed adaptive top-k instance selection strategy. • It further generates refined pixel-wise cell segmentation masks via the proposed superpixel mask refining approach based on IHC prior knowledge. • Experiments on two public datasets show that our method performs excellently in cell counting and segmentation tasks, and keeps strong generalization. Immunohistochemistry (IHC) can specifically stain immune cells, and quantitative analysis of tumor-infiltrating immune cells is crucial for the diagnosis of malignant tumors and the prognosis of immunotherapy. However, for current deep learning-based immuno-quantitative methods, it is difficult to obtain labeled data for fine-grained analysis tasks, such as cell segmentation, because manual labeling is laborious. Moreover, information contained in coarse-grained image-level labels, such as positive cell counting labels, is difficult to leverage. Here, we propose a novel multi-stage automated immuno-quantitative analysis model for cell counting and segmentation, which is solely supervised by positive cell counting labels. In order to excavate the potential information of counting labels for cell segmentation, based on the idea of Multiple Instance Learning (MIL), we transform entire image (bag) level counting results into superpixel (instance) level classification results as coarse segmentation masks, using a specifically designed adaptive top-k instance selection strategy. Furthermore, a mask refinement approach is proposed, which can refine coarse masks generated by superpixel instance classification into pixel-level pseudo masks as supervision of the segmentation network. Our method enables all-in-one immuno-quantitative analysis of cell counting and segmentation with only positive cell counting labels, alleviating the dependence on fine-grained labels and economizing labeling cost. Our work is the first to apply MIL to small-scale IHC-stained cell image segmentation task. We trained and tested our model on the public LYSTO dataset and conducted generalization tests on the NuClick IHC dataset. Our model achieved segmentation dice scores of 0.809 and 0.711, respectively, demonstrating approximately 7.44% and 3.95% performance improvements over the state-of-the-art (SOTA) methods. The experiments show that our model performs excellently in cell counting and cell segmentation tasks while maintaining strong generalization capabilities.

Many studies utilize dual-pixel (DP) sensor phase characteristics for various applications, such as depth estimation and deblurring. However, since the DP image features are entirely determined by the camera hardware, DP-depth paired datasets are very scarce, especially when performing depth estimation on customized cameras. To overcome this, studies simulate DP images using ideal optical system models. However, these simulations often violate real optical propagation laws, leading to poor generalization to real DP data. To address this, we investigate the domain gap between simulated and real DP data, and propose solutions using the Simulating DP images from ray tracing (Sdirt) scheme. The Sdirt generates realistic DP images via ray tracing and integrates them into the depth estimation training pipeline. Experimental results show that models trained with Sdirt-simulated images generalize better to real DP data. The code and collected datasets will be available at github.com/LinYark/Sdirt

Cervical cytopathology image refocusing is important for addressing the problem of defocus blur in whole slide images. However, most of current deblurring methods are developed for global motion blur instead of local defocus blur and need a lot of supervised re-training for unseen domains. In this paper, we propose a refocusing method for cervical cytopathology images via multi-scale attention features and domain normalization. Our method consists of a domain normalization net (DNN) and a refocusing net (RFN). In DNN, we adopt registration-free cycle scheme for normalizing the unseen unsupervised domains into the seen supervised domain and introduce gray mask loss and hue-encoding mask loss to ensure the consistency of cell structure and basic hue. In RFN, combining the locality and sparseness characteristics of defocus blur, we design a multi-scale refocusing network to enhance the reconstruction of cell nucleus and cytoplasm, and introduce defocus intensity estimation mask to strengthen the reconstruction of local blur. We integrate hybrid learning strategy on the supervised and unsupervised domains to make RFN achieving well refocusing on the unsupervised domain. We build a cervical cytopathology image refocusing dataset and conduct extensive experiments to demonstrate the superiority of our method compared with current deblurring state-of-the-art models. Furthermore, we prove that the refocused images help improve the performance of subsequent high-level analysis tasks. We release the refocusing dataset and source codes to promote the development of this field.

Neuron tracing from optical image is critical in understanding brain function in diseases. A key problem is to trace discontinuous filamentary structures from noisy background, which is commonly encountered in neuronal and some medical images. Broken traces lead to cumulative topological errors, and current methods were hard to assemble various fragmentary traces for correct connection. In this paper, we propose a graph connectivity theoretical method for precise filamentary structure tracing in neuron image. First, we build the initial subgraphs of signals via a region-to-region based tracing method on CNN predicted probability. CNN technique removes noise interference, whereas its prediction for some elongated fragments is still incomplete. Second, we reformulate the global connection problem of individual or fragmented subgraphs under heuristic graph restrictions as a dynamic linear programming function via minimizing graph connectivity cost, where the connected cost of breakpoints are calculated using their probability strength via minimum cost path. Experimental results on challenging neuronal images proved that the proposed method outperformed existing methods and achieved similar results of manual tracing, even in some complex discontinuous issues. Performances on vessel images indicate the potential of the method for some other tubular objects tracing.

Biomedical microscopy images with high-resolution (HR) and axial information can help analysis and diagnosis. However, obtaining such images usually takes more time and economic costs, which makes it impractical in most scenarios. In this paper, we first propose a novel Self-texture Transfer Super-resolution and Refocusing Network (STSRNet) to reconstruct HR multi-focal plane (MFP) images from a single 2D low-resolution (LR) wide field image without relying on scanning or any special devices. The proposed STSRNet consists of three parts: the backbone module for extracting features, the self-texture transfer module for transferring and fusing features, and the flexible reconstruction module for SR and refocusing. Specifically, the self-texture transfer module is designed for images with self-similarity such as cytological images and it searches for similar textures within the image and transfers to help MFP reconstruction. As for reconstruction module, it is composed of multiple pluggable components, each of which is responsible for a specific focal plane, so as to performs SR and refocusing all focal planes at one time to reduce computation. We conduct extensive experiments on cytological images and the experiments show that MFP images reconstructed by STSRNet have richer details in the axial and horizontal directions than input images. At the same time, the reconstructed MFP images also perform better than single 2D wide field images on high-level tasks. The proposed method provides relatively high-quality MFP images when real MFP images cannot be obtained, which greatly expands the application potential of LR wide-field images. To further promote the development of this field, we released our cytology dataset named RSDC for more researchers to use.

In the cytopathology screening of cervical cancer, high-resolution digital cytopathological slides are critical for the interpretation of lesion cells. However, the acquisition of high-resolution digital slides requires high-end imaging equipment and long scanning time. In the study, we propose a GAN-based progressive multi-supervised super-resolution model called PathSRGAN (pathology super-resolution GAN) to learn the mapping of real low-resolution and high-resolution cytopathological images. With respect to the characteristics of cytopathological images, we design a new two-stage generator architecture with two supervision terms. The generator of the first stage corresponds to a densely-connected U-Net and achieves 4× to 10× super resolution. The generator of the second stage corresponds to a residual-in-residual DenseBlock and achieves 10× to 20× super resolution. The designed generator alleviates the difficulty in learning the mapping from 4× images to 20× images caused by the great numerical aperture difference and generates high quality high-resolution images. We conduct a series of comparison experiments and demonstrate the superiority of PathSRGAN to mainstream CNN-based and GAN-based super-resolution methods in cytopathological images. Simultaneously, the reconstructed high-resolution images by PathSRGAN improve the accuracy of computer-aided diagnosis tasks effectively. It is anticipated that the study will help increase the penetration rate of cytopathology screening in remote and impoverished areas that lack high-end imaging equipment.

Automated detection of cancer metastases in lymph nodes has the potential to improve the assessment of prognosis for patients. To enable fair comparison between the algorithms for this purpose, we set up the CAMELYON17 challenge in conjunction with the IEEE International Symposium on Biomedical Imaging 2017 Conference in Melbourne. Over 300 participants registered on the challenge website, of which 23 teams submitted a total of 37 algorithms before the initial deadline. Participants were provided with 899 whole-slide images (WSIs) for developing their algorithms. The developed algorithms were evaluated based on the test set encompassing 100 patients and 500 WSIs. The evaluation metric used was a quadratic weighted Cohen's kappa. We discuss the algorithmic details of the 10 best pre-conference and two post-conference submissions. All these participants used convolutional neural networks in combination with pre- and postprocessing steps. Algorithms differed mostly in neural network architecture, training strategy, and pre- and postprocessing methodology. Overall, the kappa metric ranged from 0.89 to -0.13 across all submissions. The best results were obtained with pre-trained architectures such as ResNet. Confusion matrix analysis revealed that all participants struggled with reliably identifying isolated tumor cells, the smallest type of metastasis, with detection rates below 40%. Qualitative inspection of the results of the top participants showed categories of false positives, such as nerves or contamination, which could be targeted for further optimization. Last, we show that simple combinations of the top algorithms result in higher kappa metric values than any algorithm individually, with 0.93 for the best combination.

BACKGROUND: Soma localization is an important step in computational neuroscience to map neuronal circuits. However, locating somas from large-scale and complicated datasets is challenging. The challenges primarily originate from the dense distribution of somas, the diversity of soma sizes and the inhomogeneity of image contrast. RESULTS: We proposed a novel localization method based on density-peak clustering. In this method, we introduced two quantities (the local density ρ of each voxel and its minimum distance δ from voxels of higher density) to describe the soma imaging signal, and developed an automatic algorithm to identify the soma positions from the feature space (ρ, δ). Compared with other methods focused on high local density, our method allowed the soma center to be characterized by high local density and large minimum distance. The simulation results indicated that our method had a strong ability to locate the densely positioned somas and strong robustness of the key parameter for the localization. From the analysis of the experimental datasets, we demonstrated that our method was effective at locating somas from large-scale and complicated datasets, and was superior to current state-of-the-art methods for the localization of densely positioned somas. CONCLUSIONS: Our method effectively located somas from large-scale and complicated datasets. Furthermore, we demonstrated the strong robustness of the key parameter for the localization and its effectiveness at a low signal-to-noise ratio (SNR) level. Thus, the method provides an effective tool for the neuroscience community to quantify the spatial distribution of neurons and the morphologies of somas.

In this paper, neurons were cultured on a substrate above a multielectrode array, so the changes of electrophysiological activity patterns during development of the neuronal network or in response to environmental perturbations were monitored. But the complexity of these spontaneous activity patterns is not well understood. In order to solve the problem, a comprehensive method (approximate entropy (ApEn) in combination with a "sliding window" over the data) is introduced to quantify the complexity of four spontaneous activity patterns (sporadic spikes, tonic spikes, pseudobursts, and typical bursts) in cultured hippocampal neuronal networks. The results show that the dynamic curves of ApEn illustrate vivid differences between the four patterns and the values of ApEn fall into different ranges. Among these patterns, the complexity of tonic spikes is the highest while that of pseudobursts is the lowest. This suggests that the proposed method is a valid procedure for tracking the dynamic variation in neuronal signals and can distinguish the different firing patterns of neuronal networks in terms of their complexity.