Automatic nuclei segmentation and classification (NSC) is a fundamental prerequisite in digital pathology analysis as it enables the quantification of biomarkers and histopathological features for precision medicine. Nuclei appear to be small, however, global spatial distribution and brightness contrast, or color correlation between the nucleus and background, have been recognized as key rationales for accurate nuclei segmentation in actual clinical practice. Although recent great breakthroughs in medical image segmentation have been achieved by Transformer-based methods, the adaptability of segmenting and classifying nuclei from histopathological images is rarely investigated. Also, the severe overlap of nuclei and the large intra-class variability are common in clinical wild data. Prevailing methods based on polygonal representations or distance maps are limited by empirically designed post-processing strategies, resulting in ineffective segmentation of large irregular nuclei instances. To address these challenges, we propose a keypoint-guided tri-decoder Transformer (PointFormer) for NSC simultaneously. Specifically, the overall NSC task is decoupled to a multi-task learning problem, where a tri-decoder structure is employed for decoding nuclei instance, edges, and types, respectively. The nuclei detection and classification (NDC) subtask is reformulated as a semantic keypoint estimation problem. Meanwhile, introduces a novel attention-guiding strategy to capture strong inter-branch correlations and mitigate inconsistencies between multi-decoder predictions. Finally, a multi-local perception module is designed as the base building block of PointFormer to achieve local and global trade-offs and reduce model complexity. Comprehensive quantitative and qualitative experimental results on three datasets of different volumes have demonstrated the superiority of the proposed method over prevalent methods, especially for the PanNuke dataset with an achievement of 70.6% on bPQ.

Nuclei detection and segmentation are indispensable prerequisites in digital pathology research, whereas the precise segmentation of nuclei by domain experts relies heavily on global spatial information and the inter-nuclei correlation. However, previous automatic nuclei segmentation works are mostly built on convolutional neural networks, which are unable to capture long-range global context with their inherent convolutional operations. Additionally, window-based design in few transformer-based approaches limits remote token interactions. The present study introduces a novel Multi-Local Perception (MLP) network, MLPSeg, which is proposed to address the aforementioned challenging issues. Specifically, the parallel computation of depthwise separable convolution and local window attention is designed to extract local information. Then, the parallel module of local horizontal attention and local vertical attention is designed to establish the global dependency. Moreover, to model the cross-scale dependencies and narrow the contextual semantic gap, the Context Cross Attention (CCA) is introduced for optimising skip connections. A tri-decoder structure is adopted to generate nuclei instance masks, normal edge masks and clustered edge masks. The superior performance of MLPSeg for nuclei segmentation is demonstrated across two datasets with different modalities, resulting in a 2.29% - 5.82% improvement compared to the state-of-the-art methods.