An efficient and precise diagnosis of retinal diseases is a fundamental goal for auxiliary diagnostic systems in ophthalmology. Inspired by the importance of scattered subtle lesions in manual retinal disease diagnosis, recent research has achieved state-of-the-art performance by mining information related to subtle lesions, including their texture and shape. However, the spatial distribution patterns of subtle lesion areas, which are also crucial in manual diagnosis, have been overlooked in existing research. Neglecting these spatial distribution patterns (e.g., the ring distribution of microaneurysms in diabetic macular edema) may negatively impact the diagnostic process. In this paper, we introduce the Saliency-Image-Graph (SIGraph) network to capture the spatial distribution patterns of lesion areas. We first employ saliency-based perception to identify latent lesion pixels. Subsequently, we propose a novel image-graph block to efficiently capture the global distribution of abundant lesion pixels with minimal information loss. By leveraging additional distribution patterns, SIGraph achieves state-of-the-art performance with at least a 1.5% performance gain across three datasets. Furthermore, ablation studies demonstrate that our image-graph block can be integrated into other visual backbones and effectively boost performance.

Self-supervised learning (SSL) has recently achieved promising performance for 3D medical image analysis tasks. Most current methods follow existing SSL paradigm originally designed for photographic or natural images, which cannot explicitly and thoroughly exploit the intrinsic similar anatomical structures across varying medical images. This may in fact degrade the quality of learned deep representations by maximizing the similarity among features containing spatial misalignment information and different anatomical semantics. In this work, we propose a new self-supervised learning framework, namely Alice, that explicitly fulfills Anatomical invariance modeling and semantic alignment via elaborately combining discriminative and generative objectives. Alice introduces a new contrastive learning strategy which encourages the similarity between views that are diversely mined but with consistent high-level semantics, in order to learn invariant anatomical features. Moreover, we design a conditional anatomical feature alignment module to complement corrupted embeddings with globally matched semantics and inter-patch topology information, conditioned by the distribution of local image content, which permits to create better contrastive pairs. Our extensive quantitative experiments on three 3D medical image analysis tasks demonstrate and validate the performance superiority of Alice, surpassing the previous best SSL counterpart methods and showing promising ability for united representation learning. Codes are available at https://github.com/alibaba-damo-academy/Alice.

Automatic and accurate retinal disease diagnosis is critical to guide proper therapy and prevent potential vision loss. Previous works simply exploit the most discriminative features while ignoring the pathological visual clues of scattered subtle lesions. Therefore, without a comprehensive understanding of features from different lesion regions, they are vulnerable to noise from complex backgrounds and suffer from misclassification failures. In this paper, we address these limitations with a novel saliency-guided abnormality-aware transformer which explicitly captures the correlation between different lesion features from a global perspective with enhanced pathological semantics. The model has several merits. First, we propose a saliency enhancement module (SEM) which adaptively integrates disease related semantics and highlights potentially salient lesion regions. Second, to the best of our knowledge, this is the first work to explore comprehensive lesion feature dependencies via a tailored efficient self-attention. Third, with the saliency enhancement module and abnormality-aware attention, we propose a new variant of Vision Transformer models, called SatFormer, which outperforms the state-of-the-art methods on two public retinal disease classification benchmarks. Ablation study shows that the proposed components can be easily embedded into any Vision Transformers via a plug-and-play manner and effectively boost the performance.

Abstract Graph transformer has achieved remarkable success in graph‐based segmentation tasks. Inspired by this success, we propose a novel method named MeshFormer for applying the graph transformer to the semantic segmentation of high‐resolution meshes. The main challenges are the large data size, the massive model size, and the insufficient extraction of high‐resolution semantic meanings. The large data or model size necessitates unacceptably extensive computational resources, and the insufficient semantic meanings lead to inaccurate segmentation results. MeshFormer addresses these three challenges with three components. First, a boundary‐preserving simplification is introduced to reduce the data size while maintaining the critical high‐resolution information in segmentation boundaries. Second, a Ricci flow‐based clustering algorithm is presented for constructing hierarchical structures of meshes, replacing many convolutions layers for global support with only a few convolutions in hierarchy structures. In this way, the model size can be reduced to an acceptable range. Third, we design a graph transformer with cross‐resolution convolutions, which extracts richer high‐resolution semantic meanings and improves segmentation results over previous methods. Experiments show that MeshFormer achieves gains from 1.0% to 5.8% on artificial and real‐world datasets.

Detecting neurofibrillary tangles is an important procedure in the assessment of the intensity and distribution pattern of hippocampal tau pathology, which are the principal clinical phenotypes associated with Alzheimer’s disease. Existing deep learning based detectors still face a critical obstacle: the difficulty in detecting extremely small objects in high resolution images. In this paper, we propose a deep learning framework, named DeepNFT, which combines the multilevel feature aggregation pyramid network (MFAPN) and the adversarial feature generation module (AFGM) to acquire precise detection results with significantly reduced false positives. To prove its universality and robustness, DeepNFT has been validated on two datasets. Experiments show the significant performance gain of our proposed approach over state-of-the-art detectors. Ablation study shows our network components improve the performance of various backbones and detectors.

Accurate and reliable segmentation of colorectal tumors and surrounding colorectal tissues on 3D magnetic resonance images has critical importance in preoperative prediction, staging, and radiotherapy. Previous works simply combine multilevel features without aggregating representative semantic information and without compensating for the loss of spatial information caused by down-sampling. Therefore, they are vulnerable to noise from complex backgrounds and suffer from misclassification and target incompleteness-related failures. In this paper, we address these limitations with a novel adaptive lesion-aware attention network (ALA-Net) which explicitly integrates useful contextual information with spatial details and captures richer feature dependencies based on 3D attention mechanisms. The model comprises two parallel encoding paths. One of these is designed to explore global contextual features and enlarge the receptive field using a recurrent strategy. The other captures sharper object boundaries and the details of small objects that are lost in repeated down-sampling layers. Our lesion-aware attention module adaptively captures long-range semantic dependencies and highlights the most discriminative features, improving semantic consistency and completeness. Furthermore, we introduce a prediction aggregation module to combine multiscale feature maps and to further filter out irrelevant information for precise voxel-wise prediction. Experimental results show that ALA-Net outperforms state-of-the-art methods and inherently generalizes well to other 3D medical images segmentation tasks, providing multiple benefits in terms of target completeness, reduction of false positives, and accurate detection of ambiguous lesion regions.