The Segment Anything Model (SAM) has demonstrated remarkable capabilities in its performance on natural images. However, it faces considerable challenges when applied to medical datasets. Specifically, the performance of vanilla SAM is degraded and lacks generalisability when processing medical images with large domain gaps. What’s worse, many medical segmentation tasks highly demand accurate boundary identification, while existing SAM variants struggle with this need. To overcome the above challenges, we propose BA-SAM, a Segment Anything Model variant that can achieve better performance on medical images. Specifically, based on the idea of parameter-efficient fine-tuning (PEFT), we first add a parallel tuneable CNN encoder to better extract local details using convolutional operations, while most parts of the original ViT encoder in SAM are set frozen. Moreover, we use a Boundary-Aware Attention (BAA) module in the CNN-Branch to encourage the framework to better capture boundary-related features. Extensive experiments on three public datasets demonstrate that the proposed BA-SAM further improvements over existing state-of-the-art methods.

Many of the tissues/lesions in the medical images may be ambiguous. Therefore, medical segmentation is typically annotated by a group of clinical experts to mitigate personal bias. A common solution to fuse different annotations is the majority vote, e.g., taking the average of multiple labels. However, such a strategy ignores the difference between the grader expertness. Inspired by the observation that medical image segmentation is usually used to assist the disease diagnosis in clinical practice, we propose the diagnosis-first principle, which is to take disease diagnosis as the criterion to calibrate the inter-observer segmentation uncertainty. Following this idea, a framework named Diagnosis-First segmentation Framework (DiFF) is proposed. Specifically, DiFF will first learn to fuse the multi-rater segmentation labels to a single ground-truth which could maximize the disease diagnosis performance. We dubbed the fused ground-truth as Diagnosis-First Ground-truth (DF-GT). Then, the Take and Give Model (T&G Model) to segment DF-GT from the raw image is proposed. With the T&G Model, DiFF can learn the segmentation with the calibrated uncertainty that facilitate the disease diagnosis. We verify the effectiveness of DiFF on three different medical segmentation tasks: optic-disc/optic-cup (OD/OC) segmentation on fundus images, thyroid nodule segmentation on ultrasound images, and skin lesion segmentation on dermoscopic images. Experimental results show that the proposed DiFF can effectively calibrate the segmentation uncertainty, and thus significantly facilitate the corresponding disease diagnosis, which outperforms previous state-of-the-art multi-rater learning methods.

Age-related macular degeneration (AMD) is the leading cause of visual impairment among elderly in the world. Early detection of AMD is of great importance, as the vision loss caused by this disease is irreversible and permanent. Color fundus photography is the most cost-effective imaging modality to screen for retinal disorders. Cutting edge deep learning based algorithms have been recently developed for automatically detecting AMD from fundus images. However, there are still lack of a comprehensive annotated dataset and standard evaluation benchmarks. To deal with this issue, we set up the Automatic Detection challenge on Age-related Macular degeneration (ADAM), which was held as a satellite event of the ISBI 2020 conference. The ADAM challenge consisted of four tasks which cover the main aspects of detecting and characterizing AMD from fundus images, including detection of AMD, detection and segmentation of optic disc, localization of fovea, and detection and segmentation of lesions. As part of the ADAM challenge, we have released a comprehensive dataset of 1200 fundus images with AMD diagnostic labels, pixel-wise segmentation masks for both optic disc and AMD-related lesions (drusen, exudates, hemorrhages and scars, among others), as well as the coordinates corresponding to the location of the macular fovea. A uniform evaluation framework has been built to make a fair comparison of different models using this dataset. During the ADAM challenge, 610 results were submitted for online evaluation, with 11 teams finally participating in the onsite challenge. This paper introduces the challenge, the dataset and the evaluation methods, as well as summarizes the participating methods and analyzes their results for each task. In particular, we observed that the ensembling strategy and the incorporation of clinical domain knowledge were the key to improve the performance of the deep learning models.

Vascular centerline extraction from angiography images plays an important role in computer-aided diagnosis of vascular disease. To solve the common problems related to noise and inconsistent vasculatures from uneven perfusion, this paper proposes an automatic framework for accurate vascular centerline extraction from angiograms that uses multi-probability fusion-based topology optimization. In this framework, vascular region is first segmented using a learning-based method. Then, initial centerlines are obtained by applying iterative filtering operation and multi-direction indexed non-maximum suppression. Topology optimization is achieved by gap filling. A connection probability map is constructed utilizing the information of initial centerlines, texture, and orientation of vasculatures. Shortest path tracking is employed to search for optimal connections around gaps in the initial centerlines. The proposed framework is evaluated using simulative and clinical coronary angiographies. The experimental results demonstrate that the proposed method can extract centerlines with F1 score of 97.28% ± 1.2% for vasculatures in 12 clinical angiographic images. It is evident that the proposed method can extract complete and accurate vascular centerlines from angiograms and can be used to repair gaps in other filamentary structures, such as roads and retinal blood vessels. This endows our method a great potential in the analysis of filamentary structures.

Vascular tracking of coronary angiographic image sequences is one of the most clinically important tasks in the diagnostic assessment and interventional guidance of cardiac disease. It is difficult to automate this application because the vascular structure is complex; moreover, unsatisfactory angiography image quality may exacerbate the difficulty of vasculature extraction. This paper converts vascular tracking into branch matching and proposes a novel and automatic greedy soft match algorithm. Our method is based on a graph framework. A graph model building module is proposed to represent the vascular structure. Then, a greedy branch searching method is adopted to acquire all possible paths in the graph that may match the reference vessel. Finally, a soft batch matching method that combines branch descriptor and dynamic time warping is presented to select the best matching branch. The solution to the problem takes advantage of both spatial and temporal continuity between successive frames. The experimental results demonstrate that the proposed algorithm is effective and robust for vascular tracking. The F1 score of a single branch dataset, which contains 12 angiographic image sequences with 77 angiograms of contrast agent-filled vessels, is 0.89 ± 0.06 and of a vessel tree dataset which contains nine sequences with 58 angiograms is 0.88 ± 0.05. Extensive experimental results well demonstrate the superior performance of the algorithm. In addition, it provides a universal solution to address the problem of filamentary structure tracking.