The segmentation of 2D vascular structures via deep learning holds significant clinical value but is hindered by the scarcity of annotated data, severely limiting its widespread application. Developing a universal few-shot vascular segmentation model is highly desirable, yet remains challenging due to the need for extensive training and the inherent complexities of vascular imaging. In this work, we propose UniVG (Generative Data-engine Foundation Model for Universal Few-shot 2D Vascular Image Segmentation), a novel approach that learns the compositionality of vascular images and constructing a generative foundation model for robust vascular segmentation. UniVG enables the synthesis and learning of diverse and realistic vascular images through two key innovations: 1) Compositional learning for flexible and diverse vascular synthesis: It decomposes and recombines vascular structures with varying morphological features and diverse foreground-background configurations to generate richly diverse synthetic image-label pairs. 2) Few-shot generative adaptation for transferable segmentation: It fine-tunes pre-trained models with minimal annotated data to bridge the gap between synthetic and real vascular domains, synthesizing authentic and diverse vessel images for downstream few-shot vascular segmentation learning. To support our approach, we develop UniVG-58K, a large dataset comprising 58,689 vascular images across five imaging modalities, facilitating robust large-scale generative pre-training. Extensive experiments on 11 vessel segmentation tasks cross 5 modalties (only with 5 labeled images on each task) demonstrate that UniVG achieves performance comparable to fully supervised models, significantly reducing data collection and annotation costs. All code and datasets will be made publicly available at https://github.com/XinAloha/UniVG.

Accurate delineation of pancreatic tumors on Magnetic Resonance Imaging (MRI) is important for diagnosis, radiotherapy treatment planning, and outcome assessment, but remains challenging due to complex anatomy and subtle tumor appearance. In routine practice, tumor contours on MRI are produced manually, which is time-consuming and subject to inter-observer variability. Radiotherapy on MRI-Linear Accelerator (MRI-Linac) systems further requires fast and consistent Gross Tumor Volume (GTV) contours for online adaptation, yet most public pancreas tumor segmentation benchmarks focus on Computed Tomography (CT). The Pancreatic Tumor Segmentation in Therapeutic and Diagnostic MRI (PANTHER) challenge addresses this gap by benchmarking automatic pancreatic tumor segmentation on MRI. The dataset includes contrast-enhanced T1-weighted diagnostic MRI and T2-weighted MRI-Linac scans with expert pancreas and tumor annotations, organized into two tasks: (1) tumor segmentation on diagnostic MRI and (2) tumor segmentation on MRI-Linac images. Performance was evaluated using overlap metrics, distance-based metrics, and tumor volume error. The challenge attracted 285 registered participants, with 12 and 9 final submissions for Tasks 1 and 2, respectively. On diagnostic MRI, top methods achieved performance close to inter-reader agreement. Multi-reader analysis suggested that models often reproduced the contouring style of the training annotator, highlighting the importance of annotation quality and consensus. In contrast, performance on MRI-Linac images was lower and more heterogeneous, including cases of complete localization failure. PANTHER provides the first public benchmark for pancreatic tumor segmentation on MRI, showing that clinically useful automation is feasible on diagnostic MRI, while robust MRI-Linac GTV segmentation remains an open challenge.

We present a method for estimating, in real time, a 3D displacement field from a single fluoroscopic image. Our approach uses a fully convolutional network architecture to solve the associated inverse problem. Supervised learning is performed on synthetic data, using Digitally Reconstructed Radiographs as input and displacement fields as output. We use randomized Gaussian kernels to produce a synthetic training dataset with displacement fields that are smooth and diffeomorphic. In contrast to other 2D-3D registration methods, our novel data generation approach does not rely on a statistical motion model. This enables our model to accurately predict deformations unrelated to breathing or other predetermined motion patterns. As an example of clinical application, we show that our model is able to predict deformations related to percutaneous needle insertions accurately, potentially removing the need for contrast agent injection.

Lung cancer is the leading cause of cancer deaths worldwide. A potential early indicator of lung cancer is the presence of lung nodules that can be detected through screening. Open thoracotomy, a surgical approach for nodule resection, carries inherent risk which can be minimized with use of Video-Assisted Thoracoscopic Surgery (VATS); a minimally invasive alternative, reducing risks and recovery time. Precise nodule localization is crucial for efficient navigation during VATS. The utilization of intraoperative Cone-Beam Computed Tomography (CBCT), an imaging modality, can improve localization of the nodules. However, this poses a challenge when attempting to accurately align the nodule position from the CBCT to the surgical view. To address this, we propose a novel approach that segments corresponding features visible in both modalities, specifically the rib cages and Alexis O Wound Protector/Retractor (Alexis). The segmentation of these features is performed using YOLOv8 allowing image registration and alignment of the CBCT data with the surgical view. With the established correspondence, we can gauge possible camera locations and create an augmented reality overlay of the surgical site to provide real-time guidance in VATS.

International audience

Accurate segmentation of uterus, uterine fibroids, and spine from MR images is crucial for high intensity focused ultrasound (HIFU) therapy but remains still difficult to achieve because of 1) the large shape and size variations among individuals, 2) the low contrast between adjacent organs and tissues, and 3) the unknown number of uterine fibroids. To tackle this problem, in this paper, we propose a large kernel Encoder-Decoder Network based on a 2D segmentation model. The use of this large kernel can capture multi-scale contexts by enlarging the valid receptive field. In addition, a deep multiple atrous convolution block is also employed to enlarge the receptive field and extract denser feature maps. Our approach is compared to both conventional and other deep learning methods and the experimental results conducted on a large dataset show its effectiveness.

In this paper, an unsupervised cycle image registration convolutional neural network named CIRNet is developed for 3D medical image registration. Different from most deep learning based registration methods that require known spatial transforms, our proposed method is trained in an unsupervised way and predicts the dense displacement vector field. The CIRNet is composed by two image registration modules which have the same architecture and share the parameters. A cycle identical loss is designed in the CIRNet to provide additional constraints to ensure the accuracy of the predicted dense displacement vector field. The method is evaluated by the registration in 4D (3D+t) cardiac CT and MRI images respectively. Quantitative evaluation results demonstrate that our method performs better than the other two existing image registration algorithms. Especially, compared to the traditional image registration methods, our proposed network can finish 3D image registration in less than one second.

Accounting for nearly 2% of all adults, renal cell carcinomas are sensitive to laparoscopic partial nephrectomy (LPN) which needs an accurate diagnosis and localization before operation. Faced with various intensity distribution, erratic location, irregular shape, etc, the image classification and semantic segmentation on CT scans of renal tumor are challenges. This paper presents a multi-task network, segmentation and classification convolutional neural network (SCNet), for preoperative assessment of renal tumor. Via the combination of two tasks, semantic features are fed to the classification network and classification results give segmentation network feedbacks in return. Besides, a 2-step segmentation strategy is conducted to the segmentation module which improves the result by 2.8%. Our experimental results of classification and segmentation achieve 100% accuracy and 0.882 dice coefficient of tumor region respectively, which are better than the results of a single classification network and segmentation network.

In this work, a super-resolution method is proposed for indoor scenes captured by low-resolution thermal cameras. The proposed method is called Edge Focused Thermal Super-resolution (EFTS) which contains an edge extraction module enforcing the neural networks to focus on the edge of images. Utilizing edge information, our model, based on residual dense blocks, can perform super-resolution for thermal images, while enhancing the visual information of the edges. Experiments on benchmark datasets showed that our EFTS method achieves better performance in comparison to the state-of-the-art techniques.

In this paper, we address the particularly challenging problem of calibrating a stereo pair of low resolution (80 × 60) thermal cameras. We propose a new calibration method for such setup, based on sub-pixel image analysis of an adequate calibration pattern and bootstrap methods. The experiments show that the method achieves robust calibration with a quarter-pixel re-projection error for an optimal set of 35 input stereo pairs of the calibration pattern, which namely outperforms the standard OpenCV stereo calibration procedure.

Renal cancer is one of ten most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) becomes the main therapeutic approach in treating renal cancer. Accurate kidney and tumor segmentation in CT images is a prerequisite step in the surgery planning. However, automatic and accurate kidney and renal tumor segmentation in CT images remains a challenge. In this paper, we propose a new method to perform a precise segmentation of kidney and renal tumor in CT angiography images. This method relies on a three-dimensional (3D) fully convolutional network (FCN) which combines a pyramid pooling module (PPM). The proposed network is implemented as an end-to-end learning system directly on 3D volumetric images. It can make use of the 3D spatial contextual information to improve the segmentation of the kidney as well as the tumor lesion. The experiments conducted on 140 patients show that these target structures can be segmented with a high accuracy. The resulting average dice coefficients obtained for kidney and renal tumor are equal to 0.931 and 0.802 respectively. These values are higher than those obtained from the other two neural networks.

As a particular case of the finite mixture model, Rayleigh mixture model (RMM) is considered as a useful tool for medical ultrasound (US) image segmentation. However, conventional RMM relies on intensity distribution only and does not take any spatial information into account that leads to misclassification on boundaries and inhomogeneous regions. The authors proposed an improved RMM with neighbour (RMMN) information to solve this problem by introducing neighbourhood information through a mean template. The incorporation of the spatial information made RMMN more robust to noise on the boundaries. The size of the window which incorporates neighbour information was resized adaptively according to the local gradient distribution. They evaluated their model on experiments on synthetic data and real US images used by high‐intensity focused ultrasound therapy. On this data, they demonstrated that the proposed model outperforms several state‐of‐the‐art methods in terms of both segmentation accuracy and computation time.

Knowledge of left atrial (LA) anatomy is important for atrial fibrillation ablation guidance, fibrosis quantification and biophysical modelling. Segmentation of the LA from Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images is a complex problem. This manuscript presents a benchmark to evaluate algorithms that address LA segmentation. The datasets, ground truth and evaluation code have been made publicly available through the http://www.cardiacatlas.org website. This manuscript also reports the results of the Left Atrial Segmentation Challenge (LASC) carried out at the STACOM'13 workshop, in conjunction with MICCAI'13. Thirty CT and 30 MRI datasets were provided to participants for segmentation. Each participant segmented the LA including a short part of the LA appendage trunk and proximal sections of the pulmonary veins (PVs). We present results for nine algorithms for CT and eight algorithms for MRI. Results showed that methodologies combining statistical models with region growing approaches were the most appropriate to handle the proposed task. The ground truth and automatic segmentations were standardised to reduce the influence of inconsistently defined regions (e.g., mitral plane, PVs end points, LA appendage). This standardisation framework, which is a contribution of this work, can be used to label and further analyse anatomical regions of the LA. By performing the standardisation directly on the left atrial surface, we can process multiple input data, including meshes exported from different electroanatomical mapping systems.

Discusses and debates the role played by 3D visualization in medicine as a set of methods and techniques for displaying 3D spatial information related to the anatomy and the physiology of the human body.>