By focusing on metabolic and morphological tissue properties respectively, FluoroDeoxyGlucose (FDG)-Positron Emission Tomography (PET) and Computed Tomography (CT) modalities include complementary and synergistic information for cancerous lesion delineation and characterization (e.g. for outcome prediction), in addition to usual clinical variables. This is especially true in Head and Neck Cancer (HNC). The goal of the HEad and neCK TumOR segmentation and outcome prediction (HECKTOR) challenge was to develop and compare modern image analysis methods to best extract and leverage this information automatically. We present here the post-analysis of HECKTOR 2nd edition, at the 24th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2021. The scope of the challenge was substantially expanded compared to the first edition, by providing a larger population (adding patients from a new clinical center) and proposing an additional task to the challengers, namely the prediction of Progression-Free Survival (PFS). To this end, the participants were given access to a training set of 224 cases from 5 different centers, each with a pre-treatment FDG-PET/CT scan and clinical variables. Their methods were subsequently evaluated on a held-out test set of 101 cases from two centers. For the segmentation task (Task 1), the ranking was based on a Borda counting of their ranks according to two metrics: mean Dice Similarity Coefficient (DSC) and median Hausdorff Distance at 95th percentile (HD95). For the PFS prediction task, challengers could use the tumor contours provided by experts (Task 3) or rely on their own (Task 2). The ranking was obtained according to the Concordance index (C-index) calculated on the predicted risk scores. A total of 103 teams registered for the challenge, for a total of 448 submissions and 29 papers. The best method in the segmentation task obtained an average DSC of 0.759, and the best predictions of PFS obtained a C-index of 0.717 (without relying on the provided contours) and 0.698 (using the expert contours). An interesting finding was that best PFS predictions were reached by relying on DL approaches (with or without explicit tumor segmentation, 4 out of the 5 best ranked) compared to standard radiomics methods using handcrafted features extracted from delineated tumors, and by exploiting alternative tumor contours (automated and/or larger volumes encompassing surrounding tissues) rather than relying on the expert contours. This second edition of the challenge confirmed the promising performance of fully automated primary tumor delineation in PET/CT images of HNC patients, although there is still a margin for improvement in some difficult cases. For the first time, the prediction of outcome was also addressed and the best methods reached relatively good performance (C-index above 0.7). Both results constitute another step forward toward large-scale outcome prediction studies in HNC.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

Vision Transformers (ViT)-based models are witnessing an exponential growth in the medical imaging community. Among desirable properties, ViTs provide a powerful modeling of long-range pixel relationships, contrary to inherently local convolutional neural networks (CNN). These emerging models can be categorized either as hybrid-based when used in conjunction with CNN layers (CNN-ViT) or purely Transformers-based. In this work, we conduct a comparative quantitative analysis to study the differences between a range of available Transformers-based models using controlled brain tumor segmentation experiments. We also investigate to what extent such models could benefit from modality interaction schemes in a multi-modal setting. Results on the publicly-available BraTS2021 dataset show that hybrid-based pipelines generally tend to outperform simple Transformers-based models. In these experiments, no particular improvement using multi-modal interaction schemes was observed.

The extraction of blood vessels has recently experienced a widespread interest in medical image analysis. Automatic vessel segmentation is highly desirable to guide clinicians in computer-assisted diagnosis, therapy or surgical planning. Despite a good ability to extract large anatomical structures, the capacity of U-Net inspired architectures to automatically delineate vascular systems remains a major issue, especially given the scarcity of existing datasets. In this paper, we present a novel approach that integrates into deep segmentation shape priors from a Semi-Overcomplete Convolutional Auto-Encoder (S-OCAE) embedding. Compared to standard Convolutional Auto-Encoders (CAE), it exploits an over-complete branch that projects data onto higher dimensions to better characterize tiny structures. Experiments on retinal and liver vessel extraction, respectively performed on publicly-available DRIVE and 3D-IRCADb datasets, highlight the effectiveness of our method compared to U-Net trained without and with shape priors from a traditional CAE.

Standard positron emission tomography (PET) reconstruction techniques are based on maximum-likelihood (ML) optimization methods, such as the maximum-likelihood expectation-maximization (MLEM) algorithm and its variations. Most methodologies rely on a positivity constraint on the activity distribution image. Although this constraint is meaningful from a physical point of view, it can be a source of bias for low-count/high-background PET, which can compromise accurate quantification. Existing methods that allow for negative values in the estimated image usually utilize a modified log-likelihood, and therefore break the data statistics. In this paper, we propose to incorporate the positivity constraint on the projections only, by approximating the (penalized) log-likelihood function by an adequate sequence of objective functions that are easily maximized without constraint. This sequence is constructed such that there is hypo-convergence (a type of convergence that allows the convergence of the maximizers under some conditions) to the original log-likelihood, hence allowing us to achieve maximization with positivity constraint on the projections using simple settings. A complete proof of convergence under weak assumptions is given. We provide results of experiments on simulated data where we compare our methodology with the alternative direction method of multipliers (ADMM) method, showing that our algorithm converges to a maximizer, which stays in the desired feasibility set, with faster convergence than ADMM. We also show that this approach reduces the bias, as compared with MLEM images, in necrotic tumors-which are characterized by cold regions surrounded by hot structures-while reconstructing similar activity values in hot regions.

We propose a new method for the progressive enhancement of images degraded by noise and blur using shock filters. The originality of our approach lies in the iterative exploitation of nonconservative edge-based force fields normally designed for segmentation to determine both locations and amplitudes of the sharpening action. The proposed nonconservative shock filter (NCSF) produces images with strong discontinuities that can be used as an additional pre-processing step to facilitate higher level tasks such as edge detection or segmentation. NCSF is especially useful when coupled with an anisotropic diffusion equation in noisy configurations. Using the proposed method, edges that are not well defined prior to filtering can be iteratively recovered. We show processing results on various types of images, focusing on medical transrectal ultrasound and brain PET imaging.

We propose to consider linear and nonlinear image diffusion processes as the result of the transport of image values in external advection fields. Linear diffusion is seen as the continuous transport of particles in fields of random orientations, while non-linear behavior is obtained by modulating the magnitude of these fields according to local edge strength. Edge sharpening can be achieved by biasing fields towards image edges using e.g. Gradient Vector Flow. This point of view, closely related to discrete random walks and other stochastic interpretations of diffusion processes, allows for simple, stable and surprisingly efficient filters that can be implemented using a few lines of code only.

We present a multi-scale approach of tumor modeling in order to predict its evolution during radiotherapy. Within this context we focus on three different scales of tumor modeling: microscopic (individual cells in a voxel), mesoscopic (population of cells in a voxel) and macroscopic (whole tumor), with transition interfaces between these three scales. At the cellular level, the description is based on phase transfer probabilities in the cellular cycle. At the mesoscopic scale we represent populations of cells according to different stages in a cell cycle. Finally, at the macroscopic scale, the tumor description is based on the use of FDG PET image voxels. These three scales exist naturally: biological data are collected at the macroscopic scale, but the pathological behavior of the tumor is based on an abnormal cell-cycle at the microscopic scale. On the other hand, the introduction of a mesoscopic scale is essential in order to reduce the gap between the two extreme, in terms of resolution, description levels. It also reduces the computational burden of simulating a large number of individual cells. As an application of the proposed multi-scale model, we simulate the effect of oxygen on tumor evolution during radiotherapy. Two consecutive FDG PET images of 17 rectal cancer patients undergoing radiotherapy are used to simulate the tumor evolution during treatment. The simulated results are compared with those obtained on a third FDG PET image acquired two weeks after the beginning of the treatment.

Minimally-invasive (MI) procedures are becoming more popular and frequent due to their benefits such as reduced patient trauma and hospitalization time. However, several common types of MI interventions are performed under X-ray guidance, which exposes both patients and staff to harmful ionizing radiation. Radiation exposure has therefore become a major concern for the medical community. Yet, few efforts to actively reduce it by exploiting the robotic capabilities of the devices present in the surgical suite have been performed. The propagation of radiation highly depends on the X-ray source positioning. Hence, we propose an approach to optimize the imaging device's pose in order to reduce the exposure to radiation of both patient and staff, while preserving the visibility of the targeted anatomical structure in the acquired image. Our method is based on the optimization of a cost function, which takes the current context and device parameters into account to compute the overall radiation exposure. It relies on GPU-accelerated Monte Carlo methods to simulate radiation propagation and performs the optimization in quasi real-time. When evaluated on a set of standard imaging configurations, our approach is able to recommend a device's pose in a few seconds, for which the delivered dose is reduced. Such an approach can contribute to lower the probability of appearance and severity of long-term negative effects due to radiation exposure and improve overall radiation safety.

In clinical oncology, positron emission tomography (PET) imaging can be used to assess therapeutic response by quantifying the evolution of semi-quantitative values such as standardized uptake value, early during treatment or after treatment. Current guidelines do not include metabolically active tumor volume (MATV) measurements and derived parameters such as total lesion glycolysis (TLG) to characterize the response to the treatment. To achieve automatic MATV variation estimation during treatment, we propose an approach based on the change detection principle using the recent paradoxical theory, which models imprecision, uncertainty, and conflict between sources. It was applied here simultaneously to pre- and post-treatment PET scans. The proposed method was applied to both simulated and clinical datasets, and its performance was compared to adaptive thresholding applied separately on pre- and post-treatment PET scans. On simulated datasets, the adaptive threshold was associated with significantly higher classification errors than the developed approach. On clinical datasets, the proposed method led to results more consistent with the known partial responder status of these patients. The method requires accurate rigid registration of both scans which can be obtained only in specific body regions and does not explicitly model uptake heterogeneity. In further investigations, the change detection of intra-MATV tracer uptake heterogeneity will be developed by incorporating textural features into the proposed approach.

Respiratory motion is a major source of reduced quality in positron emission tomography (PET). In order to minimize its effects, the use of respiratory synchronized acquisitions, leading to gated frames, has been suggested. Such frames, however, are of low signal-to-noise ratio (SNR) as they contain reduced statistics. Super-resolution (SR) techniques make use of the motion in a sequence of images in order to improve their quality. They aim at enhancing a low-resolution image belonging to a sequence of images representing different views of the same scene. In this work, a maximum a posteriori (MAP) super-resolution algorithm has been implemented and applied to respiratory gated PET images for motion compensation. An edge preserving Huber regularization term was used to ensure convergence. Motion fields were recovered using a B-spline based elastic registration algorithm. The performance of the SR algorithm was evaluated through the use of both simulated and clinical datasets by assessing image SNR, as well as the contrast, position and extent of the different lesions. Results were compared to summing the registered synchronized frames on both simulated and clinical datasets. The super-resolution image had higher SNR (by a factor of over 4 on average) and lesion contrast (by a factor of 2) than the single respiratory synchronized frame using the same reconstruction matrix size. In comparison to the motion corrected or the motion free images a similar SNR was obtained, while improvements of up to 20% in the recovered lesion size and contrast were measured. Finally, the recovered lesion locations on the SR images were systematically closer to the true simulated lesion positions. These observations concerning the SNR, lesion contrast and size were confirmed on two clinical datasets included in the study. In conclusion, the use of SR techniques applied to respiratory motion synchronized images lead to motion compensation combined with improved image SNR and contrast, without any increase in the overall acquisition times.

The purpose of the work described in this paper was the development of a framework for the creation of a realistic positron emission tomography (PET) simulated database incorporating patient-specific variability. The ground truth used was therefore based on clinical PET/computed tomography (CT) data of oncology patients. In the first step, the NURBS-based cardiac-torso phantom was adapted to the patient's CT acquisitions to reproduce their specific anatomy while the corresponding PET acquisitions were used to derive the activity distribution of each organ of interest. Secondly, realistic tumor shapes with homogeneous or heterogeneous activity distributions were modeled based on segmentation of the PET tumor volume and incorporated in the patient-specific models obtained at the first step. Lastly, patient-specific respiratory motion was also modeled. The derived patient-specific models were subsequently combined with the PET SORTEO Monte Carlo simulation tool for the simulation of the whole-body PET acquisition process. The accuracy of the simulated datasets was assessed in comparison to the original clinical patient images. In addition, a couple of applications for such simulated images were also demonstrated. Future work will focus on the creation of a comprehensive database of simulated raw data and reconstructed whole-body images, facilitating the rigorous evaluation of image-processing algorithms in PET for oncology applications.

Accurate volume estimation in positron emission tomography (PET) is crucial for different oncology applications. The objective of our study was to develop a new fuzzy locally adaptive Bayesian (FLAB) segmentation for automatic lesion volume delineation. FLAB was compared with a threshold approach as well as the previously proposed fuzzy hidden Markov chains (FHMC) and the fuzzy C-Means (FCM) algorithms. The performance of the algorithms was assessed on acquired datasets of the IEC phantom, covering a range of spherical lesion sizes (10-37 mm), contrast ratios (4:1 and 8:1), noise levels (1, 2, and 5 min acquisitions), and voxel sizes (8 and 64 mm(3)). In addition, the performance of the FLAB model was assessed on realistic nonuniform and nonspherical volumes simulated from patient lesions. Results show that FLAB performs better than the other methodologies, particularly for smaller objects. The volume error was 5%-15% for the different sphere sizes (down to 13 mm), contrast and image qualities considered, with a high reproducibility (variation < 4%). By comparison, the thresholding results were greatly dependent on image contrast and noise, whereas FCM results were less dependent on noise but consistently failed to segment lesions < 2 cm. In addition, FLAB performed consistently better for lesions < 2 cm in comparison to the FHMC algorithm. Finally the FLAB model provided errors less than 10% for nonspherical lesions with inhomogeneous activity distributions. Future developments will concentrate on an extension of FLAB in order to allow the segmentation of separate activity distribution regions within the same functional volume as well as a robustness study with respect to different scanners and reconstruction algorithms.