The presence of tertiary lymphoid structures (TLSs) on pancreatic pathological images is an important prognostic indicator of pancreatic tumors. Therefore, TLSs detection on pancreatic pathological images plays a crucial role in diagnosis and treatment for patients with pancreatic tumors. However, fully supervised detection algorithms based on deep learning usually require a large number of manual annotations, which is time-consuming and labor-intensive. In this paper, we aim to detect the TLSs in a manner of few-shot learning by proposing a weakly supervised segmentation network. We firstly obtain the lymphocyte density maps by combining a pretrained model for nuclei segmentation and a domain adversarial network for lymphocyte nuclei recognition. Then, we establish a cross-scale attention guidance mechanism by jointly learning the coarse-scale features from the original histopathology images and fine-scale features from our designed lymphocyte density attention. A noise-sensitive constraint is introduced by an embedding signed distance function loss in the training procedure to reduce tiny prediction errors. Experimental results on two collected datasets demonstrate that our proposed method significantly outperforms the state-of-the-art segmentation-based algorithms in terms of TLSs detection accuracy. Additionally, we apply our method to study the congruent relationship between the density of TLSs and peripancreatic vascular invasion and obtain some clinically statistical results.

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

Automatic segmentation of abdominal organs in CT scans plays an important role in clinical practice. However, most existing benchmarks and datasets only focus on segmentation accuracy, while the model efficiency and its accuracy on the testing cases from different medical centers have not been evaluated. To comprehensively benchmark abdominal organ segmentation methods, we organized the first Fast and Low GPU memory Abdominal oRgan sEgmentation (FLARE) challenge, where the segmentation methods were encouraged to achieve high accuracy on the testing cases from different medical centers, fast inference speed, and low GPU memory consumption, simultaneously. The winning method surpassed the existing state-of-the-art method, achieving a 19× faster inference speed and reducing the GPU memory consumption by 60% with comparable accuracy. We provide a summary of the top methods, make their code and Docker containers publicly available, and give practical suggestions on building accurate and efficient abdominal organ segmentation models. The FLARE challenge remains open for future submissions through a live platform for benchmarking further methodology developments at https://flare.grand-challenge.org/.

This paper relates the post-analysis of the first edition of the HEad and neCK TumOR (HECKTOR) challenge. This challenge was held as a satellite event of the 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020, and was the first of its kind focusing on lesion segmentation in combined FDG-PET and CT image modalities. The challenge's task is the automatic segmentation of the Gross Tumor Volume (GTV) of Head and Neck (H&N) oropharyngeal primary tumors in FDG-PET/CT images. To this end, the participants were given a training set of 201 cases from four different centers and their methods were tested on a held-out set of 53 cases from a fifth center. The methods were ranked according to the Dice Score Coefficient (DSC) averaged across all test cases. An additional inter-observer agreement study was organized to assess the difficulty of the task from a human perspective. 64 teams registered to the challenge, among which 10 provided a paper detailing their approach. The best method obtained an average DSC of 0.7591, showing a large improvement over our proposed baseline method and the inter-observer agreement, associated with DSCs of 0.6610 and 0.61, respectively. The automatic methods proved to successfully leverage the wealth of metabolic and structural properties of combined PET and CT modalities, significantly outperforming human inter-observer agreement level, semi-automatic thresholding based on PET images as well as other single modality-based methods. This promising performance is one step forward towards large-scale radiomics studies in H&N cancer, obviating the need for error-prone and time-consuming manual delineation of GTVs.

With the unprecedented developments in deep learning, automatic segmentation of main abdominal organs seems to be a solved problem as state-of-the-art (SOTA) methods have achieved comparable results with inter-rater variability on many benchmark datasets. However, most of the existing abdominal datasets only contain single-center, single-phase, single-vendor, or single-disease cases, and it is unclear whether the excellent performance can generalize on diverse datasets. This paper presents a large and diverse abdominal CT organ segmentation dataset, termed AbdomenCT-1K, with more than 1000 (1K) CT scans from 12 medical centers, including multi-phase, multi-vendor, and multi-disease cases. Furthermore, we conduct a large-scale study for liver, kidney, spleen, and pancreas segmentation and reveal the unsolved segmentation problems of the SOTA methods, such as the limited generalization ability on distinct medical centers, phases, and unseen diseases. To advance the unsolved problems, we further build four organ segmentation benchmarks for fully supervised, semi-supervised, weakly supervised, and continual learning, which are currently challenging and active research topics. Accordingly, we develop a simple and effective method for each benchmark, which can be used as out-of-the-box methods and strong baselines. We believe the AbdomenCT-1K dataset will promote future in-depth research towards clinical applicable abdominal organ segmentation methods.

Active contour models have been widely used in image segmentation, and the level set method (LSM) is the most popular approach for solving the models, via implicitly representing the contour by a level set function. However, the LSM suffers from high computational burden and numerical instability, requiring additional regularization terms or reinitialization techniques. In this paper, we use characteristic functions to implicitly represent the contours, propose a new representation to the geodesic active contours, and derive an efficient algorithm termed the iterative convolution-thresholding method (ICTM). Compared to the LSM, the ICTM is simpler and much more efficient. In addition, the ICTM enjoys most desired features of the level set--based methods. Extensive experiments, on two-dimensional (2D) synthetic, 2D ultrasound, 3D computed tomography, and 3D magnetic resonance images for nodule, organ, and lesion segmentation demonstrate that the proposed method not only obtains comparable or even better segmentation results (compared to the LSM) but also achieves significant acceleration.

Most existing CNNs-based segmentation methods rely on local appearances learned on the regular image grid, without consideration of the object global information. This article aims to embed the object global geometric information into a learning framework via the classical geodesic active contours (GAC). We propose a level set function (LSF) regression network, supervised by the segmentation ground truth, LSF ground truth and geodesic active contours, to not only generate the segmentation probabilistic map but also directly minimize the GAC energy functional in an end-to-end manner. With the help of geodesic active contours, the segmentation contour, embedded in the level set function, can be globally driven towards the image boundary to obtain lower energy, and the geodesic constraint can lead the segmentation result to have fewer outliers. Extensive experiments on four public datasets show that (1) compared with state-of-the-art (SOTA) learning active contour methods, our method can achieve significantly better performance; (2) compared with recent SOTA methods that are designed for reducing boundary errors, our method also outperforms them with more accurate boundaries; (3) compared with SOTA methods on two popular multi-class segmentation challenge datasets, our method can still obtain superior or competitive results in both organ and tumor segmentation tasks. Our study demonstrates that introducing global information by GAC can significantly improve segmentation performance, especially on reducing the boundary errors and outliers, which is very useful in applications such as organ transplantation surgical planning and multi-modality image registration where boundary errors can be very harmful.

To better understand early brain development in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; h owever, one of the major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, the iSeg-2019 challenge (http://iseg2019.web.unc.edu) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. T raining/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participated in the iSeg-2019. In this article, 8 top-ranked methods were reviewed by detailing their pipelines/implementations, presenting experimental results, and evaluating performance across different sites in terms of whole brain, regions of interest, and gyral landmark curves. We further pointed out their limitations and possible directions for addressing the multi-site issue. We find that multi-site consistency is still an open issue. We hope that the multi-site dataset in the iSeg-2019 and this review article will attract more researchers to address the challenging and critical multi-site issue in practice.

In this paper, we propose a new three-stage segmentation framework based on a convex variant of the Mumford--Shah model and the intensity inhomogeneity information of an image. The first stage in our framework is to perform a dimension lifting method. An intensity inhomogeneity image is added as an additional channel, which results in a vector-valued image. In the second stage, a convex variant of the Mumford--Shah model is applied to each channel of the vector-valued image to obtain a smooth approximation. We use the semi--proximal alternating direction method of multipliers (sPADMM) to solve this model and prove that the sPADMM for solving this convex model has Q-linear convergence rate. In the last stage, we apply a thresholding method to the smoothed vector-valued image to get the final segmentation. Experiments demonstrate clearly that the proposed methods can provide more accurate segmentation results in comparison with five state-of-the-art methods including a deep learning approach.

In this paper, we propose a new artifact-free variational model for JPEG/JPEG 2000 decompression based on a cartoon-texture decomposition scheme. The new infimal convolution type regularization associated with total generalized variation (TGV) and shearlet transform can reconstruct piecewise smooth images with structured textures well, due to the property of shearlet of representing the positions and orientations of singularities, which can be interpreted as the oscillation texture parts. In order to enhance the qualities of reconstructed images, we incorporate an L2 cost functional into the model, then the discretization of such functional can be easily solved by the generic proximal Primal-Dual method. Numerical experiments show that our proposed model is competitive with the learning method-Trainable Nonlinear Reaction Diffusion (TNRD) [33, 34] in term of texture preservation, and outperforms the TV-based and TGV-based variational methods.

Deformable image registration is a widely used technique in the field of computer vision and medical image processing. Basically, the task of deformable image registration is to find the displacement field between the moving image and the fixed image. Many variational models are proposed for deformable image registration, under the assumption that the displacement field is continuous and smooth. However, displacement fields may be discontinuous, especially for medical images with intensity inhomogeneity, pathological tissues, or heavy noises. In the mathematical theory of elastoplasticity, when the displacement fields are possibly discontinuous, a suitable framework for describing the displacement fields is the space of functions of bounded deformation (BD). Inspired by this, we propose a novel deformable registration model, called the BD model, which allows discontinuities of displacement fields in images. The BD model is formulated in a variational framework by supposing the displacement field to be a function of BD. The existence of solutions of this model is proven. Numerical experiments on 2D images show that the BD model outperforms the classical demons model, the log-domain diffeomorphic demons model, and the state-of-the-art vectorial total variation model. Numerical experiments on two public 3D databases show that the target registration error of the BD model is competitive compared with more than ten other models.

In this study, the authors propose a variational approach based on total generalised variation (TGV) and local gradient information to fuse multi‐focus images as well as medical images of computed tomography and magnetic resonance. They use the second‐order TGV as the regularisation term and local gradient information as the fusion weight to extract image features. To compute the new model effectively, the primal‐dual algorithm is carried out. Various experiments are made to verify the effectiveness of the proposed methods.