Chest computed tomography (CT) at inspiration is often complemented by an expiratory CT to identify peripheral airways disease. Additionally, co-registered inspiratory-expiratory volumes can be used to derive various markers of lung function. Expiratory CT scans, however, may not be acquired due to dose or scan time considerations or may be inadequate due to motion or insufficient exhale; leading to a missed opportunity to evaluate underlying small airways disease. Here, we propose LungViT- a generative adversarial learning approach using hierarchical vision transformers for translating inspiratory CT intensities to corresponding expiratory CT intensities. LungViT addresses several limitations of the traditional generative models including slicewise discontinuities, limited size of generated volumes, and their inability to model texture transfer at volumetric level. We propose a shifted-window hierarchical vision transformer architecture with squeeze-and-excitation decoder blocks for modeling dependencies between features. We also propose a multiview texture similarity distance metric for texture and style transfer in 3D. To incorporate global information into the training process and refine the output of our model, we use ensemble cascading. LungViT is able to generate large 3D volumes of size 320×320×320 . We train and validate our model using a diverse cohort of 1500 subjects with varying disease severity. To assess model generalizability beyond the development set biases, we evaluate our model on an out-of-distribution external validation set of 200 subjects. Clinical validation on internal and external testing sets shows that synthetic volumes could be reliably adopted for deriving clinical endpoints of chronic obstructive pulmonary disease.

This paper presents the Population Learning followed by One Shot Learning (PLOSL) pulmonary image registration method. PLOSL is a fast unsupervised learning-based framework for 3D-CT pulmonary image registration algorithm based on combining population learning (PL) and one-shot learning (OSL). The PLOSL image registration has the advantages of the PL and OSL approaches while reducing their respective drawbacks. The advantages of PLOSL include improved performance over PL, substantially reducing OSL training time and reducing the likelihood of OSL getting stuck in local minima. PLOSL pulmonary image registration uses tissue volume preserving and vesselness constraints for registration of inspiration-to-expiration and expiration-to-inspiration pulmonary CT images. A coarse-to-fine convolution encoder-decoder CNN architecture is used to register large and small shape features. During training, the sum of squared tissue volume difference (SSTVD) compensates for intensity differences between inspiration and expiration computed tomography (CT) images and the sum of squared vesselness measure difference (SSVMD) helps match the lung vessel tree. Results show that the PLOSL (SSTVD+SSVMD) algorithm achieved subvoxel landmark error while preserving pulmonary topology on the SPIROMICS data set, the public DIR-LAB COPDGene and 4DCT data sets.

Out-of-phase ventilation occurs when local regions of the lung reach their maximum or minimum volumes at breathing phases other than the global end inhalation or exhalation phases. This paper presents the N-phase local expansion ratio (LERN) as a surrogate for lung ventilation. A common approach to estimate lung ventilation is to use image registration to align the end exhalation and inhalation 3DCT images and then analyze the resulting correspondence map. This 2-phase local expansion ratio (LER2) is limited because it ignores out-of-phase ventilation and thus may underestimate local lung ventilation. To overcome this limitation, LERNmeasures the maximum ratio of local expansion and contraction over the entire breathing cycle. Comparing LER2to LERNprovides a means for detecting and characterizing locations of the lung that experience out-of-phase ventilation. We present a novel in-phase/out-of-phase ventilation (IOV) function plot to visualize and measure the amount of high-function IOV that occurs during a breathing cycle. Treatment planning 4DCT scans collected during coached breathing from 32 human subjects with lung cancer were analyzed in this study. Results show that out-of-phase breathing occurred in all subjects and that the spatial distribution of out-of-phase ventilation varied from subject to subject. For the 32 subjects analyzed, 50% of the out-of-phase regions on average were mislabeled as low-function by LER2(high-function threshold of 1.1, IOV threshold of 1.05). 4DCT and Xenon-enhanced CT of four sheep showed that LER8is more accurate than LER2 for measuring lung ventilation.

Pulmonary fissure detection in computed tomography (CT) is a critical component for automatic lobar segmentation. The majority of fissure detection methods use feature descriptors that are hand-crafted, low-level, and have local spatial extent. The design of such feature detectors is typically targeted toward normal fissure anatomy, yielding low sensitivity to weak, and abnormal fissures that are common in clinical data sets. Furthermore, local features commonly suffer from low specificity, as the complex textures in the lung can be indistinguishable from the fissure when the global context is not considered. We propose a supervised discriminative learning framework for simultaneous feature extraction and classification. The proposed framework, called FissureNet, is a coarse-to-fine cascade of two convolutional neural networks. The coarse-to-fine strategy alleviates the challenges associated with training a network to segment a thin structure that represents a small fraction of the image voxels. FissureNet was evaluated on a cohort of 3706 subjects with inspiration and expiration 3DCT scans from the COPDGene clinical trial and a cohort of 20 subjects with 4DCT scans from a lung cancer clinical trial. On both data sets, FissureNet showed superior performance compared with a deep learning approach using the U-Net architecture and a Hessian-based fissure detection method in terms of area under the precision-recall curve (PR-AUC). The overall PR-AUC for FissureNet, U-Net, and Hessian on the COPDGene (lung cancer) data set was 0.980 (0.966), 0.963 (0.937), and 0.158 (0.182), respectively. On a subset of 30 COPDGene scans, FissureNet was compared with a recently proposed advanced fissure detection method called derivative of sticks (DoS) and showed superior performance with a PR-AUC of 0.991 compared with 0.668 for DoS.

A novel splat feature classification method is presented with application to retinal hemorrhage detection in fundus images. Reliable detection of retinal hemorrhages is important in the development of automated screening systems which can be translated into practice. Under our supervised approach, retinal color images are partitioned into nonoverlapping segments covering the entire image. Each segment, i.e., splat, contains pixels with similar color and spatial location. A set of features is extracted from each splat to describe its characteristics relative to its surroundings, employing responses from a variety of filter bank, interactions with neighboring splats, and shape and texture information. An optimal subset of splat features is selected by a filter approach followed by a wrapper approach. A classifier is trained with splat-based expert annotations and evaluated on the publicly available Messidor dataset. An area under the receiver operating characteristic curve of 0.96 is achieved at the splat level and 0.87 at the image level. While we are focused on retinal hemorrhage detection, our approach has potential to be applied to other object detection tasks.

This paper describes a framework for establishing a reference airway tree segmentation, which was used to quantitatively evaluate fifteen different airway tree extraction algorithms in a standardized manner. Because of the sheer difficulty involved in manually constructing a complete reference standard from scratch, we propose to construct the reference using results from all algorithms that are to be evaluated. We start by subdividing each segmented airway tree into its individual branch segments. Each branch segment is then visually scored by trained observers to determine whether or not it is a correctly segmented part of the airway tree. Finally, the reference airway trees are constructed by taking the union of all correctly extracted branch segments. Fifteen airway tree extraction algorithms from different research groups are evaluated on a diverse set of twenty chest computed tomography (CT) scans of subjects ranging from healthy volunteers to patients with severe pathologies, scanned at different sites, with different CT scanner brands, models, and scanning protocols. Three performance measures covering different aspects of segmentation quality were computed for all participating algorithms. Results from the evaluation showed that no single algorithm could extract more than an average of 74% of the total length of all branches in the reference standard, indicating substantial differences between the algorithms. A fusion scheme that obtained superior results is presented, demonstrating that there is complementary information provided by the different algorithms and there is still room for further improvements in airway segmentation algorithms.

Imaging techniques for measuring lung ventilation are essential tools to study lung tissue response to radiation therapy. Most of the existing CT and image registration based ventilation measurement methods use two extreme phases (end expiration and end inspiration) while ignoring the important ventilation variation within the respiratory cycle. In this paper, we present a new instantaneous ventilation measurement scheme using multiple 4DCT phase images and regularized 3D image registration, which aims at measuring the regional ventilation within the respiratory cycle. The ventilation is measured using the Jacobian (a function of local partial derivatives) of the registration displacement field. For all four animals, 39% of the total lung volume reaches maximum ventilation at T2(50% inspiration phase) during inspiration and the 75% of the total lung volume reaches maximum ventilation at T5(75% expiration phase) during expiration. The registration accuracy assessed by 200 semi automatically matched landmarks in all phases and animals show landmark error on the order of 1 mm after registration.

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

This paper proposes an algorithm to measure the width of retinal vessels in fundus photographs using graph-based algorithm to segment both vessel edges simultaneously. First, the simultaneous two-boundary segmentation problem is modeled as a two-slice, 3-D surface segmentation problem, which is further converted into the problem of computing a minimum closed set in a node-weighted graph. An initial segmentation is generated from a vessel probability image. We use the REVIEW database to evaluate diameter measurement performance. The algorithm is robust and estimates the vessel width with subpixel accuracy. The method is used to explore the relationship between the average vessel width and the distance from the optic disc in 600 subjects.

The human lungs are divided into five distinct anatomic compartments called the lobes, which are separated by the pulmonary fissures. The accurate identification of the fissures is of increasing importance in the early detection of pathologies, and in the regional functional analysis of the lungs. We have developed an automatic method for the segmentation and analysis of the fissures, based on the information provided by the segmentation and analysis of the airway and vascular trees. This information is used to provide a close initial approximation to the fissures, using a watershed transform on a distance map of the vasculature. In a further refinement step, this estimate is used to construct a region of interest (ROI) encompassing the fissures. The ROI is enhanced using a ridgeness measure, which is followed by a 3-D graph search to find the optimal surface within the ROI. We have also developed an automatic method to detect incomplete fissures, using a fast-marching based segmentation of a projection of the optimal surface. The detected incomplete fissure is used to extrapolate and smoothly complete the fissure. We evaluate the method by testing on data sets from normal subjects and subjects with mild to moderate emphysema.

High-resolution X-ray computed tomography (CT) imaging is routinely used for clinical pulmonary applications. Since lung function varies regionally and because pulmonary disease is usually not uniformly distributed in the lungs, it is useful to study the lungs on a lobe-by-lobe basis. Thus, it is important to segment not only the lungs, but the lobar fissures as well. In this paper, we demonstrate the use of an anatomic pulmonary atlas, encoded with a priori information on the pulmonary anatomy, to automatically segment the oblique lobar fissures. Sixteen volumetric CT scans from 16 subjects are used to construct the pulmonary atlas. A ridgeness measure is applied to the original CT images to enhance the fissure contrast. Fissure detection is accomplished in two stages: an initial fissure search and a final fissure search. A fuzzy reasoning system is used in the fissure search to analyze information from three sources: the image intensity, an anatomic smoothness constraint, and the atlas-based search initialization. Our method has been tested on 22 volumetric thin-slice CT scans from 12 subjects, and the results are compared to manual tracings. Averaged across all 22 data sets, the RMS error between the automatically segmented and manually segmented fissures is 1.96 +/- 0.71 mm and the mean of the similarity indices between the manually defined and computer-defined lobe regions is 0.988. The results indicate a strong agreement between the automatic and manual lobe segmentations.

The lungs exchange air with the external environment via the pulmonary airways. Computed tomography (CT) scanning can be used to obtain detailed images of the pulmonary anatomy, including the airways. These images have been used to measure airway geometry, study airway reactivity, and guide surgical interventions. Prior to these applications, airway segmentation can be used to identify the airway lumen in the CT images. Airway tree segmentation can be performed manually by an image analyst, but the complexity of the tree makes manual segmentation tedious and extremely time-consuming. We describe a fully automatic technique for segmenting the airway tree in three-dimensional (3-D) CT images of the thorax. We use grayscale morphological reconstruction to identify candidate airways on CT slices and then reconstruct a connected 3-D airway tree. After segmentation, we estimate airway branchpoints based on connectivity changes in the reconstructed tree. Compared to manual analysis on 3-mm-thick electron-beam CT images, the automatic approach has an overall airway branch detection sensitivity of approximately 73%.

Segmentation of pulmonary X-ray computed tomography (CT) images is a precursor to most pulmonary image analysis applications. This paper presents a fully automatic method for identifying the lungs in three-dimensional (3-D) pulmonary X-ray CT images. The method has three main steps. First, the lung region is extracted from the CT images by gray-level thresholding. Then, the left and right lungs are separated by identifying the anterior and posterior junctions by dynamic programming. Finally, a sequence of morphological operations is used to smooth the irregular boundary along the mediastinum in order to obtain results consistent with those obtained by manual analysis, in which only the most central pulmonary arteries are excluded from the lung region. The method has been tested by processing 3-D CT data sets from eight normal subjects, each imaged three times at biweekly intervals with lungs at 90% vital capacity. We present results by comparing our automatic method to manually traced borders from two image analysts. Averaged over all volumes, the root mean square difference between the computer and human analysis is 0.8 pixels (0.54 mm). The mean intrasubject change in tissue content over the three scans was 2.75% +/- 2.29% (mean +/- standard deviation).

Airway geometry measurements can provide information regarding pulmonary physiology and pathophysiology. There has been considerable interest in measuring intrathoracic airways in two-dimensional (2-D) slices from volumetric X-ray computed tomography (CT). Such measurements can be used to evaluate and track the progression of diseases affecting the airways. A popular airway measurement method uses the "half-max" criteria, in which the gray level at the airway wall is estimated to be halfway between the minimum and maximum gray levels along a ray crossing the edge. However, because the scanning process introduces blurring, the half-max approach may not be applicable across all airway sizes. We propose a new measurement method based on a model of the scanning process. In our approach, we examine the gray-level profile of a ray crossing the airway wall and use a maximum-likelihood method to estimate the airway inner and outer radius. Using CT scans of a physical phantom, we present results showing that the new approach is more accurate than the half-max method at estimating wall location for thin-walled airways.

The morphological skeleton and morphological shape decomposition (MSD) are two popular approaches for morphological shape representation. Each method represents an object as an algebraic combination of a number of components, where each component is given by a locus of points dilated by a specified structuring-element homothetic. This correspondence develops a theoretical comparison between the two methods. Combining the theoretical results with several representation cost measures, we make a concrete comparison of the efficiency of the two methods. The results indicate that for complex objects-i.e., objects requiring a full range of homothetic sizes in the morphological skeleton representation-the MSD represents objects more efficiently than the morphological skeleton for three of four suggested cost measures.

Mathematical morphology is well suited to capturing geometric information. Hence, morphology-based approaches have been popular for object shape representation. The two primary morphology-based approaches-the morphological skeleton and the morphological shape decomposition (MSD)-each represent an object as a collection of disjoint sets. A practical shape representation scheme, though, should give a representation that is computationally efficient to use. Unfortunately, little work has been done for the morphological skeleton and the MSD to address efficiency. We propose a flexible search-based shape representation scheme that typically gives more efficient representations than the morphological skeleton and MSD. Our method decomposes an object into a number of simple components based on homothetics of a set of structuring elements. To form the representation, the components are combined using set union and set difference operations. We use three constituent component types and a thorough cost-based search strategy to find efficient representations. We also consider allowing object representation error, which may yield even more efficient representations.

Traditional image segmentation methods typically divide an image into separate regions based on the grayscale characteristics of the image. For most real-world image-segmentation problems, however, these methods tend to produce imperfectly shaped regions that require some degree of shape modification to yield acceptable results. Choosing an appropriate sequence of operators and associated operator parameters, though, is a tedious procedure and requires much image-processing expertise. We describe a strategy for easily selecting shape-based operations. Shape information on regions in an image is provided by the user in the form of easily-specified cues. The user is not required to be an image-processing expert to apply the strategy-he need only be able to specify the desired shape properties of the regions in the image.

Solutions to real-world image-segmentation problems typically require many image-processing steps. Unfortunately, the user must decide how to construct this sequence of steps for a given problem. So far, no system has been proposed to make the construction of these processes "easy" for the user. As a result, segmentation processes are often laboriously developed by an image-processing expert. We describe a method for automatically generating image-segmentation processes for arbitrary images. Our method uses cue-based image analysis. The user provides problem-specific information via easily defined cues. Two types of cues can be defined: (1) iconic cues, which are image-based and constructed by drawing directly onto the image data; and (2) symbolic cues, which are verbally specified facts. The cues are interpreted to help select image-processing functions. The user need not be an image-processing expert-he must only understand the significance of the specified cues for a particular problem.>

Mathematical morphology is well-suited to capturing geometric information. Hence, morphology-based approaches have been popular for object shape representation. The two primary morphology-based approaches, the morphological skeleton and the morphological shape decomposition (MSD), each represent an object as a collection of disjoint sets. A practical shape representation scheme, though, should give a representation that is computationally efficient to use. Unfortunately, little work has been done for the morphological skeleton and the MSD to address efficiency. We propose a flexible search- based shape decomposition scheme that typically gives more efficient representations than the morphological skeleton or MSD. Our method decomposes an object into a number of simple components based on homothetics of a set of structuring elements. To form the representation, the components are combined using set union and set difference operations. We use three constituent component types and a thorough cost-based search strategy to find efficient representations. We also consider allowing some object representation error, which may yield even more efficient representations.