Chest computed tomography (CT) at inspiration is often complemented by an expiratory CT to identify peripheral airways disease. Additionally, co-registered inspiratory-expiratory volumes can be used to derive various markers of lung function. Expiratory CT scans, however, may not be acquired due to dose or scan time considerations or may be inadequate due to motion or insufficient exhale; leading to a missed opportunity to evaluate underlying small airways disease. Here, we propose LungViT- a generative adversarial learning approach using hierarchical vision transformers for translating inspiratory CT intensities to corresponding expiratory CT intensities. LungViT addresses several limitations of the traditional generative models including slicewise discontinuities, limited size of generated volumes, and their inability to model texture transfer at volumetric level. We propose a shifted-window hierarchical vision transformer architecture with squeeze-and-excitation decoder blocks for modeling dependencies between features. We also propose a multiview texture similarity distance metric for texture and style transfer in 3D. To incorporate global information into the training process and refine the output of our model, we use ensemble cascading. LungViT is able to generate large 3D volumes of size 320×320×320 . We train and validate our model using a diverse cohort of 1500 subjects with varying disease severity. To assess model generalizability beyond the development set biases, we evaluate our model on an out-of-distribution external validation set of 200 subjects. Clinical validation on internal and external testing sets shows that synthetic volumes could be reliably adopted for deriving clinical endpoints of chronic obstructive pulmonary disease.

This paper presents the Population Learning followed by One Shot Learning (PLOSL) pulmonary image registration method. PLOSL is a fast unsupervised learning-based framework for 3D-CT pulmonary image registration algorithm based on combining population learning (PL) and one-shot learning (OSL). The PLOSL image registration has the advantages of the PL and OSL approaches while reducing their respective drawbacks. The advantages of PLOSL include improved performance over PL, substantially reducing OSL training time and reducing the likelihood of OSL getting stuck in local minima. PLOSL pulmonary image registration uses tissue volume preserving and vesselness constraints for registration of inspiration-to-expiration and expiration-to-inspiration pulmonary CT images. A coarse-to-fine convolution encoder-decoder CNN architecture is used to register large and small shape features. During training, the sum of squared tissue volume difference (SSTVD) compensates for intensity differences between inspiration and expiration computed tomography (CT) images and the sum of squared vesselness measure difference (SSVMD) helps match the lung vessel tree. Results show that the PLOSL (SSTVD+SSVMD) algorithm achieved subvoxel landmark error while preserving pulmonary topology on the SPIROMICS data set, the public DIR-LAB COPDGene and 4DCT data sets.

Out-of-phase ventilation occurs when local regions of the lung reach their maximum or minimum volumes at breathing phases other than the global end inhalation or exhalation phases. This paper presents the N-phase local expansion ratio (LERN) as a surrogate for lung ventilation. A common approach to estimate lung ventilation is to use image registration to align the end exhalation and inhalation 3DCT images and then analyze the resulting correspondence map. This 2-phase local expansion ratio (LER2) is limited because it ignores out-of-phase ventilation and thus may underestimate local lung ventilation. To overcome this limitation, LERNmeasures the maximum ratio of local expansion and contraction over the entire breathing cycle. Comparing LER2to LERNprovides a means for detecting and characterizing locations of the lung that experience out-of-phase ventilation. We present a novel in-phase/out-of-phase ventilation (IOV) function plot to visualize and measure the amount of high-function IOV that occurs during a breathing cycle. Treatment planning 4DCT scans collected during coached breathing from 32 human subjects with lung cancer were analyzed in this study. Results show that out-of-phase breathing occurred in all subjects and that the spatial distribution of out-of-phase ventilation varied from subject to subject. For the 32 subjects analyzed, 50% of the out-of-phase regions on average were mislabeled as low-function by LER2(high-function threshold of 1.1, IOV threshold of 1.05). 4DCT and Xenon-enhanced CT of four sheep showed that LER8is more accurate than LER2 for measuring lung ventilation.

Pulmonary fissure detection in computed tomography (CT) is a critical component for automatic lobar segmentation. The majority of fissure detection methods use feature descriptors that are hand-crafted, low-level, and have local spatial extent. The design of such feature detectors is typically targeted toward normal fissure anatomy, yielding low sensitivity to weak, and abnormal fissures that are common in clinical data sets. Furthermore, local features commonly suffer from low specificity, as the complex textures in the lung can be indistinguishable from the fissure when the global context is not considered. We propose a supervised discriminative learning framework for simultaneous feature extraction and classification. The proposed framework, called FissureNet, is a coarse-to-fine cascade of two convolutional neural networks. The coarse-to-fine strategy alleviates the challenges associated with training a network to segment a thin structure that represents a small fraction of the image voxels. FissureNet was evaluated on a cohort of 3706 subjects with inspiration and expiration 3DCT scans from the COPDGene clinical trial and a cohort of 20 subjects with 4DCT scans from a lung cancer clinical trial. On both data sets, FissureNet showed superior performance compared with a deep learning approach using the U-Net architecture and a Hessian-based fissure detection method in terms of area under the precision-recall curve (PR-AUC). The overall PR-AUC for FissureNet, U-Net, and Hessian on the COPDGene (lung cancer) data set was 0.980 (0.966), 0.963 (0.937), and 0.158 (0.182), respectively. On a subset of 30 COPDGene scans, FissureNet was compared with a recently proposed advanced fissure detection method called derivative of sticks (DoS) and showed superior performance with a PR-AUC of 0.991 compared with 0.668 for DoS.

Skeletonization offers a compact representation of an object while preserving important topological and geometrical features. Literature on skeletonization of binary objects is quite mature. However, challenges involved with skeletonization of fuzzy objects are mostly unanswered. This paper presents a new theory and algorithm of skeletonization for fuzzy objects, evaluates its performance, and demonstrates its applications. A formulation of fuzzy grassfire propagation is introduced; its relationships with fuzzy distance functions, level sets, and geodesics are discussed; and several new theoretical results are presented in the continuous space. A notion of collision-impact of fire-fronts at skeletal points is introduced, and its role in filtering noisy skeletal points is demonstrated. A fuzzy object skeletonization algorithm is developed using new notions of surface- and curve-skeletal voxels, digital collision-impact, filtering of noisy skeletal voxels, and continuity of skeletal surfaces. A skeletal noise pruning algorithm is presented using branch-level significance. Accuracy and robustness of the new algorithm are examined on computer-generated phantoms and micro- and conventional CT imaging of trabecular bone specimens. An application of fuzzy object skeletonization to compute structure-width at a low image resolution is demonstrated, and its ability to predict bone strength is examined. Finally, the performance of the new fuzzy object skeletonization algorithm is compared with two binary object skeletonization methods.

Imaging techniques for measuring lung ventilation are essential tools to study lung tissue response to radiation therapy. Most of the existing CT and image registration based ventilation measurement methods use two extreme phases (end expiration and end inspiration) while ignoring the important ventilation variation within the respiratory cycle. In this paper, we present a new instantaneous ventilation measurement scheme using multiple 4DCT phase images and regularized 3D image registration, which aims at measuring the regional ventilation within the respiratory cycle. The ventilation is measured using the Jacobian (a function of local partial derivatives) of the registration displacement field. For all four animals, 39% of the total lung volume reaches maximum ventilation at T2(50% inspiration phase) during inspiration and the 75% of the total lung volume reaches maximum ventilation at T5(75% expiration phase) during expiration. The registration accuracy assessed by 200 semi automatically matched landmarks in all phases and animals show landmark error on the order of 1 mm after registration.

EMPIRE10 (Evaluation of Methods for Pulmonary Image REgistration 2010) is a public platform for fair and meaningful comparison of registration algorithms which are applied to a database of intrapatient thoracic CT image pairs. Evaluation of nonrigid registration techniques is a nontrivial task. This is compounded by the fact that researchers typically test only on their own data, which varies widely. For this reason, reliable assessment and comparison of different registration algorithms has been virtually impossible in the past. In this work we present the results of the launch phase of EMPIRE10, which comprised the comprehensive evaluation and comparison of 20 individual algorithms from leading academic and industrial research groups. All algorithms are applied to the same set of 30 thoracic CT pairs. Algorithm settings and parameters are chosen by researchers expert in the configuration of their own method and the evaluation is independent, using the same criteria for all participants. All results are published on the EMPIRE10 website (http://empire10.isi.uu.nl). The challenge remains ongoing and open to new participants. Full results from 24 algorithms have been published at the time of writing. This paper details the organization of the challenge, the data and evaluation methods and the outcome of the initial launch with 20 algorithms. The gain in knowledge and future work are discussed.

Although numerous methods to register brains of different individuals have been proposed, no work has been done, as far as we know, to evaluate and objectively compare the performances of different nonrigid (or elastic) registration methods on the same database of subjects. In this paper, we propose an evaluation framework, based on global and local measures of the relevance of the registration. We have chosen to focus more particularly on the matching of cortical areas, since intersubject registration methods are dedicated to anatomical and functional normalization, and also because other groups have shown the relevance of such registration methods for deep brain structures. Experiments were conducted using 6 methods on a database of 18 subjects. The global measures used show that the quality of the registration is directly related to the transformation's degrees of freedom. More surprisingly, local measures based on the matching of cortical sulci did not show significant differences between rigid and non rigid methods.

Two new consistent image registration algorithms are presented: one is based on matching corresponding landmarks and the other is based on matching both landmark and intensity information. The consistent landmark and intensity registration algorithm produces good correspondences between images near landmark locations by matching corresponding landmarks and away from landmark locations by matching the image intensities. In contrast to similar unidirectional algorithms, these new consistent algorithms jointly estimate the forward and reverse transformation between two images while minimizing the inverse consistency error-the error between the forward (reverse) transformation and the inverse of the the reverse (forward) transformation. This reduces the ambiguous correspondence between the forward and reverse transformations associated with large inverse consistency errors. In both algorithms a thin-plate spline (TPS) model is used to regularize the estimated transformations. Two-dimensional (2-D) examples are presented that show the inverse consistency error produced by the traditional unidirectional landmark TPS algorithm can be relatively large and that this error is minimized using the consistent landmark algorithm. Results using 2-D magnetic resonance imaging data are presented that demonstrate that using landmark and intensity information together produce better correspondence between medical images than using either landmarks or intensity information alone.

This paper presents a new method for image registration based on jointly estimating the forward and reverse transformations between two images while constraining these transforms to be inverses of one another. This approach produces a consistent set of transformations that have less pairwise registration error, i.e., better correspondence, than traditional methods that estimate the forward and reverse transformations independently. The transformations are estimated iteratively and are restricted to preserve topology by constraining them to obey the laws of continuum mechanics. The transformations are parameterized by a Fourier series to diagonalize the covariance structure imposed by the continuum mechanics constraints and to provide a computationally efficient numerical implementation. Results using a linear elastic material constraint are presented using both magnetic resonance and X-ray computed tomography image data. The results show that the joint estimation of a consistent set of forward and reverse transformations constrained by linear-elasticity give better registration results than using either constraint alone or none at all.

This paper presents diffeomorphic transformations of three-dimensional (3-D) anatomical image data of the macaque occipital lobe and whole brain cryosection imagery and of deep brain structures in human brains as imaged via magnetic resonance imagery. These transformations are generated in a hierarchical manner, accommodating both global and local anatomical detail. The initial low-dimensional registration is accomplished by constraining the transformation to be in a low-dimensional basis. The basis is defined by the Green's function of the elasticity operator placed at predefined locations in the anatomy and the eigenfunctions of the elasticity operator. The high-dimensional large deformations are vector fields generated via the mismatch between the template and target-image volumes constrained to be the solution of a Navier-Stokes fluid model. As part of this procedure, the Jacobian of the transformation is tracked, insuring the generation of diffeomorphisms. It is shown that transformations constrained by quadratic regularization methods such as the Laplacian, biharmonic, and linear elasticity models, do not ensure that the transformation maintains topology and, therefore, must only be used for coarse global registration.

A general automatic approach is presented for accommodating local shape variation when mapping a two-dimensional (2-D) or three-dimensional (3-D) template image into alignment with a topologically similar target image. Local shape variability is accommodated by applying a vector-field transformation to the underlying material coordinate system of the template while constraining the transformation to be smooth (globally positive definite Jacobian). Smoothness is guaranteed without specifically penalizing large-magnitude deformations of small subvolumes by constraining the transformation on the basis of a Stokesian limit of the fluid-dynamical Navier-Stokes equations. This differs fundamentally from quadratic penalty methods, such as those based on linearized elasticity or thin-plate splines, in that stress restraining the motion relaxes over time allowing large-magnitude deformations. Kinematic nonlinearities are inherently necessary to maintain continuity of structures during large-magnitude deformations, and are included in all results. After initial global registration, final mappings are obtained by numerically solving a set of nonlinear partial differential equations associated with the constrained optimization problem. Automatic regridding is performed by propagating templates as the nonlinear transformations evaluated on a finite lattice become singular. Application of the method to intersubject registration of neuroanatomical structures illustrates the ability to account for local anatomical variability.