Groupwise registration has been an essential technique for addressing the growing demands for batch processing of medical images collected from various sites or time points. It is vital to select or construct an atlas for a given group when performing groupwise registration. This paper proposes a novel atlas generation framework with a latent diffusion model that generates an atlas from noise by leveraging the latent vectors of all subjects in the group. By incorporating the attention mechanism, the model accommodates groups of arbitrary sizes and enhances scalability during the atlas generation. A minimum group construction module is introduced to create minimal group units and their corresponding atlases for training the diffusion model. Experimental results on brain MRI slices demonstrate that the proposed method can generate comparable or higher quality atlases compared to baselines while significantly reducing runtime. To conclude, the method effectively balances image quality, registration actuary, atlas unbiasedness, and computational efficiency.

Template generation is a critical step in groupwise image registration, which involves aligning a group of subjects into a common space. While existing methods can generate high-quality template images, they often incur substantial time costs or are limited by fixed group scales. In this paper, we present InstantGroup, an efficient groupwise template generation framework based on variational autoencoder (VAE) models that leverage latent representations' arithmetic properties, enabling scalability to groups of any size. InstantGroup features a Dual VAE backbone with shared-weight twin networks to handle pairs of inputs and incorporates a Displacement Inversion Module (DIM) to maintain template unbiasedness and a Subject-Template Alignment Module (STAM) to improve template quality and registration accuracy. Experiments on 3D brain MRI scans from the OASIS and ADNI datasets reveal that InstantGroup dramatically reduces runtime, generating templates within seconds for various group sizes while maintaining superior performance compared to state-of-the-art baselines on quantitative metrics, including unbiasedness and registration accuracy.

This paper introduces an effective and efficient framework for retinal vessel segmentation. First, we design a Transformer-CNN hybrid model in which a Transformer module is inserted inside the U-Net to capture long-range interactions. Second, we design a dual-path decoder in the U-Net framework, which contains two decoding paths for multi-task outputs. Specifically, we train the extra decoder to predict vessel skeletons as an auxiliary task which helps the model learn balanced features. The proposed framework, named as TSNet, not only achieves good performances in a fully supervised learning manner but also enables a rough skeleton annotation process. The annotators only need to roughly delineate vessel skeletons instead of giving precise pixel-wise vessel annotations. To learn with rough skeleton annotations plus a few precise vessel annotations, we propose a skeleton semi-supervised learning scheme. We adopt a mean teacher model to produce pseudo vessel annotations and conduct annotation correction for roughly labeled skeletons annotations. This learning scheme can achieve promising performance with fewer annotation efforts. We have evaluated TSNet through extensive experiments on five benchmarking datasets. Experimental results show that TSNet yields state-of-the-art performances on retinal vessel segmentation and provides an efficient training scheme in practice.

Image segmentation is essential to medical image analysis as it provides the labeled regions of interest for the subsequent diagnosis and treatment. However, fully-supervised segmentation methods require high-quality annotations produced by experts, which is laborious and expensive. In addition, when performing segmentation on another unlabeled image modality, the segmentation performance will be adversely affected due to the domain shift. Unsupervised domain adaptation (UDA) is an effective way to tackle these problems, but the performance of the existing methods is still desired to improve. Also, despite the effectiveness of recent Transformer-based methods in medical image segmentation, the adaptability of Transformers is rarely investigated. In this paper, we present a novel UDA framework using a Transformer for building a cross-modality segmentation method with the advantages of learning long-range dependencies and transferring attentive information. To fully utilize the attention learned by the Transformer in UDA, we propose Meta Attention (MA) and use it to perform a fully attention-based alignment scheme, which can learn the hierarchical consistencies of attention and transfer more discriminative information between two modalities. We have conducted extensive experiments on cross-modality segmentation using three datasets, including a whole heart segmentation dataset (MMWHS), an abdominal organ segmentation dataset, and a brain tumor segmentation dataset. The promising results show that our method can significantly improve performance compared with the state-of-the-art UDA methods.

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

Affine registration is indispensable in a comprehensive medical image registration pipeline. However, only a few studies focus on fast and robust affine registration algorithms. Most of these studies utilize convolutional neural networks (CNNs) to learn joint affine and non-parametric registration, while the standalone performance of the affine subnetwork is less explored. Moreover, existing CNN-based affine registration approaches focus either on the local mis-alignment or the global orientation and position of the input to predict the affine transformation matrix, which are sensitive to spatial initialization and exhibit limited generalizability apart from the training dataset. In this paper, we present a fast and robust learning-based algorithm, Coarse-to-Fine Vision Transformer (C2FViT), for 3D affine medical image registration. Our method naturally leverages the global connectivity and locality of the convolutional vision transformer and the multi-resolution strategy to learn the global affine registration. We evaluate our method on 3D brain atlas registration and template-matching normalization. Comprehensive results demonstrate that our method is superior to the existing CNNs-based affine registration methods in terms of registration accuracy, robustness and generalizability while preserving the runtime advantage of the learning-based methods. The source code is available at https://github.com/cwmok/C2FViT.

In this paper, we present a Deep Convolutional Neural Networks (CNNs) for fully automatic brain tumor segmentation for both high- and low-grade gliomas in MRI images. Unlike normal tissues or organs that usually have a fixed location or shape, brain tumors with different grades have shown great variation in terms of the location, size, structure, and morphological appearance. Moreover, the severe data imbalance exists not only between the brain tumor and non-tumor tissues, but also among the different sub-regions inside brain tumor (e.g., enhancing tumor, necrotic, edema, and non-enhancing tumor). Therefore, we introduce a hybrid model to address the challenges in the multi-modality multi-class brain tumor segmentation task. First, we propose the dynamic focal Dice loss function that is able to focus more on the smaller tumor sub-regions with more complex structures during training, and the learning capacity of the model is dynamically distributed to each class independently based on its training performance in different training stages. Besides, to better recognize the overall structure of the brain tumor and the morphological relationship among different tumor sub-regions, we relax the boundary constraints for the inner tumor regions in coarse-to-fine fashion. Additionally, a symmetric attention branch is proposed to highlight the possible location of the brain tumor from the asymmetric features caused by growth and expansion of the abnormal tissues in the brain. Generally, to balance the learning capacity of the model between spatial details and high-level morphological features, the proposed model relaxes the constraints of the inner boundary and complex details and enforces more attention on the tumor shape, location, and the harder classes of the tumor sub-regions. The proposed model is validated on the publicly available brain tumor dataset from real patients, BRATS 2019. The experimental results reveal that our model improves the overall segmentation performance in comparison with the state-of-the-art methods, with major progress on the recognition of the tumor shape, the structural relationship of tumor sub-regions, and the segmentation of more challenging tumor sub-regions, e.g., the tumor core, and enhancing tumor.

Diffeomorphic deformable image registration is crucial in many medical image studies, as it offers unique, special features including topology preservation and invertibility of the transformation. Recent deep learning-based deformable image registration methods achieve fast image registration by leveraging a convolutional neural network (CNN) to learn the spatial transformation from the synthetic ground truth or the similarity metric. However, these approaches often ignore the topology preservation of the transformation and the smoothness of the transformation which is enforced by a global smoothing energy function alone. Moreover, deep learning-based approaches often estimate the displacement field directly, which cannot guarantee the existence of the inverse transformation. In this paper, we present a novel, efficient unsupervised symmetric image registration method which maximizes the similarity between images within the space of diffeomorphic maps and estimates both forward and inverse transformations simultaneously. We evaluate our method on 3D image registration with a large scale brain image dataset. Our method achieves state-of-the-art registration accuracy and running time while maintaining desirable diffeomorphic properties.

Groupwise registration is an important and challenging task for medical image processing and analysis. Traditional methods focus on generating a template and performing pairwise registration, which can be time-consuming to converge. In this paper, we propose an unsupervised end-to-end group-wise registration framework with multi-step mechanisms to progressively refine outputs. The framework can generate the displacement field for each subject directly without templates. Customized loss functions are designed to optimize the model and reduce the bias of generated common space. We experiment on 2D brain MRI coronal slices from OASIS and compare the results with two baseline methods using Dice score criterion. Results show that our framework achieves state-of-the-art performance with a much lower time cost.

There is a trend that digitalized clinical data increases dramatically every year. Part of data is multi-modal with imaging and non-imaging data such as phenotypic and genetic information. Though the success of CNNs has empowered a wide range of applications in learning from the imaging data, incorporating both the imaging and non-imaging data complementarily to improve the diagnostic quality is still challenging. To tackle this challenge, we propose a novel graph-convolutional model which is based on the proposed concept of edge adapter for learning an adaptive population graph from a multi-modal database. The edge adapter can be jointly optimized with the proposed graph convolutional neural network for semi-supervised node classification. Experimental results on two challenging multimodal medical databases demonstrate the potential of our method in learning from multi-modal data for disease diagnosis.

In this paper, a novel 3D deep learning network is proposed for brain MR image segmentation with randomized connection, which can decrease the dependency between layers and increase the network capacity. The convolutional LSTM and 3D convolution are employed as network units to capture the long-term and short-term 3D properties respectively. To assemble these two kinds of spatial-temporal information and refine the deep learning outcomes, we further introduce an efficient graph-based node selection and label inference method. Experiments have been carried out on two publicly available databases and results demonstrate that the proposed method can obtain competitive performances as compared with other state-of-the-art methods.

In this paper, a novel label fusion method is proposed for brain magnetic resonance image segmentation. This label fusion method is formulated on a graph, which embraces both label priors from atlases and anatomical priors from target image. To represent a pixel in a comprehensive way, three kinds of feature vectors are generated, including intensity, gradient, and structural signature. To select candidate atlas nodes for fusion, rather than exact searching, randomized k-d tree with spatial constraint is introduced as an efficient approximation for high-dimensional feature matching. Feature sensitive label prior (FSLP), which takes both the consistency and variety of different features into consideration, is proposed to gather atlas priors. As FSLP is a non-convex problem, one heuristic approach is further designed to solve it efficiently. Moreover, based on the anatomical knowledge, parts of the target pixels are also employed as the graph seeds to assist the label fusion process, and an iterative strategy is utilized to gradually update the label map. The comprehensive experiments carried out on two publicly available databases give results to demonstrate that the proposed method can obtain better segmentation quality.

In this paper, a novel unified framework for atlas-based segmentation is proposed, consisting of two main components: forward deformation and label refinement. A newly designed distance constraint on mesh edges is enforced with contrast sensitivity in forward deformation based on Markov random field. With the edge distance constraint, the object shapes in the atlas and the target images can remain similar during deformation. Considering the shape variations caused by individual difference, we then develop a label refinement process embracing patch registration and label fusion to compensate the small variations around the structural surfaces. As the anatomical correspondences determined in forward deformation can differ from that in label refinement, the conventional one-to-one correspondence constraint can be relaxed in our framework. Experiments on two publicly available databases IBSR and LPBA40 demonstrate that our method can obtain better performance as compared with other state-of-the-art methods.

In this paper, a novel label inference method encoded with local and global patch priors is introduced for the segmentation of subcortical structures in brain MR images. Due to the serious overlap of intensity profiles among different tissues in brain MR images, the conventional patch prior estimated with similarity measurement can be adversely impacted and become misleading during the final label inference procedure. As such, to obtain a more discriminative patch representation, we propose to capture local patch prior using sparse learning. Besides the local and low-level patch prior, the high-level structural properties of each subcortical structure are also taken into consideration and global patch prior is extracted with Convolutional Neural Networks. Experiments have been carried out on two publicly available datasets and results indicate that the proposed method can obtain the best performance as compared with other state-of-the-art methods.

In this paper, we propose a new framework for tackling face recognition problem. The face recognition problem is formulated as groupwise deformable image registration and feature matching problem. The main contributions of the proposed method lie in the following aspects: (1) Each pixel in a facial image is represented by an anatomical signature obtained from its corresponding most salient scale local region determined by the survival exponential entropy (SEE) information theoretic measure. (2) Based on the anatomical signature calculated from each pixel, a novel Markov random field based groupwise registration framework is proposed to formulate the face recognition problem as a feature guided deformable image registration problem. The similarity between different facial images are measured on the nonlinear Riemannian manifold based on the deformable transformations. (3) The proposed method does not suffer from the generalizability problem which exists commonly in learning based algorithms. The proposed method has been extensively evaluated on four publicly available databases: FERET, CAS-PEAL-R1, FRGC ver 2.0, and the LFW. It is also compared with several state-of-the-art face recognition approaches, and experimental results demonstrate that the proposed method consistently achieves the highest recognition rates among all the methods under comparison.

In this paper, we propose a graph-based method for 3D vessel tree structure segmentation based on a new tubularity Markov tree model (TMT), which works as both new energy function and graph construction method. With the help of power-watershed implementation [7], a global optimal segmentation can be obtained with low computational cost. Different with other graph-based vessel segmentation methods, the proposed method does not depend on any skeleton and ROI extraction method. The classical issues of the graph-based methods, such as shrinking bias and sensitivity to seed point location, can be solved with the proposed method thanks to vessel data fidelity obtained with TMT. The proposed method is compared with some classical graph-based image segmentation methods and two up-to-date 3D vessel segmentation methods, and is demonstrated to be more accurate than these methods for 3D vessel tree segmentation. Although the segmentation is done without ROI extraction, the computational cost for the proposed method is low (within 20 seconds for 256-256-144 image).

In this paper, we propose an optimal segmentation method for vascular forest structure based on graph cuts framework, which has widely been used in recent years because of its global optimal object segmentation property. However, shrinking bias, a classical issue of the graph cuts methods, sets up a barrier for the use of these methods on elongated structures such as blood vessels, especially the complex vascular tree and forest structures. To deal with this problem, a new graph construction method and a new energy function are proposed in this paper. The global optimal segmentation of vascular forest structure can be obtained more efficiently, while the shrinking bias can be overcome by the proposed method. The method is compared with a classical graph cuts method [1] and two methods [2, 3] for vascular tree structure segmentation, and is demonstrated to be more accurate on both the synthetic and clinical images, especially on noisy images. Different from many other tree structure segmentation methods, the proposed method does not have to consider the bifurcations explicitly.

Segmentation of intensity varying and low-contrast structures is an extremely challenging and rewarding task. In computer-aided diagnosis of intracranial aneurysms, segmenting the high-intensity major vessels along with the attached low-contrast aneurysms is essential to the recognition of this lethal vascular disease. It is particularly helpful in performing early and noninvasive diagnosis of intracranial aneurysms using phase contrast magnetic resonance angiographic (PC-MRA) images. The major challenges of developing a PC-MRA-based segmentation method are the significantly varying voxel intensity inside vessels with different flow velocities and the signal loss in the aneurysmal regions where turbulent flows occur. This paper proposes a novel intensity-based algorithm to segment intracranial vessels and the attached aneurysms. The proposed method can handle intensity varying vasculatures and also the low-contrast aneurysmal regions affected by turbulent flows. It is grounded on the use of multirange filters and local variances to extract intensity-based image features for identifying contrast varying vasculatures. The extremely low-intensity region affected by turbulent flows is detected according to the topology of the structure detected by multirange filters and local variances. The proposed method is evaluated using a phantom image volume with an aneurysm and four clinical cases. It achieves 0.80 dice score in the phantom case. In addition, different components of the proposed method-the multirange filters, local variances, and topology-based detection-are evaluated in the comparison between the proposed method and its lower complexity variants. Owing to the analogy between these variants and existing vascular segmentation methods, this comparison also exemplifies the advantage of the proposed method over the existing approaches. It analyzes the weaknesses of these existing approaches and justifies the use of every component involved in the proposed method. It is shown that the proposed method is capable of segmenting blood vessels and the attached aneurysms on PC-MRA images.

The goal of nonrigid image registration is to find a suitable transformation such that the transformed moving image becomes similar to the reference image. The image registration problem can also be treated as an optimization problem, which tries to minimize an objective energy function that measures the differences between two involved images. In this paper, we consider image matching as the process of aligning object boundaries in two different images. The registration energy function can be defined based on the total energy associated with the object boundaries. The optimal transformation is obtained by finding the equilibrium state when the total energy is minimized, which indicates the object boundaries find their correspondences and stop deforming. We make an analogy between the above processes with the dislocation system in physics. The object boundaries are viewed as dislocations (line defects) in crystal. Then the well-developed dislocation energy is used to derive the energy assigned to object boundaries in images. The newly derived registration energy function takes the global gradient information of the entire image into consideration, and produces an orientation-dependent and long-range interaction between two images to drive the registration process. This property of interaction endows the new registration framework with both fast convergence rate and high registration accuracy. Moreover, the new energy function can be adapted to realize symmetric diffeomorphic transformation so as to ensure one-to-one matching between subjects. In this paper, the superiority of the new method is theoretically proven, experimentally tested and compared with the state-of-the-art SyN method. Experimental results with 3-D magnetic resonance brain images demonstrate that the proposed method outperforms the compared methods in terms of both registration accuracy and computation time.

Liquid chromatography mass spectrometry (LC-MS) is widely used in comparing proteomes for disease biomarker discovery. An LC-MS experiment produces a 2-D image, where the mass-to-charge ratio and the chromatographic retention time are the coordinates, and the signal intensities represent the abundance of detected peptides. However, there is always a non-linear retention time difference across replicate LC-MS images due to machine drift, such that synchronization of LC-MS images must be performed prior to any further analysis. In this paper, we propose a multi-resolution image alignment scheme to synchronize LC-MS images. Dynamic Time Warping (DTW) is used to reconcile the time differences among images and Kullback-Leibler distance (KLD) is used as a local distance measure. Our proposed scheme has been validated using two real data sets, and promising results have been obtained.

In this paper, we propose a graph-based method to find the optimal cross section boundary for vessel segmentation. The voxels on the cross sectional plane are assumed to lay on the circles around the centerline point. The voxels on the circles with different radii are then transformed to a graph, by which the objective of finding the optimal boundary is converted to choosing the optimal path in the graph. A new cost function for the edge cost of the graph is proposed to obtain a smooth, optimal boundary of the cross section. Based on the optimal cross section boundary, we also propose a method for stenosis detection and quantification. The proposed method for segmentation and stenosis detection has been evaluated to be accurate and highly computationally efficient.

There are two main issues that make nonrigid image registration a challenging task. First, voxel intensity similarity may not be necessarily equivalent to anatomical similarity in the image correspondence searching process. Second, during the imaging process, some interferences such as unexpected rotations of input volumes and monotonic gray-level bias fields can adversely affect the registration quality. In this paper, a new feature-based nonrigid image registration method is proposed. The proposed method is based on a new type of image feature, namely, uniform spherical region descriptor (USRD), as signatures for each voxel. The USRD is rotation and monotonic gray-level transformation invariant and can be efficiently calculated. The registration process is therefore formulated as a feature matching problem. The USRD feature is integrated with the Markov random field labeling framework in which energy function is defined for registration. The energy function is then optimized by the α-expansion algorithm. The proposed method has been compared with five state-of-the-art registration approaches on both the simulated and real 3-D databases obtained from the BrainWeb and Internet Brain Segmentation Repository, respectively. Experimental results demonstrate that the proposed method can achieve high registration accuracy and reliable robustness behavior.

We present an energy-minimization-based framework for locating the centerline and estimating the width of tubelike objects from their structural network with a nonparametric model. The nonparametric representation promotes simple modeling of nested branches and n -way furcations, i.e., structures that abound in an arterial network, e.g., a cerebrovascular circulation. Our method is capable of extracting the entire vascular tree from an angiogram in a single execution with a proper initialization. A succinct initial model from the user with arterial network inlets, outlets, and branching points is sufficient for complex vasculature. The novel method is based upon the theory of principal curves. In this paper, theoretical extension to grayscale angiography is discussed, and an algorithm to find an arterial network as principal curves is also described. Quantitative validation on a number of simulated data sets, synthetic volumes of 19 BrainWeb vascular models, and 32 Rotterdam Coronary Artery volumes was conducted. We compared the algorithm to a state-of-the-art method and further tested it on two clinical data sets. Our algorithmic outputs-lumen centers and flow channel widths-are important to various medical and clinical applications, e.g., vasculature segmentation, registration and visualization, virtual angioscopy, and vascular atlas formation and population study.

Segmentation of deep brain structures is a challenging task for MRI images due to blurry structure boundaries, small object size and irregular shapes. In this paper, we present a new atlas-based segmentation method. It first uses a prior spatial dependency tree to constrain the relative positions between different deep brain structures and determine an optimal sequence for the structure by-structure segmentation. After positioning the structures, the segmentation result is further fine tuned by a non-rigid registration procedure between the atlas image and the target image using the histogram of the gradient magnitudes lying on the structure boundaries. The pro posed method has been applied on a publicly available MRI brain database and can achieve comparatively high segmentation accuracy.

Non-rigid image registration is widely used in medical image analysis and processing. We recently proposed a novel learning-based similarity measure for non-rigid image registration. The novel similarity measure is constructed by using two Kullback-Leibler distances (KLD), which are based on the a priori knowledge of the joint intensity distribution of a pre-aligned image pair. In this paper, we propose a new formulation for the novel KLD based similarity measure such that it can be exploited in Markov random field (MRF) based non-rigid registration framework with the graph-cuts algorithm. We have compared the proposed formulation against two other similarity measures under the same MRF-based framework, and two state-of-the-art approaches. According to the experimental results, it is demonstrated that the proposed method can achieve high registration accuracy.

Vessel enhancement is a primary preprocessing step for vessel segmentation and visualization of vasculatures. In this paper, a new vessel enhancement technique is proposed in order to produce accurate vesselness measures and vessel direction estimations that are less subject to local intensity abnormalities. The proposed method is called vessel enhancement using local line integrals and variational optimization (VE-LLI-VO). First, vessel enhancement using local line integrals (VE-LLI) is introduced in which a vessel model is embedded by regarding a vessel segment as a straight line based upon the second order information of the local line integrals. Useful quantities similar to the eigenvalues and eigenvectors of the Hessian matrix are produced. Moreover, based upon the local line integrals, junctions can be detected and handled effectively. This can help deal with the bifurcation suppression problem which exists in the Hessian-based enhancement methods. Then a more generic curve model is embedded to model vessels and a variational optimization (VO) framework is introduced to generate optimized vesselness measures. Experiments have been conducted on both synthetic images and retinal images. It is experimentally demonstrated that VE-LLI-VO produces improved performance as compared with the widely used techniques in terms of both vesselness measurement and vessel direction estimation.

In this paper, a new scheme to address the face recognition problem is proposed. Different from traditional face recognition approaches which represent each facial image by a single feature vector as the classification problem, the proposed method establishes a new way to formulate the face recognition problem as a deformable image registration problem. The main contributions of the paper lie in the following aspects: (i) Each pixel is represented by an anatomical feature signature calculated from its corresponding best scale salient region by using a new salient region detector based on the survival exponential entropy (SEE); (ii) The face recognition problem is formulated as a deformable image registration problem, the deformation model is represented by a Markov random field (MRF) labeling framework. Explicit pixel correspondence is established by the deformation framework. (iii) The survival exponential entropy based normalized mutual information (SEE-NMI) is proposed and integrated with the MRF based deformation model as the similarity measure to reflect the similarity between two facial images. The proposed method is evaluated on the FERET and FRGC version 2 databases and compared with several state-of-the-art face recognition approaches. Experimental results show that the proposed method achieves the highest recognition rate among all the compared approaches.

This paper proposes a new texture classification approach. There are two main contributions in the proposed method. First, input texture images are transformed to the composite Fourier domain (CFD) by using both the local and global Fourier transforms. The composite Fourier domain is rotation invariant and preserves the contextual information for the texture images in the original spatial domain. Second, the null-space based linear discriminant analysis (nLDA) is adopted to find the optimal representations of the texture images in the composite Fourier domain. This paper proposes a systematic way to cooperate subspace learning methods for texture classification in the frequency domain, which cannot be directly applied in the spatial domain for texture classification. The proposed method is evaluated on both the Brodatz and CUReT databases and compared with several state-of-the-art texture classification approaches. Experimental results show that the proposed method achieves the highest classification rate among all the compared methods.

Non-rigid image registration plays an important role in medical image analysis. Recently, Tang and Chung proposed to model the non-rigid medical image registration problem as an energy minimization framework. The optimization was done by using the graph-cuts algorithm via alpha-expansions. However, the dissimilarity measure used in the energy function of this graph-cuts based method was restricted to the sum of absolute differences (SAD) and the sum of squared differences (SSD). In this paper, to utilize an advanced dissimilarity measure, such as mutual information (MI), we adopt an approximation of MI to the graph-cuts based method. Exploiting the mutual information is valuable as it can capture complex statistical relationships between the intensities of the image pair without a priori knowledge of those relationships. We have compared the proposed method against the original graph-cuts based methods, and two state-of-the-art approaches. The experimental results demonstrate that the proposed method can achieve lower registration errors.

A new feature based nonrigid image registration method for magnetic resonance (MR) brain images is presented in this paper. Each image voxel is represented by a rotation invariant feature vector, which is computed by passing the input image volumes through a new bank of symmetric alpha stable (SalphaS) filters. There are three main contributions presented in this paper. First, this work is motivated by the fact that the frequency spectrums of the brain MR images often exhibit non-Gaussian heavy-tail behavior which cannot be satisfactorily modeled by the conventional Gabor filters. To this end, we propose the use of SalphaS filters to model such behavior and show that the Gabor filter is a special case of the SalphaS filter. Second, the maximum response orientation (MRO) selection criterion is designed to extract rotation invariant features for registration tasks. The MRO selection criterion also significantly reduces the number of dimensions of feature vectors and therefore lowers the computation time. Third, in case the segmentations of the input image volumes are available, the Fisher's separation criterion (FSC) is introduced such that the discriminating power of different feature types can be directly compared with each other before performing the registration process. Using FSC, weights can also be assigned automatically to different voxels in the brain MR images. The weight of each voxel determined by FSC reflects how distinctive and salient the voxel is. Using the most distinctive and salient voxels at the initial stage to drive the registration can reduce the risk of being trapped in the local optimum during image registration process. The larger the weight, the more important the voxel. With the extracted feature vectors and the associated weights, the proposed method registers the source and the target images in a hierarchical multiresolution manner. The proposed method has been intensively evaluated on both simulated and real 3-D datasets obtained from BrainWeb and Internet Brain Segmentation Repository (IBSR), respectively, and compared with HAMMER, an extended version of HAMMER based on local histograms (LHF), FFD, Demons, and the Gabor filter based registration method. It is shown that the proposed method achieves the highest registration accuracy among the five widely used image registration methods.

In this paper, we present a new approach for segmentation of tubular structures in 2D images providing minimal interaction. The main objective is to extract centerlines and boundaries of the vessels at the same time. The first step is to represent the trajectory of the vessel not as a 2D curve but to go up a dimension and represent the entire vessel as a 3D curve, where each point represents a 2D disc (two coordinates for the center point and one for the radius). The 2D vessel structure is then obtained as the envelope of the family of discs traversed along this 3D curve. Since this 2D shape is defined simply from a 3D curve, we are able to fully exploit minimal path techniques to obtain globally minimizing trajectories between two or more user supplied points using front propagation. The main contribution of our approach consists on building a multi-resolution metric that guides the propagation in this 3D space. We have chosen to exploit the tubular structure of the vessels one wants to extract to built an anisotropic metric giving higher speed on the center of the vessels and also when the minimal path tangent is coherent with the vessel's direction. This measure is required to be robust against the disturbance introduced by noise or adjacent structures with intensity similar to the target vessel. Indeed, if we examine the flux of the projected image gradient along a given direction on a circle of a given radius (or scale), one can prove that this flux is maximal at the center of the vessel, in its direction and with its exact radius. This approach is called optimally oriented flux. Combining anisotropic minimal paths techniques and optimally oriented flux we obtain promising results on noisy synthetic and real data.

Non-rigid image registration is widely used in medical image analysis and image processing. It remains a challenging research problem due to its smoothness requirement and high degree of freedoms in the deformation process. A method is proposed to solve non-rigid image registration via graph-cuts algorithm by modeling the registration process as a discrete labeling problem. A displacement label (vector) is assigned to each pixel in the source image to indicate the corresponding position in the floating image. The whole system is then optimized by using the graph-cuts algorithm via alpha-expansions. As the initial point is not required for the graph-cuts algorithm, the method proposed is a single-level registration. In this paper, rather single-level, we enable multi-level non-rigid image registration using graph-cuts by passing the deformation field of the current resolution level to the successive finer one. By applying the proposed multi-level registration method, the number of labels used in each level is greatly reduced due to lower image resolution being used in coarser levels. Therefore, the speed of the registration process is improved. We compare our results with the original single-level version, DEMONS and FFD. It is found that our method improves the speed of non-rigid image registration by 50% and meanwhile maintains similar robustness and registration accuracy with the single-level version.

This paper proposes a new face recognition method. There are two novelties in the proposed method. First, a new saliency measure function is designed to detect the most salient regions in facial images and determine their corresponding best scales. Second, a new type of image feature, called local gradient orientation binary pattern (LGOBP) is proposed, which captures the neighborhood gradient orientation information which is not considered in the conventional local binary patterns (LBP) to give more discriminant power. LGOBPs are extracted from the most salient regions selected by the proposed saliency measure function. The proposed method is evaluated on the FRGC version 2 database by comparing it with several widely used methods. Experimental results show that the proposed method achieves the highest recognition rate among all the compared methods.

Spherical flux is the flux inside a spherical region, and it is very useful in the analysis of tubular structures in magnetic resonance angiography and computed tomographic angiography. The conventional approach is to estimate the spherical flux in the spatial domain. Its running time depends on the sphere radius quadratically, which leads to very slow spherical flux computation when the sphere size is large. This paper proposes a more efficient implementation for spherical flux computation in the Fourier domain. Our implementation is based on the reformulation of the spherical flux calculation using the divergence theorem, spherical step function, and the convolution operation. With this reformulation, most of the calculations are performed in the Fourier domain. We show how to select the frequency subband so that the computation accuracy can be maintained. It is experimentally demonstrated that, using the synthetic and clinical phase contrast magnetic resonance angiographic volumes, our implementation is more computationally efficient than the conventional spatial implementation. The accuracies of our implementation and that of the conventional spatial implementation are comparable. Finally, the proposed implementation can definitely benefit the computation of the multiscale spherical flux with a set of radii because, unlike the conventional spatial implementation, the time complexity of the proposed implementation does not depend on the sphere radius.

This paper proposes a novel approach to extract image features for texture classification. The proposed features are robust to image rotation, less sensitive to histogram equalization and noise. It comprises of two sets of features: dominant local binary patterns (DLBP) in a texture image and the supplementary features extracted by using the circularly symmetric Gabor filter responses. The dominant local binary pattern method makes use of the most frequently occurred patterns to capture descriptive textural information, while the Gabor-based features aim at supplying additional global textural information to the DLBP features. Through experiments, the proposed approach has been intensively evaluated by applying a large number of classification tests to histogram-equalized, randomly rotated and noise corrupted images in Outex, Brodatz, Meastex, and CUReT texture image databases. Our method has also been compared with six published texture features in the experiments. It is experimentally demonstrated that the proposed method achieves the highest classification accuracy in various texture databases and image conditions.

This paper presents a new method to enforce inverse consistency in nonrigid image registration and matching. Conventional approaches assume diffeomorphic transformation, implicitly or explicitly. However, the inherent smoothness constraint discourages discontinuity consideration. We propose a post-processing algorithm that integrates the input forward and backward fields, which are output by existing registration/matching algorithms, to produce more robust results. Given such a pair of input fields, our algorithm alternately refines the fields by tensor belief propagation, and enforces inverse consistency in stochastic sense by generalized total least squares fitting. To show the efficacy of our stochastic inverse consistency approach, we first present results on very noisy fields. We then demonstrate improvement on existing stereo matching where occlusion is naturally handled by localizing violations of inverse consistency. Finally, we propose a novel application on image stitching, where stochastic inverse consistency is employed in structure deformation, in order to seamlessly align overlapping images with severe misalignment in structure and intensity.

In this paper, based on the SalphaS distributions, we design SalphaS filters and use the filters as a new feature extraction method for non-rigid medical image registration. In brain MR images, the energy distributions of different frequency bands often exhibit heavy-tailed behavior. Such non-Gaussian behavior is essential for non-rigid image registration but cannot be satisfactorily modeled by the conventional Gabor filters. This leads to unsatisfactory modeling of voxels located at the salient regions of the images. To this end, we propose the SalphaS filters for modeling the heavy-tailed behavior of the energy distributions of brain MR images, and show that the Gabor filter is a special case of the SalphaS filter. The maximum response orientation selection criterion is defined for each frequency band to achieve rotation invariance. In our framework, if the brain MR images are already segmented, each voxel can be automatically assigned a weighting factor based on the Fisher's separation criterion and it is shown that the registration performance can be further improved. The proposed method has been compared with the free-form-deformation based method, Demons algorithm and a method using Gabor features by conducting non-rigid image registration experiments. It is observed that the proposed method achieves the best registration accuracy among all the compared methods in both the simulated and real datasets obtained from the BrainWeb and IBSR respectively.

In this paper, we propose a new feature extraction method, which is robust against rotation and histogram equalization for texture classification. To this end, we introduce the concept of advanced local binary patterns (ALBP), which reflects the local dominant structural characteristics of different kinds of textures. In addition, to extract the global spatial distribution feature of the ALBP patterns, we incooperate ALBP with the aura matrix measure as the second layer to analyze texture images. The proposed method has three novel contributions, (a) The proposed ALBP approach captures the most essential local structure characteristics of texture images (i.e. edges, corners); (b) the proposed method extracts global information by using Aura matrix measure based on the spatial distribution information of the dominant patterns produced by ALBP; and (c) the proposed method is robust to rotation and histogram equalization. The proposed approach has been compared with other widely used texture classification techniques and evaluated by applying classification tests to randomly rotated and histogram equalized images in two different texture databases: Brodatz and CUReT. The experimental results show that the classification accuracy of the proposed method exceeds the ones obtained by other image features.

This paper introduces a novel energy minimization method, namely iterated cross entropy with partition strategy (ICEPS), into the Markov random field theory. The solver, which is based on the theory of cross entropy, is general and stochastic. Unlike some popular optimization methods such as belief propagation (BP) and graph cuts (GC), ICEPS makes no assumption on the form of objective functions and thus can be applied to any type of Markov random field (MRF) models. Furthermore, compared with deterministic MRF solvers, it achieves higher performance of finding lower energies because of its stochastic property. We speed up the original cross entropy algorithm by partitioning the MRF site set and assure the effectiveness by iterating the algorithm. In the experiments, we apply ICEPS to two MRF models for medical image segmentation and show the aforementioned advantages of ICEPS over other popular solvers such as iterated conditional modes (ICM) and GC.

Markov random field (MRF) theory has been widely applied to the challenging problem of image segmentation. In this paper, we propose a new nontexture segmentation model using compound MRFs, in which the original label MRF is coupled with a new boundary MRF to help improve the segmentation performance. The boundary model is relatively general and does not need prior training on boundary patterns. Unlike some existing related work, the proposed method offers a more compact interaction between label and boundary MRFs. Furthermore, our boundary model systematically takes into account all the possible scenarios of a single edge existing in a 3 x 3 neighborhood and, thus, incorporates sophisticated prior information about the relation between label and boundary. It is experimentally shown that the proposed model can segment objects with complex boundaries and at the same time is able to work under noise corruption. The new method has been applied to medical image segmentation. Experiments on synthetic images and real clinical datasets show that the proposed model is able to produce more accurate segmentation results and satisfactorily keep the delicate boundary. It is also less sensitive to noise in both high and low signal-to-noise ratio regions than some of the existing models in common use.

Accurate detection of vessel boundaries is particularly important for a precise extraction of vasculatures in magnetic resonance angiography (MRA). In this paper, we propose the use of weighted local variance (WLV)-based edge detection scheme for vessel boundary detection in MRA. The proposed method is robust against changes of intensity contrast of edges and capable of giving high detection responses on low contrast edges. These robustness and capabilities are essential for detecting the boundaries of vessels in low contrast regions of images, which can contain intensity inhomogeneity, such as bias field, interferences induced from other tissues, or fluctuation of the speed related vessel intensity. The performance of the WLV-based edge detection scheme is studied and shown to be able to return strong and consistent detection responses on low contrast edges in the experiments. The proposed edge detection scheme can be embedded naturally in the active contour models for vascular segmentation. The WLV-based vascular segmentation method is tested using MRA image volumes. It is experimentally shown that the WLV-based edge detection approach can achieve high-quality segmentation of vasculatures in MRA images.

Although the presence of local minima is one of the major problems in high-dimensional image registration, only a few experimental works have been carried out to address this problem. In this study, a 3D-2D vascular image feature-based registration is done by producing Digital Reconstructed Radiographs (DRRs) of 3D images to match against the target 2D images. In addition, we propose a global optimization method based on the use of Powell's method at different resolution levels. To search the global minimum as effectively as possible, a large set of sample test points are systematically generated. The values of dissimilarity to the registered images in lower resolution environment are calculated. Powell's method is then applied to those test points with the lowest values for further minimization in the higher resolution. It is experimentally shown that our method can identify the global optimum in a normal clinical setting. The findings can have potential usage in the reconstruction of 3D models (e.g. guide wires) based on 2D medical visual information.

Computer-assisted detection and segmentation of blood vessels in angiography are crucial for endovascular treatments and embolization. In this article, I give an overview of the image segmentation methods using the features developed recently at our laboratory. Our current research directions are also highlighted

To segment a whole object from an image is an essential and challenging task in image processing. In this paper, we propose a hybrid segmentation algorithm which combines prior shape information with normalized cut. With the help of shape information, we can utilize normalized cut to correctly segment the target whose boundary may be corrupted by noise or outliers. At the same time, we introduce the use of segmentation results of the normalized cut to guide the shape model, and thus avoid searching the shape space. The proposed method was demonstrated to be effective by our experiments on both synthetic and real data.

This paper proposes a novel facial expression recognition approach based on two sets of features extracted from the face images: texture features and global appearance features. The first set is obtained by using the extended local binary patterns in both intensity and gradient maps and computing the Tsallis entropy of the Gabor filtered responses. The second set of features is obtained by performing null-space based linear discriminant analysis on the training face images. The proposed method is evaluated by extensive experiments on the JAFFE database, and compared with two widely used facial expression recognition approaches. Experimental results show that the proposed approach maintains high recognition rate in a wide range of resolution levels and outperforms the other alternative methods.

Endovascular treatment plays an important role in the minimally invasive treatment of patients with vascular diseases, a major cause of morbidity and mortality worldwide. Given a segmentation of an angiography, quantitative analysis of abnormal structures can aid radiologists in choosing appropriate treatments and apparatuses. However, effective quantitation is only attainable if the abnormalities are identified from the vasculature. To achieve this, a novel method is developed, which works on the simpler shape of normal vessels to identify different vascular abnormalities (viz. stenotic atherosclerotic plaque, and saccular and fusiform aneurysmal lumens) in an indirect fashion, instead of directly manipulating the complex-shaped abnormalities. The proposed method has been tested on three synthetic and 17 clinical data sets. Comparisons with two related works are also conducted. Experimental results show that our method can produce satisfactory identification of the abnormalities and approximations of the ideal post-treatment vessel lumens. In addition, it can help increase the repeatability of the measurement of clinical parameters significantly.

Image segmentation is defined as partitioning an image into non-overlapping regions based on the intensity or texture. The active contour methods provide an effective way for segmentation, in which the boundary of an object (usually with large image gradient value) is detected by an evolving curve. But, these methods have limitations due to the fact that real images may have objects with complex geometric structures and shapes, and are often corrupted by noise. Developing more robust and accurate active contour methods has been an active research area since the idea of the methods was proposed. In this paper, we propose a new active contour method and apply the method to medical image segmentation. This new method uses a long-ranged interaction between image boundaries and the moving curves, which is inspired by the elastic interaction between line defects in solids (dislocations). The new method is more efficient and effective, especially in detecting thin, weak and blurred structures such as the images of blood vessels.

Markov random field (MRF) theory has widely been applied to segmentation in noisy images. This paper proposes a new MRF method. First, it couples the original labeling MRF with a boundary MRF that can help improve the performance of segmentation. Second, the boundary model is general and does not need prior training. Third, unlike existing related work, our model offers more compact interaction between the two MRFs. Experiments on synthetic images and real clinical datasets show that the proposed approach is able to produce good segmentation results, especially removing noise in low signal-to-noise ratio regions.

Image segmentation is a fundamental problem in early computer vision. In segmentation of flat shaded, nontextured objects in real-world images, objects are usually assumed to be piecewise homogeneous. This assumption, however, is not always valid with images such as medical images. As a result, any techniques based on this assumption may produce less-than-satisfactory image segmentation. In this work, we relax the piecewise homogeneous assumption. By assuming that the intensity nonuniformity is smooth in the imaged objects, a novel algorithm that exploits the coherence in the intensity profile to segment objects is proposed. The algorithm uses a novel smoothness prior to improve the quality of image segmentation. The formulation of the prior is based on the coherence of the local structural orientation in the image. The segmentation process is performed in a Bayesian framework. Local structural orientation estimation is obtained with an orientation tensor. Comparisons between the conventional Hessian matrix and the orientation tensor have been conducted. The experimental results on the synthetic images and the real-world images have indicated that our novel segmentation algorithm produces better segmentations than both the global thresholding with the maximum likelihood estimation and the algorithm with the multilevel logistic MRF model.

In this paper, we present an approach to segmenting the brain vasculature in phase contrast magnetic resonance angiography (PC-MRA). According to our prior work, we can describe the overall probability density function of a PC-MRA speed image as either a Maxwell-uniform (MU) or Maxwell-Gaussian-uniform (MGU) mixture model. An automatic mechanism based on Kullback-Leibler divergence is proposed for selecting between the MGU and MU models given a speed image volume. A coherence measure, namely local phase coherence (LPC), which incorporates information about the spatial relationships between neighboring flow vectors, is defined and shown to be more robust to noise than previously described coherence measures. A statistical measure from the speed images and the LPC measure from the phase images are combined in a probabilistic framework, based on the maximum a posteriori method and Markov random fields, to estimate the posterior probabilities of vessel and background for classification. It is shown that segmentation based on both measures gives a more accurate segmentation than using either speed or flow coherence information alone. The proposed method is tested on synthetic, flow phantom and clinical datasets. The results show that the method can segment normal vessels and vascular regions with relatively low flow rate and low signal-to-noise ratio, e.g., aneurysms and veins.

We present a new multimodal image registration method based on the a priori knowledge of the class label mappings between two segmented input images. A joint class histogram between the image pairs is estimated by assigning each bin value equal to the total number of occurrences of the corresponding class label pairs. The discrepancy between the observed and expected joint class histograms should be minimized when the transformation is optimal. Kullback-Leibler distance (KLD) is used to measure the difference between these two histograms. Based on the probing experimental results on a synthetic dataset as well as a pair of precisely registered 3D clinical volumes, we show that, with the knowledge of the expected joint class histogram, our method obtained longer capture range and fewer local optimal points as compared with the conventional mutual information (MI) based registration method. We also applied the proposed method to a 2D-3D rigid registration problems between DSA and MRA volumes. Based on manually selected markers, we found that the accuracies of our method and the MI-based method are comparable. Moreover, our method is more computationally efficient than the MI-based method.

In sensory data fusion and integration consideration, sensor independence is a common assumption. We demonstrate the impact of including dependent information in the sensory data combination process. The team consensus approach based on information entropy can improve the measurement accuracy remarkably. The major benefits of the approach are: (a) the simple linear combination of the weighted initial local estimates for each sensor; and (b) the low order bivariate likelihood functions which can be represented easily. A comparison of the team consensus approach with the Bayesian approach is presented.

It is common to assume sensor independence in the sensory data fusion and integration. The authors previously (1997, 1998) illustrated that the team consensus approach based on information entropy can remarkably improve the measurement accuracy. The major benefits of the approach are (a) the simple linear combination of the weighted initial expected estimates for each sensor; and (b) the low order bivariate likelihood functions which can be represented easily. In this paper, we demonstrate specifically both the positive and negative impacts of including dependent information in sensory data combination process; and show how the measurable consensus uncertainty level can be derived. A comparison of the team consensus approach with the Bayesian approach is presented.

In this paper, a decentralized approach based on the team consensus approach and Markovian model is proposed to integrate multisensory data. A team of sensors can estimate the local and global uncertainties utilizing self-entropy and conditional-entropy measures of the sensors. Consensus can be reached based on the initial expected values and "uncertainty" weights assigned by the sensors. The proposed approach is compared with the Bayesian approach via experiments on two independent sensors. Results showed that consensus reached are comparable. However, there are factors that indicated the decentralized approach requires less communication and computational effort to reach consensus among sensors.