Quantification of carotid atherosclerosis is important in monitoring patients at risk of cardiovascular events and in evaluating therapies. High-resolution 3D carotid magnetic resonance imaging (MRI) has been developed to provide extended coverage of the carotid arteries. However, the extended coverage poses a challenge as several hundreds of 2D axial images are required to be segmented for analysis. We propose a multi-dimensional hybrid framework that requires only a sparse set of manual segmentation. Dense surrogate ground truth boundaries required to train the framework are automatically generated by propagating the sparse manual segmentation using the proposed region of interest (ROI) U-Net. Furthermore, the Point U-Net was developed to generate surrogate ground truth for carotid branches without manual segmentation. The proposed framework leverages the advantages of 3D and 2D convolution neural networks (CNNs) to segment the outer wall and lumen from 3D MRI. The 3D multiscale U-Net provides a rough outer wall segmentation, which serves as the ROI to guide outer wall and lumen segmentation by the 2D ROI U-Net. The 3D Multiscale U-Net localizes the ROI automatically, bypassing the need for manual ROI identification. The 3D Multiscale U-Net was further improved by a 3D inception module installed at the bottleneck and the novel loss functions that promote longitudinal continuity and minimize the overlap of the internal and external carotid arteries. Extensive evaluation on the publicly available Carotid Artery Vessel Wall Segmentation challenge dataset shows that our approach outperforms the top-ranked solution in the challenge and state-of-the-art segmentation methods.

The large amount of data available in the modern big data era opens new opportunities to expand our knowledge by integrating information from heterogeneous sources. Multiview learning has recently achieved tremendous success in deriving complementary information from multiple data modalities. This article proposes a framework called multiview latent space projection (MVLSP) to integrate features extracted from multiple sources in a discriminative way to facilitate binary and multiclass classifications. Our approach is associated with three innovations. First, most existing multiview learning algorithms promote pairwise consistency between two views and do not have a natural extension to applications with more than two views. MVLSP finds optimum mappings from a common latent space to match the feature space in each of the views. As the matching is performed on a view-by-view basis, the framework can be readily extended to multiview applications. Second, feature selection in the common latent space can be readily achieved by adding a class view, which matches the latent space representations of training samples with their corresponding labels. Then, high-order view correlations are extracted by considering feature-label correlations. Third, a technique is proposed to optimize the integration of different latent patterns based on their correlations. The experimental results on the prostate image dataset demonstrate the effectiveness of the proposed method.

The fusion of multiview data sets, in which features of each sample are categorized into distinct groups, is increasingly important in the big data era. Successful multiview learning approaches have mechanisms to enforce consensus and/or complementarity among views. This article introduces a framework called the consensus and complementarity-based multiview latent space projection (MVLSP-2C) that enforces both principles simultaneously. Consensus is established by extracting and representing information shared by all views in a shared latent space, whereas complementarity among views is achieved by the representation in view-specific spaces. As the diversity of the multiview feature representation benefits classification performance, MVLSP-2C minimizes the similarity between the shared and view-specific representations, thereby improving diversity. The driving principle of MVLSP-2C is that the latent space representation is obtained by optimally projecting it to match the original feature space representation on a view-by-view basis. Unlike pairwise consensus methods that enforce consistency between two views, matching on a view-by-view basis allows extensions to settings with more than two views. A related and important advantage of this per-view matching design is that a class view can be readily incorporated to learn a supervised representation that facilitates subsequent classification. As the class view is added without an assumption on the exclusivity of classes, MVLSP-2C is equally applicable to multiclass single-label and multilabel classifications. MVLSP-2C further optimizes the integration of latent variables based on their correlation. Extensive experiments in multiclass and multiview image datasets show that MVLSP-2C produces more accurate classification results as compared to state-of-the-art methods.

Segmentation of the carotid section encompassing the common carotid artery (CCA), the bifurcation and the internal carotid artery (ICA) from three-dimensional ultrasound (3DUS) is required to measure the vessel wall volume (VWV) and localized vessel-wall-plus-plaque thickness (VWT), shown to be sensitive to treatment effect. We proposed an approach to combine a centerline extraction network (CHG-Net) and a dual-stream centerline-guided network (DSCG-Net) to segment the lumen-intima (LIB) and media-adventitia boundaries (MAB) from 3DUS images. Correct arterial location is essential for successful segmentation of the carotid section encompassing the bifurcation. We addressed this challenge by using the arterial centerline to enhance the localization accuracy of the segmentation network. The CHG-Net was developed to generate a heatmap indicating high probability regions for the centerline location, which was then integrated with the 3DUS image by the DSCG-Net to generate the MAB and LIB. The DSCG-Net includes a scale-based and a spatial attention mechanism to fuse multi-level features extracted by the encoder, and a centerline heatmap reconstruction side-branch connected to the end of the encoder to increase the generalization ability of the network. Experiments involving 224 3DUS volumes produce a Dice similarity coefficient (DSC) of 95.8±1.9% and 92.3±5.4% for CCA MAB and LIB, respectively, and 93.2±4.4% and 89.0±10.0% for ICA MAB and LIB, respectively. Our approach outperformed four state-of-the-art 3D CNN models, even after their performances were boosted by centerline guidance. The efficiency afforded by the framework would allow it to be incorporated into the clinical workflow for improved quantification of plaque change.

Multilabel learning, which handles instances associated with multiple labels, has attracted much attention in recent years. Many extant multilabel feature selection methods target global feature selection, which means feature selection weights for each label are shared by all instances. Also, many extant multilabel classification methods exploit global label selection, which means labels correlations are shared by all instances. In real-world objects, however, different subsets of instances may share different feature selection weights and different label correlations. In this article, we propose a novel framework with local feature selection and local label correlation, where we assume instances can be clustered into different groups, and the feature selection weights and label correlations can only be shared by instances in the same group. The proposed framework includes a group-specific feature selection process and a label-specific group selection process. The former process projects instances into different groups by extracting the instance-group correlation. The latter process selects labels for each instance based on its related groups by extracting the group-label correlation. In addition, we also exploit the intergroup correlation. These three kinds of group-based correlations are combined to perform effective multilabel classification. The experimental results on various datasets validate the effectiveness of our approach.

While three-dimensional (3D) late gadolinium-enhanced (LGE) magnetic resonance (MR) imaging provides good conspicuity of small myocardial lesions with short acquisition time, it poses a challenge for image analysis as a large number of axial images are required to be segmented. We developed a fully automatic convolutional neural network (CNN) called cascaded triplanar autoencoder M-Net (CTAEM-Net) to segment myocardial scar from 3D LGE MRI. Two sub-networks were cascaded to segment the left ventricle (LV) myocardium and then the scar within the pre-segmented LV myocardium. Each sub-network contains three autoencoder M-Nets (AEM-Nets) segmenting the axial, sagittal and coronal slices of the 3D LGE MR image, with the final segmentation determined by voting. The AEM-Net integrates three features: (1) multi-scale inputs, (2) deep supervision and (3) multi-tasking. The multi-scale inputs allow consideration of the global and local features in segmentation. Deep supervision provides direct supervision to deeper layers and facilitates CNN convergence. Multi-task learning reduces segmentation overfitting by acquiring additional information from autoencoder reconstruction, a task closely related to segmentation. The framework provides an accuracy of 86.43% and 90.18% for LV myocardium and scar segmentation, respectively, which are the highest among existing methods to our knowledge. The time required for CTAEM-Net to segment LV myocardium and the scar was 49.72 ± 9.69s and 120.25 ± 23.18s per MR volume, respectively. The accuracy and efficiency afforded by CTAEM-Net will make possible future large population studies. The generalizability of the framework was also demonstrated by its competitive performance in two publicly available datasets of different imaging modalities.

Preterm neonates with a very low birth weight of less than 1,500 grams are at increased risk for developing intraventricular hemorrhage (IVH), which is a major cause of brain injury in preterm neonates. Quantitative measurements of ventricular dilatation or shrinkage play an important role in monitoring patients and evaluating treatment options. 3D ultrasound (US) has been developed to monitor ventricle volume as a biomarker for ventricular changes. However, ventricle volume as a global indicator does not allow for precise analysis of local ventricular changes, which could be linked to specific neurological problems often seen in the patient population later in life. In this work, a 3D+t spatial-temporal deformable registration approachis proposed, which is applied to the analysis of the detailed local changes of preterm IVH neonatal ventricles from 3D US images. In particular, a novel sequential convex/dual optimization algorithm is introduced to extract the optimal 3D+t spatial-temporal deformable field, which simultaneously optimizes the sequence of 3D deformation fieldswhile enjoying both efficiencyand simplicity in numerics. The developed registration technique was evaluated by comparing two manually extracted ventricle surfaces from the baseline and the registered follow-up images using the metrics of Dice similarity coefficient (DSC), mean absolute surface distance (MAD), and maximum absolute surface distance (MAXD). The performed experiments using 14 patients with 5 time-point images per patient show that the proposed 3D+t registration approach accurately recovered the longitudinal deformation of ventricle surfaces from 3D US images. The proposed approach may be potentially used to analyse the change pattern of cerebral ventricles of IVH patients, their response to different treatment options, and to elucidate the deficiencies that a patient could have later in life. To the best of our knowledge, this paper reports the first study on the longitudinalanalysis of neonatal ventricular system from 3D US images.

Atherosclerosis at the carotid bifurcation resulting in cerebral emboli is a major cause of ischemic stroke. Most strokes associated with carotid atherosclerosis can be prevented by lifestyle/dietary changes and pharmacological treatments if identified early by monitoring carotid plaque changes. Registration of 3-D ultrasound (US) images of carotid plaque obtained at different time points is essential for sensitive monitoring of plaque changes in volume and surface morphology. This registration technique should be nonrigid, since different head positions during image acquisition sessions cause relative bending and torsion in the neck, producing nonlinear deformations between the images. We modeled the movement of the neck using a "twisting and bending" model with only six parameters for nonrigid registration. We evaluated the algorithm using 3-D US carotid images acquired at two different head positions to simulate images acquired at different times. We calculated the mean registration error (MRE) between the segmented vessel surfaces in the target image and the registered image using a distance-based error metric after applying our "twisting and bending" model-based nonrigid registration algorithm. We achieved an average registration error of 0.80 +/-0.26 mm using our nonrigid registration technique, which was a significant improvement in registration accuracy over rigid registration, even with reduced degrees-of-freedom compared to the other nonrigid registration algorithms.