The fusion of hyperspectral images (HSIs) and multispectral images (MSIs) has been widely focused in the field of remote sensing image processing. Traditional fusion approaches have no consideration of the topological consistency between the target high-resolution HSI (HR-HSI) and the input HSI/MSI. To remedy such deficiency, in this letter, we propose a novel topological consistency network (TCNet) for HSI and MSI fusion. Specifically, two degradation models are firstly constructed to form the spatial mapping between the input HSI and the target HR-HSI as well as the spectral mapping between the MSI and the HR-HSI. With this way, the basic observation model is formulated. The spectral topological consistency loss and spatial topological consistency loss are then established by imposing graph construction on spectral vectors and spatial features, which are added into the basic observation model to acquire the overall loss of TCNet. Thirdly, the gradient descent based optimization strategy is designed to obtain the solutions of the overall loss and determine the final fusion mapping network. Extensive experiments have been implemented to validate that the proposed TCNet method achieves more competitive performance compared with several state-of-the-art approaches.

High resolution hyperspectral fusion imaging (HRHFI) is an important topic in the remote sensing related tasks, which integrates advantages of hyperspectral images (HSIs) and multipectral images (MSIs) to generate the high resolution HSIs (HR-HSIs). Traditional methods fail to maintain multi-scale change information and directional information of HSIs and MSIs during the HRHFI process, limiting the performance. To solve this challenge, a novel hierarchical gradient preserved network (HGPN) is proposed for HRHFI. Concretely, we begin by establishing the foundational observation model, which serves to make an initial estimate of HR-HSI so that the basic observation loss is thus constructed. Subsequently, we integrate both multi-scale spectral gradient preservation loss and multi-scale spatial gradient preservation loss into the basic observation loss, which results in the overall loss function of the proposed HGPN method. Further, an optimization strategy based on alternating directions is devised to solve the overall loss and derive the corresponding solutions, which finalizes the imaging fusion mapping. Different from traditional approaches, the proposed HGPN method can achieve hierarchical fusion, which retains multi-scale gradient information of input HSI/MSI for HRHFI. Extensive experiments have been implemented to verify that the proposed HGPN method achieves more competitive performance than several state-of-the-art approaches in the HRHFI task.

The characteristic of neurodegenerative disorders is the progressive impairment of memory and other cognitive functions. However, these existing imaging genetic methods only use longitudinal imaging phenotypes straightforwardly, ignoring the latent pattern of the longitudinal data in the progression process. The phenotypes across multiple time-points may exhibit the latent pattern that can be used to facilitate the understanding of the progression process. Accordingly, in this paper, we explore underlying complementary information from multiple time-points and simultaneously seek the underlying latent representation. With the complementarity of multiple time-points, the latent representation depicts data more comprehensively than each individual time-point, therefore mining effective longitudinal phenotype latent pattern representation. Specifically, we first propose two latent pattern representation (LPR) for longitudinal imaging phenotypes: linear LPR (lLPR), based on linear relationships between latent representation and each time-point, and nonlinear LPR (nonlLPR), based on neural networks to deal with nonlinear relationships. Then, we calculate the imaging genetic association based on the latent pattern representation. Finally, we conduct the experiments on both synthetic and real longitudinal imaging genetic data. Related experimental results validate that our proposed approach outperforms several competing algorithms, establishes strong associations, and discovers consistent longitudinal imaging genetic biomarkers, thereby guiding disease interpretation.

In the brain imaging genomic tasks, it is challenging to provide accurate prior knowledge for estimating the association between quantitative traits (QTs) extracted from neuroimaging and genetic markers like single-nucleotide polymorphisms (SNPs). The hidden structural patterns in data limit the discovery of disease-related biomarkers. To this end, we present a deep ring-wise block network (RB-Net) for association analysis and brain disease diagnosis. Specifically, we first construct a new hidden structural pattern, namely, ring-wise block pattern, that satisfies both block and ring properties within the data before the association analysis. Subsequently, a RB-Net is developed via using an auto-encoder (AE) to represent imaging genomic data. Furthermore, we design the approach for joint association learning and automated brain disease diagnosis. Additionally, the optimization scheme based on alternating update is presented for solve the built ring-wise block-perception layer model. The performance of the designed method has been experimentally assessed on the brain imaging genomic data from the Alzheimer's Disease Neuroimaging Initiative (ADNI). The results validate that the proposed approach outperforms some competing approaches, establishes strong associations, and identifies crucial regions of interest (ROIs) across different imaging phenotypes associated with genetic risk biomarkers, thereby guiding disease interpretation and diagnosis prediction.

Deep learning has achieved outstanding success in the hyperspectral image (HSI) classification task. Almost all the current deep learning methods are used to conduct classification predictions by leveraging the output features from the deepest layer, which generally ignore the attention to multilayer outputs, so that the capability of hierarchical representation is limited. To remedy such deficiency, in this article, we propose to build a novel deep network form, called tensored deep autoencoder network (TDAE), for HSI classification. For this method, the tensor decomposition constraint item is built and introduced into a deep autoencoders network with a fully connection layer. It not only achieves the integration of multilayer output features but also captures the structure information among outputs. By such way, the network’s ability for hierarchical representation is significantly enhanced. Furthermore, to solve such built model, we further design an alternating update optimization scheme and obtain the desired feature forms. The features are further input into the fully connection layer to generate the label of the given HSI. Extensive experiments have been conducted to validate that the proposed TDAE method achieves more competitive performance compared with several state-of-the-art approaches.

Brain imaging genetics is a widely focused topic, which has achieved the great successes in the diagnosis of complex brain disorders. In clinical practice, most existing data fusion approaches extract features from homogeneous data, neglecting the heterogeneous structural information among imaging genetic data. In addition, the number of labeled samples is limited due to the cost and time of manually labeling data. To remedy such deficiencies, in this work, we present a multimodal fusion-based hypergraph transductive learning (MFHT) for clinical diagnosis. Specifically, for each modality, we first construct a corresponding similarity graph to reflect the similarity between subjects using the label prior. Then, the multiple graph fusion approach based on theoretical convergence guarantee is designed for learning a unified graph harnessing the structure of entire data. Finally, to fully exploit the rich information of the obtained graph, a hypergraph transductive learning approach is designed to effectively capture the complex structures and high-order relationships in both labeled and unlabeled data to achieve the diagnosis results. The brain imaging genetic data of the Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets are used to experimentally explore our developed method. Related results show that our method is well applied to the analysis of brain imaging genetic data, which accounts for genetics, brain imaging (region of interest (ROI) node features), and brain imaging (connectivity edge features) to boost the understanding of disease mechanism as well as improve clinical diagnosis.

Deep learning has achieved many successes in the field of the hyperspectral image (HSI) classification. Most of existing deep learning-based methods have no consideration of feature distribution, which may yield lowly separable and discriminative features. From the perspective of spatial geometry, one excellent feature distribution form requires to satisfy both properties, i.e., block and ring. The block means that in a feature space, the distance of intraclass samples is close and the one of interclass samples is far. The ring represents that all class samples are overall distributed in a ring topology. Accordingly, in this article, we propose a novel deep ring-block-wise network (DRN) for the HSI classification, which takes full consideration of feature distribution. To obtain the good distribution used for high classification performance, in this DRN, a ring-block perception (RBP) layer is built by integrating the self-representation and ring loss into a perception model. By such way, the exported features are imposed to follow the requirements of both block and ring, so as to be more separably and discriminatively distributed compared with traditional deep networks. Besides, we also design an optimization strategy with alternating update to obtain the solution of this RBP layer model. Extensive results on the Salinas, Pavia Centre, Indian Pines, and Houston datasets have demonstrated that the proposed DRN method achieves the better classification performance in contrast to the state-of-the-art approaches.

This article presents a novel deep network with irregular convolutional kernels and self-expressive property (DIKS) for the classification of hyperspectral images (HSIs). Specifically, we use the principal component analysis (PCA) and superpixel segmentation to obtain a series of irregular patches, which are regarded as convolutional kernels of our network. With such kernels, the feature maps of HSIs can be adaptively computed to well describe the characteristics of each object class. After multiple convolutional layers, features exported by all convolution operations are combined into a stacked form with both shallow and deep features. These stacked features are then clustered by introducing the self-expression theory to produce final features. Unlike most traditional deep learning approaches, the DIKS method has the advantage of self-adaptability to the given HSI due to building irregular kernels. In addition, this proposed method does not require any training operations for feature extraction. Because of using both shallow and deep features, the DIKS has the advantage of being multiscale. Due to introducing self-expression, the DIKS method can export more discriminative features for HSI classification. Extensive experimental results are provided to validate that our method achieves better classification performance compared with state-of-the-art algorithms.

MOTIVATION: As a rising research topic, brain imaging genetics aims to investigate the potential genetic architecture of both brain structure and function. It should be noted that in the brain, not all variations are deservedly caused by genetic effect, and it is generally unknown which imaging phenotypes are promising for genetic analysis. RESULTS: In this work, genetic variants (i.e. the single nucleotide polymorphism, SNP) can be correlated with brain networks (i.e. quantitative trait, QT), so that the connectome (including the brain regions and connectivity features) of functional brain networks from the functional magnetic resonance imaging data is identified. Specifically, a connection matrix is firstly constructed, whose upper triangle elements are selected to be connectivity features. Then, the PageRank algorithm is exploited for estimating the importance of different brain regions as the brain region features. Finally, a deep self-reconstruction sparse canonical correlation analysis (DS-SCCA) method is developed for the identification of genetic associations with functional connectivity phenotypic markers. This approach is a regularized, deep extension, scalable multi-SNP-multi-QT method, which is well-suited for applying imaging genetic association analysis to the Alzheimer's Disease Neuroimaging Initiative datasets. It is further optimized by adopting a parametric approach, augmented Lagrange and stochastic gradient descent. Extensive experiments are provided to validate that the DS-SCCA approach realizes strong associations and discovers functional connectivity and brain region phenotypic biomarkers to guide disease interpretation. AVAILABILITY AND IMPLEMENTATION: The Matlab code is available at https://github.com/meimeiling/DS-SCCA/tree/main. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Sparse representation (SR)-based methods have achieved numerous successes in the fusion of hyperspectral and multispectral images (HSIs and MSIs). However, in many SR-based fusion methods, due to patch dividing, it is hard to make pixel values across boundaries of contiguous patches be exactly consistent, which limits the ability to preserve scene details. To remedy such deficiency, a fusion framework is proposed for HSIs and MSIs by using convolutional sparse representation (FCS). This novel fusion method consists of three stages: 1) the spectral dictionary is trained by the convolutional sparse dictionary learning algorithm to extract spectral information from HSIs; 2) hyperspectral and multispectral transferring matrices are estimated to map HSIs and MSIs onto the space of high-resolution hyperspectral images (HR-HSIs); and 3) we construct the convolutional sparse fusion model for HR-HSIs. Different from those traditional patch-based SR fusion methods, the FCS method focuses on the whole images instead of dividing patches, which can suppress the limitation of scene detail preservation caused by sparse coding on independent patches. Also, it belongs to a kind of online learning without lots of training samples. The Pavia dataset and the Paris dataset are used to evaluate the performance of our method. Experimental results indicate that the FCS method achieves much fusion performance compared with commonly used and state-of-the-art algorithms.

Hyperspectral image (HSI) classification is a current research hotspot. Most existing methods usually export discriminative features with low-quality distribution and low information utilization, which may induce classification performance degeneration. To remedy such deficiencies, we propose a diagonalized low-rank learning (DLRL) model for HSI classification in this study. Specifically, a classwise regularization is used to capture the classwise block-diagonal structure of low-rank representation, which can further cluster the represented HSI pixels from one class into the same subspace and extract features with well-ordered distribution. Such a regularization assists to easily and correctly classify HSIs. In addition, we combine sparsity and collaboration to extract more discriminative features for guaranteeing high information utilization, i.e., a tradeoff of sparsity and collaboration is sought to acquire both correlations among HSI pixels and characteristics of each pixel. By this way, rich information in the HSI can be fully used for good feature extraction. Further, the estimated feature representation is used as an input to the support vector machine (SVM) classifier for HSI classification. Extensive experiments have been done to validate that the proposed DLRL method achieves much classification performance in contrast to several state-of-the-art algorithms.

Deep learning is a popular and effective technique for the hyperspectral image (HSI) classification. Current deep learning-based methods have numerous free parameters to be trained. They may be unavailable once lacking training samples. In addition, these approaches only use the features from the deepest layer and exclude shallow features, which is a kind of information loss. To remedy such deficiencies, in this work, we construct a novel deep encoder with kernel-wise Taylor series (EKTS) for the HSI classification. More specifically, we introduce the Taylor series to approximate the role of deep networks for feature extraction. Because the original Taylor series is linear, the kernel theory is used to build the kernel-wise Taylor series to encode the HSI data and extract deep nonlinear features. Furthermore, an alternating iterative optimization strategy is developed to obtain the outputs of all layers of the built deep encoder. Subsequently, we stack the outputs of all layers to obtain the final features that integrate both shallow and deep features. At last, the support vector machine (SVM) is adopted to deal with the obtained final features so that label results can be predicted. Unlike current deep learning methods, our EKTS has no free parameters to be trained and combines the advantages of both shallow and deep features to predict labels. Sufficient experimental analysis has been performed to verify the greater classification performance of our EKTS method compared with many state-of-the-art approaches.

For most hyperspectral image (HSI) classification methods, each feature code is usually individually learned, and which is susceptible to noise. To remedy such deficiency, we propose a binary feature representation method with context -aware attention (BFCA) for HSI classification in this study. In this model, local spectrum modules (LSMs) are first built by segmenting each HSI pixel vector into several parts and computing the differences between the central value and its neighborhoods in each part. The LSMs can observe the changes of spectral values, and can fully find and use spectral information for HSI classification. Second, projections with hash functions are learned, which aim to map and quantize each LSM into a binary vector. Each binary vector is limited with one shift between 0 and 1 to guarantee spectral context-awareness. Once projection matrix obtained by optimizing the proposed model, binary vectors all samples are calculated, which are further classified by SVM. Unlike current approaches, our BFCA exploits spectral contextual information of LSMs to improve the stability and robustness of feature representation and HSI classification. Extensive experiments have been given to validate the superiority of our BFCA.

Hyperspectral image (HSI) classification is a widely focused topic. Existing methods export discriminative features with low-quality distribution, which may induces classification performance degeneration. To remedy this deficiency, we propose a group-aware low-rank representation (GAL-RR) model to classify HSIs in this paper. Specifically, a group-wise regularization is introduced to capture the block-diagonal group structure of low-rank representation. With such regularization, the HSI pixels from one class are well clustered into the same subspace. In other words, the HSI feature representation is guaranteed with high-quality distribution and good discriminative information. In order to solve this built non-convex model, we design a proximal alternating optimization scheme. Furthermore, the SVM classifier is used to classify these low-rank representation forms. Extensive experiments are provided in this paper to verify the effectiveness and superiority of the proposed GALRR classification algorithm.

Sparse canonical correlation analysis (SCCA) is a bi-multivariate technique used in imaging genetics to identify complex multi-SNP-multi-QT associations. However, the traditional SCCA algorithm has been designed to seek a linear correlation between the SNP genotype and brain imaging phenotype, ignoring the discriminant similarity information between within-class subjects in brain imaging genetics association analysis. In addition, multi-modality brain imaging phenotypes are extracted from different perspectives and imaging markers from the same region consistently showing up in multimodalities may provide more insights for the mechanistic understanding of diseases. In this paper, a novel multi-modality discriminant SCCA algorithm (MD-SCCA) is proposed to overcome these limitations as well as to improve learning results by incorporating valuable discriminant similarity information into the SCCA algorithm. Specifically, we first extract the discriminant similarity information between within-class subjects by the sparse representation. Second, the discriminant similarity information is enforced within SCCA to construct a discriminant SCCA algorithm (D-SCCA). At last, the MD-SCCA algorithm is adopted to fully explore the relationships among different modalities of different subjects. In experiments, both synthetic dataset and real data from the Alzheimer's Disease Neuroimaging Initiative database are used to test the performance of our algorithm. The empirical results have demonstrated that the proposed algorithm not only produces improved cross-validation performances but also identifies consistent cross-modality imaging genetic biomarkers.

In the brain imaging genetic studies, it is a challenging task to estimate the association between quantitative traits (QTs) extracted from neuroimaging data and genetic markers such as single-nucleotide polymorphisms (SNPs). Most of the existing association studies are based on the extensions of sparse canonical correlation analysis (SCCA) for the identification of complex bi-multivariate associations, which can take the specific structure and group information into consideration. However, they often take the original data as input without considering its underlying complex multi-subspace structure, which will deteriorate the performance of the following integrative analysis. Accordingly, in this paper, the self-expressive property is exploited for the reconstruction of the original data before the association analysis, which can well describe the similarity structure. Specifically, we first apply the within-class similarity information to construct self-expressive networks by sparse representation. Then, we use the fusion method to iteratively fuse the self-expressive networks from multi-modality brain phenotypes into one network. Finally, we calculate the imaging genetic association based on the fused self-expressive network. We conduct the experiments on both single-modality and multi-modality phenotype data. Related experimental results validate that our method can not only better estimate the potential association between genetic markers and quantitative traits but also identify consistent multi-modality imaging genetic biomarkers to guide the interpretation of Alzheimer's disease.

MOTIVATION: Neuroimaging genetics is an emerging field to identify the associations between genetic variants [e.g. single-nucleotide polymorphisms (SNPs)] and quantitative traits (QTs) such as brain imaging phenotypes. However, most of the current studies focus only on the associations between brain structure imaging and genetic variants, while neglecting the connectivity information between brain regions. In addition, the brain itself is a complex network, and the higher-order interaction may contain useful information for the mechanistic understanding of diseases [i.e. Alzheimer's disease (AD)]. RESULTS: A general framework is proposed to exploit network voxel information and network connectivity information as intermediate traits that bridge genetic risk factors and disease status. Specifically, we first use the sparse representation (SR) model to build hyper-network to express the connectivity features of the brain. The network voxel node features and network connectivity edge features are extracted from the structural magnetic resonance imaging (sMRI) and resting-state functional magnetic resonance imaging (fMRI), respectively. Second, a diagnosis-aligned multi-modality regression method is adopted to fully explore the relationships among modalities of different subjects, which can help further mine the relation between the risk genetics and brain network features. In experiments, all methods are tested on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. The experimental results not only verify the effectiveness of our proposed framework but also discover some brain regions and connectivity features that are highly related to diseases. AVAILABILITY AND IMPLEMENTATION: The Matlab code is available at http://ibrain.nuaa.edu.cn/2018/list.htm.