Multi-modal neuroimaging data, including magnetic resonance imaging (MRI) and fluorodeoxyglucose positron emission tomography (PET), have greatly advanced the computer-aided diagnosis of Alzheimer's disease (AD) by providing shared and complementary information. However, the problem of incomplete multi-modal data remains inevitable and challenging. Conventional strategies that exclude subjects with missing data or synthesize missing scans either result in substantial sample reduction or introduce unwanted noise. To address this issue, we propose an Incomplete Multi-modal Disentanglement Learning method (IMDL) for AD diagnosis without missing scan synthesis, a novel model that employs a tiny Transformer to fuse incomplete multi-modal features extracted by modality-wise variational autoencoders adaptively. Specifically, we first design a cross-modality contrastive learning module to encourage modality-wise variational autoencoders to disentangle shared and complementary representations of each modality. Then, to alleviate the potential information gap between the representations obtained from complete and incomplete multi-modal neuroimages, we leverage the technique of adversarial learning to harmonize these representations with two discriminators. Furthermore, we develop a local attention rectification module comprising local attention alignment and multi-instance attention rectification to enhance the localization of atrophic areas associated with AD. This module aligns inter-modality and intra-modality attention within the Transformer, thus making attention weights more explainable. Extensive experiments conducted on ADNI and AIBL datasets demonstrated the superior performance of the proposed IMDL in AD diagnosis, and a further validation on the HABS-HD dataset highlighted its effectiveness for dementia diagnosis using different multi-modal neuroimaging data (i.e., T1-weighted MRI and diffusion tensor imaging).

Graph Attention Network (GAT) has a wide range of applications in HSI classification. The GAT-based semi-supervised learning approach enables the integration of valuable information from both labeled and unlabeled samples, effectively reducing the model’s reliance on labeled data. However, the node-wise training approach of GAT often overlooks the inherent global feature of graph data and the long-range dependencies among nodes, thereby limiting the model’s generalization ability on unlabeled data. Therefore, we propose a semi-supervised HSI classification model based on the dynamic evolution graph attention network (DEGAT). The main contributions: 1)We design a dynamic graph evolution mechanism (DGEM) that enables the model to capture the interactive information between local graph attention coefficients and the global graph structure, thus obtaining more discriminative graph representations. 2)DEGAT utilizes the multi-scale mechanism and message-passing mechanism to capture the information of nodes with long-range dependencies, extracting richer spatial-spectral features. State-of-the-art results are achieved with very few labeled training samples on two typical benchmark HSI datasets, where the overall accuracy reaches 95.12% and 98.76% respectively.

Cortical surface registration plays a crucial role in aligning cortical functional and anatomical features across individuals. However, conventional registration algorithms are computationally inefficient. Recently, learning-based registration algorithms have emerged as a promising solution, significantly improving processing efficiency. Nonetheless, there remains a gap in the development of a learning-based method that exceeds the state-of-the-art conventional methods simultaneously in computational efficiency, registration accuracy, and distortion control, despite the theoretically greater representational capabilities of deep learning approaches. To address the challenge, we present SUGAR, a unified unsupervised deep-learning framework for both rigid and non-rigid registration. SUGAR incorporates a U-Net-based spherical graph attention network and leverages the Euler angle representation for deformation. In addition to the similarity loss, we introduce fold and multiple distortion losses to preserve topology and minimize various types of distortions. Furthermore, we propose a data augmentation strategy specifically tailored for spherical surface registration to enhance the registration performance. Through extensive evaluation involving over 10,000 scans from 7 diverse datasets, we showed that our framework exhibits comparable or superior registration performance in accuracy, distortion, and test-retest reliability compared to conventional and learning-based methods. Additionally, SUGAR achieves remarkable sub-second processing times, offering a notable speed-up of approximately 12,000 times in registering 9,000 subjects from the UK Biobank dataset in just 32 min. This combination of high registration performance and accelerated processing time may greatly benefit large-scale neuroimaging studies.

Convolutional neural networks (CNNs) have attained remarkable performance in hyperspectral image (HSI) classification. However, the existing CNNs are restricted by their limited receptive field in HSI classification. Recently, transformer networks have proved to be promising in many tasks thanks to the global receptive field, but they easily ignore some local information that is important for HSI classification. In this letter, we propose a novel method entitled convolutional transformer network (CTN) for HSI classification. In order to make full use of spectral information and spatial information, the method adopts center position encoding (CPE) to merge spectral features and pixel positions. Furthermore, the proposed method introduces convolutional transformer (CT) blocks. It effectively combines convolution and transformer structures together to capture local–global features of HSI patches, which is contributive for HSI classification. Experimental results on public datasets demonstrate the superiority of our method compared with several state-of-the-art classification methods. The codes of this work will be available athttps://github.com/sky8791to facilitate reproducibility.

Infancy is a critical period for the human brain development, and brain age is one of the indices for the brain development status associated with neuroimaging data. The difference between the predicted age based on neuroimaging and the chronological age can provide an important early indicator of deviation from the normal developmental trajectory. In this study, we utilize the Graph Convolutional Network (GCN) to predict the infant brain age based on resting-state fMRI data. The brain connectivity obtained from rs-fMRI can be represented as a graph with brain regions as nodes and functional connections as edges. However, since the brain connectivity is a fully connected graph with features on edges, current GCN cannot be directly used for it is a node-based method for sparse graphs. Hence, we propose an edge-based Graph Path Convolution (GPC) method, which aggregates the information from different paths and can be naturally applied on dense graphs. We refer the whole model as Brain Connectivity Graph Convolutional Networks (BC-GCN). Further, two upgraded network structures are proposed by including the residual and attention modules, referred as BC-GCN-Res and BC-GCN-SE to emphasize the information of the original data and enhance influential channels. Moreover, we design a two-stage coarse-to-fine framework, which determines the age group first and then predicts the age using group-specific BC-GCN-SE models. To avoid accumulated errors from the first stage, a cross-group training strategy is adopted for the second stage regression models. We conduct experiments on infant fMRI scans from 6 to 811 days of age. The coarse-to-fine framework shows significant improvements when being applied to several models (reducing error over 10 days). Comparing with state-of-the-art methods, our proposed model BC-GCN-SE with coarse-to-fine framework reduces the mean absolute error of the prediction from >70 days to 49.9 days. The code is now available at https://github.com/SCUT-Xinlab/BC-GCN.

Prediction of the chronological age based on neuroimaging data is important for brain development analysis and brain disease diagnosis. Although many researches have been conducted for age prediction of older children and adults, little work has been dedicated to infants. To this end, this paper focuses on predicting infant age from birth to 2-year old using brain MR images, as well as identifying some related biomarkers. However, brain development during infancy is too rapid and heterogeneous to be accurately modeled by the conventional regression models. To address this issue, a two-stage prediction method is proposed. Specifically, our method first roughly predicts the age range of an infant and then finely predicts the accurate chronological age based on a learned, age-group-specific regression model. Combining this two-stage prediction method with another complementary one-stage prediction method, a hierarchical rough-to-fine (HRtoF) model is built. HRtoF effectively splits the rapid and heterogeneous changes during a long time period into several short time ranges and further mines the discrimination capability of cortical features, thus reaching high accuracy in infant age prediction. Taking 8 types of cortical morphometric features from structural MRI as predictors, the effectiveness of our proposed HRtoF model is validated using an infant dataset including 50 healthy subjects with 251 longitudinal MRI scans from 14 to 797 days. Comparing with five state-of-the-art regression methods, HRtoF model reduces the mean absolute error of the prediction from >48 days to 32.1 days. The correlation coefficient of the predicted age and the chronological age reaches 0.963. Moreover, based on HRtoF, the relative contributions of the eight types of cortical features for age prediction are also studied.

Effective fusion of structural magnetic resonance imaging (sMRI) and functional magnetic resonance imaging (fMRI) data has the potential to boost the accuracy of infant age prediction thanks to the complementary information provided by different imaging modalities. However, functional connectivity measured by fMRI during infancy is largely immature and noisy compared to the morphological features from sMRI, thus making the sMRI and fMRI fusion for infant brain analysis extremely challenging. With the conventional multimodal fusion strategies, adding fMRI data for age prediction has a high risk of introducing more noises than useful features, which would lead to reduced accuracy than that merely using sMRI data. To address this issue, we develop a novel model termed as disentangled-multimodal adversarial autoencoder (DMM-AAE) for infant age prediction based on multimodal brain MRI. Specifically, we disentangle the latent variables of autoencoder into common and specific codes to represent the shared and complementary information among modalities, respectively. Then, cross-reconstruction requirement and common-specific distance ratio loss are designed as regularizations to ensure the effectiveness and thoroughness of the disentanglement. By arranging relatively independent autoencoders to separate the modalities and employing disentanglement under cross-reconstruction requirement to integrate them, our DMM-AAE method effectively restrains the possible interference cross modalities, while realizing effective information fusion. Taking advantage of the latent variable disentanglement, a new strategy is further proposed and embedded into DMM-AAE to address the issue of incompleteness of the multimodal neuroimages, which can also be used as an independent algorithm for missing modality imputation. By taking six types of cortical morphometric features from sMRI and brain functional connectivity from fMRI as predictors, the superiority of the proposed DMM-AAE is validated on infant age (35 to 848 days after birth) prediction using incomplete multimodal neuroimages. The mean absolute error of the prediction based on DMM-AAE reaches 37.6 days, outperforming state-of-the-art methods. Generally, our proposed DMM-AAE can serve as a promising model for prediction with multimodal data.

Hyperspectral data contains abundant information in spectral domain, which is very useful for mineral classification and geological body mapping. But, due to the lack of labeled data, it is difficult to get an acceptable result by just using the small number of labeled data. We adopt a semi-supervised method called CNN, which can effectively extract inner features of hyperspectral image to classify hyperspectral data. However, with constraint to the size of receptive field, it can hardly get higher level features. We propose dilated CNN for mineral classification of hyperspectral data. At the same size of kernels, dilated CNN has bigger receptive field. At the meanwhile, it can get higher accuracy of classification. We test our model on hyperspectral data at Tianshan area, where is rich in minerals. From the result, we can find that our method can get a great result on the mineral classification task, which can be used for making geological map.

Classifying Hyperspectral images with few training samples is a challenging problem. The generative adversarial networks (GAN) are promising techniques to address the problems. GAN constructs an adversarial game between a discriminator and a generator. The generator generates samples that are not distinguishable by the discriminator, and the discriminator determines whether or not a sample is composed of real data. In this paper, by introducing multilayer features fusion in GAN and a dynamic neighborhood voting mechanism, a novel algorithm for HSIs classification based on 1-D GAN was proposed. Extracting and fusing multiple layers features in discriminator, and using a little labeled samples, we fine-tuned a new sample 1-D CNN spectral classifier for HSIs. In order to improve the accuracy of the classification, we proposed a dynamic neighborhood voting mechanism to classify the HSIs with spatial features. The obtained results show that the proposed models provide competitive results compared to the state-of-the-art methods.

Because the collection of ground-truth labels is difficult, expensive, and time-consuming, classifying hyperspectral images (HSIs) with few training samples is a challenging problem. In this letter, we propose a novel semisupervised algorithm for the classification of hyperspectral data by training a customized generative adversarial network (GAN) for hyperspectral data. The GAN constructs an adversarial game between a discriminator and a generator. The generator generates samples that are not distinguishable by the discriminator, and the discriminator determines whether or not a sample is composed of real data. We design a semisupervised framework for HSI data based on a 1-D GAN (HSGAN). This framework enables the automatic extraction of spectral features for HSI classification. When HSGAN is trained using unlabeled hyperspectral data, the generator can generate hyperspectral samples that are similar to the real data, while the discriminator contains the features, which can be used to classify hyperspectral data with only a small number of labeled samples. The performance of the HSGAN is evaluated on the Airborne Visible Infrared Imaging Spectrometer image data, and the results show that the proposed framework achieves very promising results with a small number of labeled samples.

In this letter, a band-selection approach based on the deep convolutional neural network (CNN) and distance density (DD) is proposed. This method effectively mitigates the curse of dimensionality for hyperspectral images (HSIs). First, we use the hyperspectral full-band data to train a custom 1-D CNN to obtain a well-trained model. Second, we select band combinations based on DD. Using the rectified linear unit, which is the activation function of the CNN that is only activated with a nonzero value, we can effectively test the band combinations without retraining the model. Finally, the method selects the band combinations with the highest precision as the final selected bands. This precision measure is a new criterion for band selection. To further improve the performance, a data augmentation method based on DD is also proposed. To justify the effectiveness of the proposed method, experiments are conducted on two HSIs. The results show that the proposed method can acquire more satisfactory results than traditional methods.

Dependence degree (DD) and importance degree (ID) of patterns are crucial for pattern appraisement and model reconstruction. In rough set data analysis (RSDA), DD and ID lack robustness because the lower approximation set is terribly unstable under the indiscernibility relation perturbation. Statistical inference is a good way to deal with this instability. However, the fixed-value hypothesis testing and interval estimation of DD and ID were only discussed by the $\chi ^2$ test, which merely suits for two contingency tables (CT) with the same number of elements, and their nonzero elements must exist at the same positions. These requirements are too strict for practical applications. In this paper, the Kolmogorov-Smirnov (K-S) goodness-of-fit test is introduced to generalize the statistical inferences of DD and ID. As the bridge between data and corresponding measures, CT lies at the core of RSDA. By transforming CT to a random sample of a hypothesized random variable, the elementary algorithm for the goodness-of-fit test of contingency tables and the advanced algorithm for the goodness-of-fit test of contingency tables are proposed based on K-S statistic to implement goodness-of-fit tests of CTs. Better than $\chi ^2$ test, all CTs, even with different numbers and different positions of nonzero elements, are permitted. Subsequently, by generating a CT with expected DD value, the fixed-value hypothesis of DD is transformed to the goodness-of fit test between the original and expected CTs. Three algorithms, i.e., hypothesis test of dependence degree based on the K-S test, region estimation of the dependence degree, and significance test and region estimation of importance degree of attribute set, are proposed as fixed-value hypothesis tests and region estimations of DD and ID. These algorithms can be used to verify the importance of attributes and choose the attribute subset with the highest likelihood of maintaining the original discrimination ability. Experiments verify that the discrimination ability, disturbance tolerance ability, and stability under varying discretization strategies of DD and ID are significantly enhanced by the proposed algorithms.

The classic methods, such as FCM, often fail to carry out accurate modeling for the high-level fuzzy uncertainty, and then cause the classification error that should not be ignored in the application. Fortunately, the type-2 fuzzy set is a tool to handle this type of uncertainty. An adaptive interval-valued type-2 fuzzy C-Means clustering algorithm (A-IT2FCM) is proposed, including:(1) a proper modeling method for interval-valued type-2 fuzzy set;(2) an effective type reduction approach by adaptively searching the equivalent type-1 fuzzy sets for the type-2. Three different type-2 fuzzy clustering algorithms are used: the algorithm based on Karnik-Mendel type reduction, a method based on simple type reduction, and A-IT2FCM presented in this article. The experimental data are two data windows of SPOT5 image from Zhuhai and Beijing, China. Results show that, A-IT2FCM outperforms the other algorithms compared. Especially when obvious density difference exists between objects in the data, A-IT2FCM can achieve more accurate class boundaries and higher classification accuracy.

Statistical inference about dependence degree (DD) and importance degree (ID) of variables in an information system is crucial for variables appraisement and model reconstruction. However, in rough set data analysis (RSDA), the literature is restricted to validate independence or test whether the degree is significantly big, while the fixed value test and interval estimation for related measurements have been ignored. Because these important issues have not been addressed, we cannot determine whether the data support expert opinions and compare the features in depth. To enhance the integrity of statistical inference for DD and ID in an RSDA, fixed value tests and interval estimations of DD and ID are presented in this paper. With multinomial distribution as the carrier for statistical information in the databases, the fixed value test of DD is successfully transformed into a restricted estimation of multinomial distribution and a goodness-of-fit test for distributions. The fixed value test and interval estimation algorithms for DD and ID are then presented in detail and illustrated with examples. Explicit expressions for the DD and ID interval estimation, DD confidence curves, and the limit theory for DD and ID are shown. Furthermore, the effectiveness and discrimination of the proposed algorithms are validated using the Car evaluation, Tic-Tac-Toe endgame, and Fisher's Iris databases.

Sensitivity analysis is used to estimate the effective variable (set) in a system. It plays an important role in model simplification, quality assurance of models and codes, identification of crucial regions in the parameter space and so on. Many interesting results have been obtained in the sensitivity analysis of traditional feed forward neural network. But the related work hasn't been done in fuzzy neural network (FNN). In this paper, we choose fuzzy inference neural network (FINN), which has the advantages of representing the uncertain information and higher approximation capability, to study the sensitivity analysis of FNN. We firstly propose spFINN which is the simplification of a FINN introduced by Takatoshi Nishina and present a corresponding training algorithm. The simplification can make the sensitivity analysis process easier without reducing the approximation capability. Then a procedure called FINNSI is proposed to evaluate the sensitivity of input variable of spFINN. As the kernel idea of FINNSI, the way of separation and integration of the sensitivity of neurons is introduced because the sensitivity of neurons is transferred from the last layer (output layer) to the first layer (input layer). The procedure in FINNSI can be easily generalized to the sensitivity analysis of other fuzzy systems or networks. Finally, we discuss the application of FINNSI in band selection. The sensitive bands can be found to help the subsequent steps of spectral data processing such as object recognition and mineral classification.