The use of imaging and genetic data for biomarker detection and disease diagnosis can deepen the understanding of disease pathogenesis and assist in clinical diagnosis. However, current methods face two major challenges: 1) the significant heterogeneity between multimodal data hampers modality fusion and 2) effectively exploring consistency and variability information from similar diseases for enhancing model performance is difficult. In this paper, we propose a novel unified framework, termed dual adaptive disentangled representation learning (DADRL), to simultaneously achieve disease-shared and disease-specific biomarker detection as well as disease diagnosis. Our DADRL comprises three components: 1) a biology information constraints-based modality fusion strategy is applied to adaptively explore inter- and intra-modal correlations, thereby effectively fusing multimodal data; 2) a unified framework that integrates modality fusion and disease diagnosis is proposed to mine disease-related information for simultaneously accomplishing disease-related biomarker detection and disease diagnosis; and 3) disentangled representation learning and several adaptive metric constraints are incorporated into the unified framework to adaptively separate disease-specific information from disease-shared feature representations for effectively identifying disease-shared and disease-specific biomarkers, thereby deepening the understanding of disease pathogenesis. Extensive experiments on multiple real datasets and simulated data demonstrate that our method significantly improves performance of biomarker detection and disease diagnosis.

Noninvasively predicting the status of isocitrate dehydrogenase (IDH) and chromosome arms 1p/19q preoperatively on multisequence magnetic resonance imaging (MRI) images is helpful for prognosis and optimal therapy planning of patients with gliomas. However, effectively learning discriminative features from MRI images for predicting IDH mutation and 1p/19q codeletion status remains challenging due to the high heterogeneity of gliomas. A dual structural feature exploration and alignment network (DSFEAnet) was proposed to effectively explore representative features associated with the intratumoral and marginal heterogeneity of gliomas for accurate prediction. First, a match and mismatch feature extraction (MMFE) module was introduced to extract image structural features related to intratumoral heterogeneity, such as information associated with tumor core localization and T2-fluid-attenuated inversion recovery (FLAIR) mismatch sign. Second, a graph-based geometry exploration (GGE) module was developed to explore graph structural features related to marginal heterogeneity. In this module, the vertex associations and variations perpendicularly along a 3-D tumor surface were integrated as a graph, which can effectively perceive changes in locations, sizes, and marginal textures of gliomas, thus enhancing the feature representational ability to describe glioma heterogeneity. Finally, a dual structural feature alignment (DSFA) module was incorporated to narrow the gaps among intra- and interstructural features. It can adaptively align and fuse different features and thus further improve the overall prediction performance. The proposed DSFEAnet was evaluated using a multicenter dataset, and its robustness was demonstrated on an independent clinical dataset. Specifically, preoperative MRI images of 560 glioma samples were collected from publicly available The Cancer Imaging Archive (TCIA) ( $n =203$ , age: 51.82 (15.21) years, male/female: 109/94, IDH-mutant/wild-type: 90/113, 1p/19q-codeleted/noncodeleted: 27/176), Nanfang hospital ( $n =136$ , age: 41.96 (12.29) years, male/female: 80/56, IDH-mutant/wild-type: 53/83, 1p/19q-codeleted/noncodeleted: 31/105), and Zhujiang hospital ( $n =221$ , age: 43.82 (17.36) years, male/female: 134/87, IDH-mutant/wild-type: 94/127, 1p/19q-codeleted/noncodeleted: 34/187). Our DSFEAnet achieved an AUC of 87.72% for IDH mutation status prediction and an AUC of 80.52% for 1p/19q codeletion status prediction in the Nanfang hospital dataset. Finally, the interpretability of the proposed modules was assessed to highlight the effectiveness of our method. Overall, the DSFEAnet exhibits great potential for predicting IDH mutation and 1p/19q codeletion status.

Ocean surface wind speed is important for numerical prediction of the marine environment, marine disaster monitoring, sea–steam interaction, meteorological prediction, climate research, and so on. At present, ocean surface wind speed retrieval models extract DDM features from the delay-Doppler domain, the angle of which is single, where certain detail information cannot be effectively extracted from the image. To improve the accuracy of wind speed retrieval, starting from the feature extraction, this letter proposes a frequency-informed neural network (FINN)-based wind speed retrieval. First, the wind speed retrieval model based on the delay-Doppler domain and frequency domain is constructed, based on the extraction of features from the DDM delay-Doppler domain, the features from the DDM frequency domain are also extracted, so the wind speed retrieval is performed separately by extracting features from different angles. Then, a multimodel fusion scheme based on dynamic weights is designed, which can dynamically weight submodels according to different input samples, and realize the dynamic fusion of delay-Doppler-domain retrieval results and frequency-domain retrieval results. Finally, the improved gradient loss (IG Loss) function is proposed. Contrast experiments and ablation studies prove that the present algorithm has excellent retrieval performance.

Isocitrate dehydrogenase (IDH) is one of the most important genotypes in patients with glioma because it can affect treatment planning. Machine learning-based methods have been widely used for prediction of IDH status (denoted as IDH prediction). However, learning discriminative features for IDH prediction remains challenging because gliomas are highly heterogeneous in MRI. In this paper, we propose a multi-level feature exploration and fusion network (MFEFnet) to comprehensively explore discriminative IDH-related features and fuse different features at multiple levels for accurate IDH prediction in MRI. First, a segmentation-guided module is established by incorporating a segmentation task and is used to guide the network in exploiting features that are highly related to tumors. Second, an asymmetry magnification module is used to detect T2-FLAIR mismatch sign from image and feature levels. The T2-FLAIR mismatch-related features can be magnified from different levels to increase the power of feature representations. Finally, a dual-attention feature fusion module is introduced to fuse and exploit the relationships of different features from intra- and inter-slice feature fusion levels. The proposed MFEFnet is evaluated on a multi-center dataset and shows promising performance in an independent clinical dataset. The interpretability of the different modules is also evaluated to illustrate the effectiveness and credibility of the method. Overall, MFEFnet shows great potential for IDH prediction.

Automatic vertebral osteophyte recognition in Digital Radiography is of great importance for the early prediction of degenerative disease but is still a challenge because of the tiny size and high inter-class similarity between normal and osteophyte vertebrae. Meanwhile, common sampling strategies applied in Convolution Neural Network could cause detailed context loss. All of these could lead to an incorrect positioning predicament. In this paper, based on important pathological priors, we define a set of potential lesions of each vertebra and propose a novel Pathological Priors Inspired Network (PPIN) to achieve accurate osteophyte recognition. PPIN comprises a backbone feature extractor integrating with a Wavelet Transform Sampling module for high-frequency detailed context extraction, a detection branch for locating all potential lesions and a classification branch for producing final osteophyte recognition. The Anatomical Map-guided Filter between two branches helps the network focus on the specific anatomical regions via the generated heatmaps of potential lesions in the detection branch to address the incorrect positioning problem. To reduce the inter-class similarity, a Bilateral Augmentation Module based on the graph relationship is proposed to imitate the clinical diagnosis process and to extract discriminative contextual information between adjacent vertebrae in the classification branch. Experiments on the two osteophytes-specific datasets collected from the public VinDr-Spine database show that the proposed PPIN achieves the best recognition performance among multitask frameworks and shows strong generalization. The results on a private dataset demonstrate the potential in clinical application. The Class Activation Maps also show the powerful localization capability of PPIN. The source codes are available in https://github.com/Phalo/PPIN.

The Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) dynamically monitors water depth in shallow waters around islands and reefs. Noise removal is a prerequisite for accurate reconstruction of seafloor topography based on ICESat-2 data products. To this end, we propose a density and distance-based method (DDBM) to extract seafloor signal photons from ICESat-2 data. The DDBM first separates the photons into three parts: above water, water surface, and water column. The water-column photons consist of seafloor signal photons and noise photons. The DDBM adopts a two-step denoising strategy to remove noise in water-column photons to obtain pure and complete signal photons. In the first step, an orientation-variable adaptive ellipse filter is developed, which can adaptively adjust the parameters according to the water depth to remove low-density noise photons. In the second step, a novel distance-based filter (DBF) is designed for stubborn high-density noise clusters. These noise clusters are far from the signal photons, and the DBF removes them by a distance threshold derived from the Otsu threshold method. We select three high-density ICESat-2 datasets to validate the DDBM. Compared with the reference data, the comprehensive evaluation indexes$F$of the DDBM in all datasets are above 0.99, and the highest is 0.998, showing superior performance.

Occluded person re-identification (re-id) aims to match occluded person images to holistic ones. Most existing works focus on matching collective-visible body parts by discarding the occluded parts. However, only preserving the collective-visible body parts causes great semantic loss for occluded images, decreasing the confidence of feature matching. On the other hand, we observe that the holistic images can provide the missing semantic information for occluded images of the same identity. Thus, compensating the occluded image with its holistic counterpart has the potential for alleviating the above limitation. In this paper, we propose a novel Reasoning and Tuning Graph Attention Network (RTGAT), which learns complete person representations of occluded images by jointly reasoning the visibility of body parts and compensating the occluded parts for the semantic loss. Specifically, we self-mine the semantic correlation between part features and the global feature to reason the visibility scores of body parts. Then we introduce the visibility scores as the graph attention, which guides Graph Convolutional Network (GCN) to fuzzily suppress the noise of occluded part features and propagate the missing semantic information from the holistic image to the occluded image. We finally learn complete person representations of occluded images for effective feature matching. Experimental results on occluded benchmarks demonstrate the superiority of our method.

Macrovascular invasion (MaVI) is a major threat to survival in hepatocellular carcinoma (HCC), which should be treated as early as possible to ensure safety and efficacy. In this aspect, MaVI prediction can be helpful. However, MaVI prediction is difficult because of the inter-class similarity and intra-class variation of HCC in computed tomography (CT) images. Moreover, existing methods fail to include clinical priori knowledge associated with HCC, leading to incomprehensive information extraction. In this paper, we proposed a prior knowledge-aware fusion network (PKAFnet) to accurately achieve MaVI prediction in CT images. First, a perception module was presented to extract features related to tumor marginal heterogeneity in the graph domain, which contributed to rotation invariance and captured intensity variations of tumor margin. Second, a tumor segmentation network was built to obtain global information of a 3D tumor image and information associated with tumor internal heterogeneity in the image domain. Finally, multi-domain features associated with the tumor margin and tumor region were combined by using a multi-domain attentional feature fusion module. Thus, by incorporating MaVI-related prior knowledge, our PKAFnet can alleviate overfitting, which can improve the discriminative ability. The proposed PKAFnet was validated on a multi-center dataset, and remarkable performance was achieved in an independent testing set. Moreover, the interpretability of perception module and segmentation network were presented in our paper, which illustrated the effectiveness and credibility of PKAFnet. Therefore, the proposed method showed great application potential for MaVI prediction.

MOTIVATION: The association analysis between genetic variants and imaging phenotypes must be carried out to understand the inherited neuropsychiatric disorders via imaging genetic studies. Given the high dimensionality in imaging and genetic data, traditional methods based on massive univariate regression entail large computational cost and disregard many-to-many correlations between phenotypes and genetic variants. Several multivariate imaging genetic methods have been proposed to alleviate the above problems. However, most of these methods are based on the l1 penalty, which might cause the over-selection of variables and thus mislead scientists in analyzing data from the field of neuroimaging genetics. RESULTS: To address these challenges in both statistics and computation, we propose a novel co-sparse reduced-rank regression model that identifies complex correlations in a dimensional reduction manner. We developed an iterative algorithm based on a group primal dual-active set formulation to detect simultaneously important genetic variants and imaging phenotypes efficiently and precisely via non-convex penalty. The simulation studies showed that our method achieved accurate and stable performance in parameter estimation and variable selection. In real application, the proposed approach successfully detected several novel Alzheimer's disease-related genetic variants and regions of interest, which indicate that our method may be a valuable statistical toolbox for imaging genetic studies. AVAILABILITY AND IMPLEMENTATION: The R package csrrr, and the code for experiments in this article is available in Github: https://github.com/hailongba/csrrr. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Imaging genetics is an effective tool used to detect potential biomarkers of Alzheimer's disease (AD) in imaging and genetic data. Most existing imaging genetics methods analyze the association between brain imaging quantitative traits (QTs) and genetic data [e.g., single nucleotide polymorphism (SNP)] by using a linear model, ignoring correlations between a set of QTs and SNP groups, and disregarding the varied associations between longitudinal imaging QTs and SNPs. To solve these problems, we propose a novel temporal group sparsity regression and additive model (T-GSRAM) to identify associations between longitudinal imaging QTs and SNPs for detection of potential AD biomarkers. We first construct a nonparametric regression model to analyze the nonlinear association between QTs and SNPs, which can accurately model the complex influence of SNPs on QTs. We then use longitudinal QTs to identify the trajectory of imaging genetic patterns over time. Moreover, the SNP information of group and individual levels are incorporated into the proposed method to boost the power of biomarker detection. Finally, we propose an efficient algorithm to solve the whole T-GSRAM model. We evaluated our method using simulation data and real data obtained from AD neuroimaging initiative. Experimental results show that our proposed method outperforms several state-of-the-art methods in terms of the receiver operating characteristic curves and area under the curve. Moreover, the detection of AD-related genes and QTs has been confirmed in previous studies, thereby further verifying the effectiveness of our approach and helping understand the genetic basis over time during disease progression.

Spine parsing (i.e., multi-class segmentation of vertebrae and intervertebral discs (IVDs)) for volumetric magnetic resonance (MR) image plays a significant role in various spinal disease diagnoses and treatments of spine disorders, yet is still a challenge due to the inter-class similarity and intra-class variation of spine images. Existing fully convolutional network based methods failed to explicitly exploit the dependencies between different spinal structures. In this article, we propose a novel two-stage framework named SpineParseNet to achieve automated spine parsing for volumetric MR images. The SpineParseNet consists of a 3D graph convolutional segmentation network (GCSN) for 3D coarse segmentation and a 2D residual U-Net (ResUNet) for 2D segmentation refinement. In 3D GCSN, region pooling is employed to project the image representation to graph representation, in which each node representation denotes a specific spinal structure. The adjacency matrix of the graph is designed according to the connection of spinal structures. The graph representation is evolved by graph convolutions. Subsequently, the proposed region unpooling module re-projects the evolved graph representation to a semantic image representation, which facilitates the 3D GCSN to generate reliable coarse segmentation. Finally, the 2D ResUNet refines the segmentation. Experiments on T2-weighted volumetric MR images of 215 subjects show that SpineParseNet achieves impressive performance with mean Dice similarity coefficients of 87.32 ± 4.75%, 87.78 ± 4.64%, and 87.49 ± 3.81% for the segmentations of 10 vertebrae, 9 IVDs, and all 19 spinal structures respectively. The proposed method has great potential in clinical spinal disease diagnoses and treatments.

MOTIVATION: Imaging genetics is mainly used to reveal the pathogenesis of neuropsychiatric risk genes and understand the relationship between human brain structure, functional and individual differences. Increasingly, the brain-wide imaging phenotypes in voxels are available to test the association with genetic markers. A challenge with analyzing such data is their high dimensionality and complex relationships. RESULTS: To tackle this challenge, we introduce a weighed distance correlation (wdCor) that can assess the association between genetic markers and voxel-based imaging data. Importantly, the wdCor test takes the voxel-based data as a whole multivariate phenotype, which preserves the spatial continuity and might enhance the power. Besides, an adaptive permutation procedure is introduced to determine the P-values of the wdCor test and also alleviate the computational burden in GWAS. In extensive simulation studies, wdCor achieves much better performances compared to the original distance correlation. We also successfully apply wdCor to conduct a large-scale analysis on data from the Alzheimer's disease neuroimaging project (ADNI). AVAILABILITY AND IMPLEMENTATION: Our wdCor method provides new research directions and ideas for multivariate analysis of high-dimensional data, it can also be used as a tool for scientific analysis of imaging genetics research in practical applications. The R package wdcor, and the code for reproducing all results in this article is available in Github: https://github.com/yangyuhui0129/wdcor. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Unsupervised person reidentification has received more attention due to its wide real-world applications. In this paper, we propose a novel method named fuzzy multilayer clustering (FMC) for unsupervised person reidentification. The proposed FMC learns a new feature space using a multilayer perceptron for clustering in order to overcome the influence of complex pedestrian images. Meanwhile, the proposed FMC generates fuzzy labels for unlabeled pedestrian images, which simultaneously considers the membership degree and the similarity between the sample and each cluster. We further propose the fuzzy label regularization (FLR) to train the convolutional neural network (CNN) using pedestrian images with fuzzy labels in a supervised manner. The proposed FLR could regularize the CNN training process and reduce the risk of overfitting. The effectiveness of our method is validated on three large-scale person reidentification databases, i.e., Market-1501, DukeMTMC-reID, and CUHK03.

MOTIVATION: The detection of potential biomarkers of Alzheimer's disease (AD) is crucial for its early prediction, diagnosis and treatment. Voxel-wise genome-wide association study (VGWAS) is a commonly used method in imaging genomics and usually applied to detect AD biomarkers in imaging and genetic data. However, existing VGWAS methods entail large computational cost and disregard spatial correlations within imaging data. A novel method is proposed to solve these issues. RESULTS: We introduce a novel method to incorporate spatial correlations into a VGWAS framework for the detection of potential AD biomarkers. To consider the characteristics of AD, we first present a modification of a simple linear iterative clustering method for spatial grouping in an anatomically meaningful manner. Second, we propose a spatial-anatomical similarity matrix to incorporate correlations among voxels. Finally, we detect the potential AD biomarkers from imaging and genetic data by using a fast VGWAS method and test our method on 708 subjects obtained from an Alzheimer's Disease Neuroimaging Initiative dataset. Results show that our method can successfully detect some new risk genes and clusters of AD. The detected imaging and genetic biomarkers are used as predictors to classify AD/normal control subjects, and a high accuracy of AD/normal control classification is achieved. To the best of our knowledge, the association between imaging and genetic data has yet to be systematically investigated while building statistical models for classifying AD subjects to create a link between imaging genetics and AD. Therefore, our method may provide a new way to gain insights into the underlying pathological mechanism of AD. AVAILABILITY AND IMPLEMENTATION: https://github.com/Meiyan88/SASM-VGWAS.

Hippocampus segmentation plays a significant role in mental disease diagnoses, such as Alzheimer's disease, epilepsy, and so on. Patch-based multi-atlas segmentation (PBMAS) approach is a popular method for hippocampus segmentation and has achieved a promising result. However, the PBMAS approach needs high computation cost due to registration and the segmentation accuracy is subject to the registration accuracy. In this paper, we propose a novel method based on iterative local linear mapping (ILLM) with the representative and local structure-preserved feature embedding to achieve accurate and robust hippocampus segmentation with no need for registration. In the proposed approach, semi-supervised deep autoencoder (SSDA) exploits unsupervised deep autoencoder and local structure-preserved manifold regularization to nonlinearly transform the extracted magnetic resonance (MR) patch to embedded feature manifold, whose adjacent relationship is similar to the signed distance map (SDM) patch manifold. Local linear mapping is used to preliminarily predict SDM patch corresponding to the MR patch. Subsequently, threshold segmentation generates a preliminary segmentation. The ILLM refines the segmentation result iteratively by ensuring the local constraints of embedded feature manifold and SDM patch manifold using a space-constrained dictionary update. Thus, a refined segmentation is obtained with no need for registration. The experiments on 135 subjects from ADNI dataset show that the proposed approach is superior to the state-of-the-art PBMAS and classification-based approaches with mean Dice similarity coefficients of 0.8852±0.0203 and 0.8783 ± 0.0251 for bilateral hippocampus segmentation of 1.5T and 3.0T datasets, respectively.

Conducting an accurate motion correction of liver dynamic contrast-enhanced magnetic resonance (DCE-MR) imaging remains challenging because of intensity variations caused by contrast agents. Such variations lead to the failure of the traditional intensity-based registration method. To address this problem, we propose a correlation-weighted sparse representation framework to separate the contrast agent from original liver DCE-MR images. This framework allows the robust registration of motion components over time without intensity variances. Existing sparse coding techniques recover a 3D image containing only contrast agents (named contrast enhancement component) from a manually labeled dictionary, whose column has the same size with the original 3D volume (3D-t mode). The high dimension of the recovery target (3D volume) and the indistinguishability between the unenhanced and enhanced images make accurate coding difficult. In this paper, we predefine an ideal time-intensity curve containing only contrast agents (named contrast agent curve) and recover it from the transpose dictionary (t-3D mode), whose column has been updated into the original time-intensity curves. The low dimension of the target (1D curve) and the significant intergroup difference between contrast agent curves and non-contrast agent curves can estimate a series of pure contrast agent curves. A "correlation-weighted" constraint is introduced for the selection of a coding subset with more contrast agent curves, leading to an efficient and accurate sparse recovery process. Then, the contrast enhancement component can be estimated by the solved sparse coefficients' map and the ideal curve and subtracted from the original DCE-MRI. Finally, we register the de-enhanced images and apply the obtained deformation fields for the original DCE-MRI to achieve the goal of motion correction. We conduct the experiments on both simulated and real liver DCE-MRI data. Compared with other state-of-the-art DCE-MRI registration methods, the experimental results show that our method achieves a better registration performance with less computational efficiency.

Recently, visual Internet of Things (VIoT) has been deployed in many critical missions that are related to society security, such as anti-terrorism, abnormal event detection, crisis monitoring surveillance, etc. In this paper, we focus on a key fundamental problem in VIoT, person reidentification, which aims to correlate people with appropriate labels. The same person captured by various visual sensors in VIoT appears different significantly. To effectively measure the similarity between image pairs, we propose a novel method named discriminative structural metric learning (DSML), which utilizes intraregion metric, weak extra-region metric and extra-region metric to fully mine the structural information of pedestrian in a local way. According to DSML, we obtain a vector where each element is a local similarity score between two subregions. In order to aggregate the local similarity scores into a global one, we further propose a novel aggregating similarity score method named discriminative subregion aggregating (DSA). The DSA could learn the discriminative subregion by assigning different weights for each subregion. The experimental results demonstrate that the proposed method achieves better performance than the state-of-the-art methods.

Accurate segmentation of the prostate in computed tomography (CT) images is important in image-guided radiotherapy; however, difficulties remain associated with this task. In this study, an automatic framework is designed for prostate segmentation in CT images. We propose a novel image feature extraction method, namely, variant scale patch, which can provide rich image information in a low dimensional feature space. We assume that the samples from different classes lie on different nonlinear submanifolds and design a new segmentation criterion called local independent projection (LIP). In our method, a dictionary containing training samples is constructed. To utilize the latest image information, we use an online updated strategy to construct this dictionary. In the proposed LIP, locality is emphasized rather than sparsity; local anchor embedding is performed to determine the dictionary coefficients. Several morphological operations are performed to improve the achieved results. The proposed method has been evaluated based on 330 3-D images of 24 patients. Results show that the proposed method is robust and effective in segmenting prostate in CT images.