The continuous advancements in life science technology have enabled spatial transcriptome technology to achieve an impressive level of resolution at the single-cell level. This technology has emerged as a crucial method for studying the cellular composition and differentiation states of tissues, investigating cell-cell interactions, and unraveling the molecular mechanisms underlying diseases and developmental processes. A key component in this analysis is the accurate segmentation of cells. However, existing segmentation methods often fail to fully leverage the valuable information provided by spatial transcriptomics, leading to inaccurate cell segmentation. In this study, we introduce SSL-CST, a cell segmentation for single-cell spatial transcriptome method based on self-supervised learning. SSL-CST employs a pre-trained model for foundational contour segmentation. Following the denoising process, it utilizes a self-supervised neural network to correct the cell boundaries to obtain accurate cell boundaries. Through this approach, SSL-CST outperforms other state-of-the-art methods in various tests conducted on multiple datasets. The improved segmentation provided by SSL-CST further enhances the analysis of single-cell spatial expression, providing effective tools for biological discovery.

Recent advances in spatial transcriptomics have enabled the simultaneous measurement of gene expression profiles and spatial location information, offering a more comprehensive and in-depth view for studying the tissue microenvironment. Spatial domain identification is a crucial step in analyzing spatial transcriptomics. However, current methods have poor accuracy and visualization because they lack self-adaptability to different tissue data, and moreover, they cannot effectively extract spatial location information. To address these issues, we propose an adaptive graph contrastive learning framework based on multi-head graph attention networks (GATCL) for spatial domain identification. Specifically, we design a data augmentation module to mask and shuffle the pre-processed gene expression data to generate more differentiated negative samples. In addition, we construct the multi-head graph attention networks (MHGAT) to encode gene expression profiles and spatial location information. More importantly, we design an adaptive graph contrastive learning model that works both with positive and negative samples from spatial transcriptomics. We introduce the attention pooling mechanism to dynamically and adaptively aggregate the spots' neighborhood information, and to improve the model's generalization ability for different spatial transcriptomics data. Furthermore, we design a discriminator that adds spectral normalization to bilinear functions. Experimental results on DLPFC, breast cancer, and mouse somatosensory cortex datasets demonstrate that the average Adjusted Rand Index (ARI) scores are 0.5746, 0.6182, and 0.6496, respectively, significantly outperforming baseline methods. More importantly, GATCL provides a more detailed visualization of different spatial transcriptomics data.

Recent advances in spatial transcriptomics have enabled the integration of gene expression profiles with precise spatial coordinates, which have facilitated the exploration of tumor occurrence and development mechanisms, as well as the development of more effective targeted and immunotherapy approaches for tumor treatment. Deciphering cell type represents a critical challenge in spatial transcriptomics research. Existing methods are limited by the pervasive “dropout” events in spatial transcriptomics, hindering their ability to fully capture the relationship between spatial location and gene expression, thereby compromising the performance of cell type deconvolution. To address these limitations, we propose a spatial-aware masked graph transformer-diffusion model (SAMGTD) for enhanced cell type deconvolution in spatial transcriptomics. For spatial transcriptomics, the masked graph transformer model is designed to adaptively capture complex dependencies between spatial locations and gene expression. It employs a masking strategy that guides the model to focus on important local information during training, while the multi-head attention mechanism captures global context. More importantly, the spatial diffusion model is constructed to achieve the dual enhancement of spatial transcriptomics, including denoising and data imputation. It incorporates the multi-head attention mechanism and residual blocks, effectively addressing the “dropout” issue commonly encountered in spatial transcriptomics. For scRNA-seq, we construct a variational autoencoder to reduce noise interference while preserving key gene expression information. Finally, we construct a spatial-aware contrastive learning model to integrate scRNA-seq and spatial transcriptomics for cell type deconvolution. Experiments conducted on three datasets demonstrate that SAMGTD outperforms baseline methods.

Lysine crotonylation (Kcr) is an important protein modification occurring after translation in biology, serving an essential function in a range of biological processes in both plants and animals, including the regulation of gene expression, the maintenance of cellular metabolic balance, and the enhancement of photosynthesis. Exploring the detection of Kcr sites is essential for uncovering their biological functions. Nonetheless, conventional experimental approaches for detection are often time-consuming, expensive, and hindered by various technical constraints, making the precise identification of Kcr sites a significant challenge. This study seeks to develop a computational approach for the rapid and accurate prediction of Kcr sites in plant non-histone proteins. We introduce a novel deep learning framework named DeepNhKcr, which integrates the protein language model (ESM2) with a bidirectional long short-term memory (BiLSTM) network. To address the challenge of data imbalance, the model replaces the conventional cross-entropy loss with the focal loss function. In addition, DeepNhKcr combines advanced deep learning approaches with traditional protein encoding strategies to enable effective feature extraction and integration. This method not only significantly boosts the accuracy of predicting Kcr sites in non-histone proteins of plants. but also provides interpretability, shedding light on the potential links between key sequence characteristics and their biological roles. DeepNhKcr delivers outstanding results, surpassing existing machine learning and deep learning models, and demonstrating excellent performance in both five-fold cross-validation and independent test experiments. Moreover, the model integrates interpretability analysis techniques to investigate the connections between important sequence features and their biological roles. DeepNhKcr acts as a powerful method for detecting Kcr sites in plant non-histone proteins and is anticipated to greatly advance future studies in plant Kcr site prediction.

2'-O-methylation (2OM) of ribose is a widespread RNA modification that significantly impacts RNA stability, structure, and function. Accurately predicting 2OM sites is crucial for understanding RNA's biological functions and related pathologies. Traditional detection methods pose challenges such as resource intensiveness, potential RNA sample damage, and high costs. However, recent advancements in machine learning, particularly deep learning techniques, offer rapid and cost-effective prediction solutions. In this study, we introduce DeepR2OM, a novel method integrating feature selection and deep learning for 2OM sites prediction. DeepR2OM encodes sequences using eight RNA descriptors, employs feature selection algorithms to reduce dimensions, and then utilizes a deep learning network for training. After evaluating various deep learning architectures, we selected Convolutional Neural Network (CNN), Multi-Head Self-Attention mechanism, and Deep Neural Network (DNN) as our final prediction models. Experimental results demonstrate DeepR2OM's effectiveness, achieving 87.1% accuracy (ACC), 85.5% recall rate (Recall), 87.9% precision (PRE), and a Matthews correlation coefficient (MCC) of 75.7% on an independent test set. This tool serves as a valuable resource for exploring the functional and bioinformatic aspects of 2OM sites.

Along with the development of the Industrial Internet of Things, human activity recognition (HAR) has received widespread attention in many fields. Support Vector Machine (SVM), is widely used by researchers for human activity recognition. However, the inherent difference of signal properties from different sensors and various orientations is potentially lost when using the vector-based SVM for human activity recognition. What's more, the outlier sensitivity problem of the standard SVM reduces the accuracy of human activity recognition. To tackle this problem, we present a tensor-based feature representation model and a weighted support tensor machine (WSTM) for human activity recognition. Specifically, tensor-based representations are first used to model features from different sensors and various orientations to retain the latent relationship. In addition, the weighted support tensor machine is proposed to classify the human activities in tensor space while avoiding the outlier sensitivity problem. Experimental results demonstrate the proposed WSTM algorithm.

Accurate cancer prognosis is critical for improving patient outcomes. However, existing models often struggle to handle high-dimensional and heterogeneous multi-omics data. This paper proposes DADRSurv, a novel deep learning method that effectively integrates multi-omics data, including mRNA and miRNA, with clinical variables. The proposed architecture uniquely combines a dual variational autoencoder (VAE) with a dual recurrent neural network (RNN). Specifically, the model utilizes cascaded encoders for robust feature extraction from high-dimensional omics data and employs an RNN with a timeaware attention mechanism to dynamically capture complex temporal relationships for survival prediction. Extensive experiments on TCGA breast (BRCA) and ovarian (OV) cancer datasets demonstrate that DADRSurv outperforms various state-of-theart models in prognostic prediction, achieving concordance index (C-index) scores of 0.763 (BRCA) and 0.662 (OV) on multi-modal data. Furthermore, the model also effectively stratiffes patients into distinct high- and low-risk groups (Log-rank$p<0.0001$). The experimental results prove its power as a robust framework for cancer survival analysis and shows its potential to enhance clinical decision-making.

Spatial domain identification is a central task in spatial transcriptomics (ST) data analysis. We present GDST, a novel graph contrastive learning framework that leverages graph diffusion to enhance spatial domain delineation in ST data. In contrast to prior approaches based on random perturbations or masking strategies, GDST introduces biologically inspired diffusion augmentation to simulate intercellular signal propagation while preserving the intrinsic topological structure of the spatial graph. To further reduce noise from irrelevant neighbors, a graph attention mechanism is incorporated to enable adaptive neighborhood aggregation. Extensive evaluations on five benchmark ST datasets spanning three experimental platforms demonstrate that GDST consistently outperforms several state-of-the-art deep learning models in spatial domain identification.

Reconstructing three-dimensional (3D) structures from consecutive tissue slices is essential to uncover tissue architecture and intercellular interactions. However, some of the current 3D reconstruction methods rely on two-stage pipelines and precise point-to-point registration, and they are overly reliant on expert knowledge. Others assume globally consistent spatial omics patterns, and they fail to account for spatial heterogeneity. To overcome the above limitations, we propose STAlignDRL, a deep reinforcement learning framework for spatial transcriptomics 3D reconstruction which can achieve end-to-end consecutive slice alignment. In detail, the proposed algorithm works in three steps. First, we perform data preprocessing to obtain the features of the slices and extract similar local contours and action coefficients between the slices. Next, we formulate the slice alignment operation of 3D reconstruction as a sequential decision-making process. Finally, we construct an end-to-end fully connected deep reinforcement learning network to achieve high-precision slice alignment. Notably, we are the first to propose a reinforcement learning-based method in the context of spatial transcriptomics 3D reconstruction, providing new insights into the 3D reconstruction of consecutive spatial transcriptomics slices. To validate our method, we conduct experiments on the Breast Cancer, DLPFC 3, and Mouse Hippocampus datasets, and compare it with the PASTE, STAligner, ST-GEARS, STitch3D, and SANTO methods on the Mouse Hippocampus dataset. The experimental results show that STAlign-DRL outperforms several recent state-of-the-art methods.

Identifying genes associated with phenotypes is cru-cial for discovering biological pathways and molecular mechanisms. Although heterogeneous graph representations serve as powerful tools for modeling complex phenotype-gene relationships, existing approaches face several limitations in prediction accuracy. Our framework addresses this challenge in three stages. First, we integrate three distinct datasets to construct a phenotype-gene heterogeneous graph. This graph is then processed by a Heterogeneous Graph Attention Network (HAN) to generate informative node embeddings through adaptively aggregating features from diverse neighbor types. Finally, these embeddings are passed to a relation-specific encoder-decoder architecture. Here, the encoder refines structural dependencies, which are then used by the decoder to produce association scores for link prediction. Experimental results demonstrate that our model significantly outperforms existing heterogeneous graph-based methods, achieving higher precision and recall.

Real-world knowledge graphs abound with higher-order logical relations that simple triples, limited to pairwise connections, fail to represent. Thus, capturing higher-order logical relations involving multiple entities has garnered significant attention. However, existing methods ignore the structural information in higher-order relations. To this end, we propose a higher-order logical knowledge representation learning method, named LORE, which leverages network motifs, the patterns/subgraphs that naturally capture the structural information in graphs, to extract higher-order features and ultimately, learn effective representations of knowledge graphs. Compared to existing approaches, LORE aggregates the attribute features of entities with the extracted higher-order logical relations to form enhanced representations of knowledge graphs. In particular, three aggregators (i.e., Hadamard, Connection, and Summation) are proposed and employed. Extensive experiments have been conducted on six real-world datasets for two downstream tasks (i.e., entity classification and link prediction). The results show that LORE outperforms baselines significantly and consistently.

Cancer survival prediction is an important direction in precision medicine, aiming to help clinicians tailor treatment regimens for patients. With the rapid development of high-throughput sequencing and computational pathology technologies, survival prediction has shifted from clinical features to joint modeling of multi-omics data and pathology images. However, existing multimodal learning methods struggle to effectively learn pathology-omics interactions due to the lack of proper alignment of multimodal data before fusion. In this paper, we propose POMP, a pathology-omics multimodal pre-training framework jointly learned with three training tasks for integrating pathological images and omics data for cancer survival prediction. To better perform cross-modal learning, we introduce a pathology-omics contrastive learning method to align the pathology and omics information. POMP leverages the principle of pre-trained models and explores the benefit of aligning multimodal information from the same patient, achieving state-of-the-art results on six cancer datasets from the Cancer Genome Atlas (TCGA). We also show that our contrastive learning method allows us to exploit the cosine similarity of pathological images and omics data as the survival risk score, which can further boost prediction performance compared with other commonly used methods. The code is available at https://github.com/SuixueWang/POMP.

Deep multi-view clustering has attracted increasing attention in the pattern mining of data. However, most of them perform self-learning mechanisms in a single space, ignoring the fruitful structural information hidden in different-level feature spaces. Meanwhile, they conduct the reconstruction constraint to learn generalized representations of samples, failing to explore the discriminative ability of complementary and consistent information. To address the challenges, a multi-granularity invariant structure clustering scheme (MASTER) is proposed to define a bottom-up process that extracts multi-level information in sample, neighborhood, and category granularities from low-level, high-level, and semantics feature space, respectively. Specifically, it leverages the self-learning reconstruction with information-theoretic overclustering to capture invariant sample structure in the low-level feature space. Then, it models data diffusion of the clustering process in the reliable neighborhood to capture invariant local structure in the high-level feature space. Meanwhile, it defines dual divergences induced by the space geometry to capture invariant global structure in the semantics space. Finally, extensive experiments on 8 real-world datasets show that MASTER achieves state-of-the-art performance compared to 11 baselines.

With the development of wearable cardiac ultrasound devices, it is no longer sufficient to solely rely on doctors for diagnosing long-term echocardiogram videos. Automated diagnosis of echocardiogram videos has now become a research hotspot. Existing studies only analyze echocardiogram video through discriminative models, which have limited question-answering capabilities. Therefore, this study innovatively proposes a large language model with cardiac ultrasound diagnostic capabilities—EchoGPT. EchoGPT integrates the robust communication and comprehension capabilities of large language models (LLMs) with the diagnostic prowess of traditional medical models, empowering patients to obtain accurate medical indicator data and comprehend their health conditions through interactive questioning with the model. The model is capable of local deployment on personal computers, effectively safe guarding user privacy. EchoGPT operates through three main components: left ventricle segmentation, left ventricular ejection fraction LVEF prediction, and finetuning of video-text LLMs. Experimental results demonstrate EchoGPT’s superior accuracy in predicting LVEF compared to other models, and positive feedback from professional physicians through questionnaire surveys, validating its potential in practical applications. The demo is available at https://github.com/zhuqh19/EchoGPT.

Most visual-guided image inpainting methods based on generative adversarial networks (GANs) struggle when the missing region has weak correlations with the surrounding visual context. Recently, diffusion-based methods guided by textual context have been proposed to address this limitation by leveraging additional semantic information to restore corrupted objects. However, these models typically involve more parameters and exhibit slower generation speeds compared to GAN-based approaches. To address this problem, we propose a novel text-guided image inpainting model, the cross-modal semantic fusion generative adversarial network (CSF-GAN). CSF-GAN is designed as a one-stage GAN with the following key contributions. First, a novel semantic fusion module (SFM) is introduced to integrate sentence- and word-level textual context into the inpainting process, enabling more effective guidance from multi-granularity semantic information. Second, a newly designed word-level local discriminator provides detailed feedback to the generator, enhancing the accuracy of generated content in alignment with word-level semantics. Third, two loss functions, the inpainting loss and edge loss, are employed to enhance both structural coherence and textural realism in the generated results. Extensive experiments on two benchmark datasets demonstrate that CSF-GAN outperforms state-of-the-art methods.

Recently, multi-view data has gradually attracted attention. However, real-world applications often face Partial View-aligned Problem (PVP) and Partially Sample-missing Problem (PSP) due to data loss or corruption. Existing methods addressing PVP typically focus only on learning from the information of aligned data, while ignoring unaligned data where samples exist but lack alignment relationships. This introduces PSP, which does not inherently exist in the data, leading to biased learning of the data's information. For PSP, due to varying degrees of missing data, incomplete spatial structures can cause clustering centers-shifted problem, resulting in the model learning incorrect correspondences and biased spatial structures.To tackle them, we propose a novel method called Dual Robust Unbiased Multi-View Clustering for Incomplete and Unpaired Information (DRUMVC). To our knowledge, this is the first noise-robust and unbiased multi-view clustering method capable of simultaneously addressing both PVP and PSP. Specifically, DRUMVC leverages aligned and complete samples as a bridge to construct high-quality correspondences for samples lacking cross-view relationship information due to PVP or PSP. Additionally, we employ a dual noise-robust contrastive learning loss to mitigate the impact of noise potentially introduced during the pair construction. Experiments on several challenging datasets demonstrate the superiority of our proposed method.

Multimodal feature fusion, by integrating the complementary information from each modality, can effectively capture complex features in real-world data. However, in many use cases, such as boiler combustion monitoring, factors including equipment failure, inconsistent sensor sampling frequencies, and network delays often cause data collected from different modalities to suffer from missing modality and temporal asynchrony. This leads to the incompleteness and disorderliness of multimodal data. To address these issues, previous studies have proposed several data fusion methods that align the cluster centers before fusion. However, these approaches have two key limitations: 1) they do not guarantee a high alignment accuracy of data pairs at the sample level, and 2) they do not address the issue of significant discrepancies in data sizes across different classes, which impacts the subsequent data fusion performance.

RNA-binding proteins (RBPs) play crucial roles in gene regulation. Their dysregulation has been increasingly linked to neurodegenerative diseases, liver cancer, and lung cancer. Although experimental methods like CLIP-seq accurately identify RNA-protein binding sites, they are time-consuming and costly. To address this, we propose RMDNet-a deep learning framework that integrates CNN, CNN-Transformer, and ResNet branches to capture features at multiple sequence scales. These features are fused with structural representations derived from RNA secondary structure graphs. The graphs are processed using a graph neural network with DiffPool. To optimize feature integration, we incorporate an improved dung beetle optimization algorithm, which adaptively assigns fusion weights during inference. Evaluations on the RBP-24 benchmark show that RMDNet outperforms state-of-the-art models including GraphProt, DeepRKE, and DeepDW across multiple metrics. On the RBP-31 dataset, it demonstrates strong generalization ability, while ablation studies on RBPsuite2.0 validate the contributions of individual modules. We assess biological interpretability by extracting candidate binding motifs from the first-layer CNN kernels. Several motifs closely match experimentally validated RBP motifs, confirming the model's capacity to learn biologically meaningful patterns. A downstream case study on YTHDF1 focuses on analyzing interpretable spatial binding patterns, using a large-scale prediction dataset and CLIP-seq peak alignment. The results confirm that the model captures localized binding signals and spatial consistency with experimental annotations. Overall, RMDNet is a robust and interpretable tool for predicting RNA-protein binding sites. It has broad potential in disease mechanism research and therapeutic target discovery. The source code is available https://github.com/cskyan/RMDNet .

With the rapid development of Internet of Things (IoT), biometric-based authentication systems have been widely used for access control. The wide application of biometric recognition systems has brought convenience but also raised privacy and security concerns. When unprotected templates are stolen, it will permanently leak the original biometric data. Therefore, it is important to ensure the security of biometric templates while meeting the real-time device requirements. Motivated by these issues, in this paper, we proposed a scheme based on partial Cake-cutting transformation and spherical hashing to generate cancelable binary face templates. Firstly, with external random parameters, partial Cake-cutting transformation is established to introduce randomness and preserve the relative distance similarity of face features. Then spherical hashing is utilized to encode the face features into protected binary codes. The protected template has the advantages of high entropy value, low storage consumption, and fast generation speed. Extensive experiments conducted on LFW, CFPW, and CASIA-FaceV5 databases along with theoretical analyses indicate that the proposed scheme shows good matching accuracy and strong resistance to various attacks. Besides, the protected templates can achieve equal or even better accuracy than the unprotected counterparts. Furthermore, the proposed scheme also satisfies the requirements of cancelable biometrics, i.e., irreversibility, revocability, and unlinkability.

Anticancer peptide (ACP) are short peptides with anti-cancer properties that have generated increasing attention in recent years due to their low toxicity, minimal side effects, and their ability to precisely target and kill cancer cells. Traditionally, identifying ACP has relied on experimental methods, which are time-consuming and labor-intensive. While deep learning-based prediction methods have made significant progress, there is still room for improvement in achieving optimal performance. In this study, we present ACP-ESM2, a deep learning framework based on the Evolutionary Scale Modeling 2 (ESM2) pre-trained model, which captures rich evolutionary information from protein sequences. By combining ESM2 with convolutional neural network (CNN) that excels at detecting local patterns, ACP-ESM2 offers a highly accurate tool for ACP prediction. The experimental results indicate that ACP-ESM2 shows significant improvements over best-existing recognition techniques on the Test1 set, with enhancements of 2.3%, 7.2%, 12.6%, and 5% in ACC, SN, SP, and MCC, respectively. Notably, on the Test2 set, ACP-ESM2 achieves an accuracy of 97.6%, showcasing its exceptional robustness. This establishes ACP-ESM2 as an efficient and precise tool for predicting anticancer peptides.

Zero-shot learning (ZSL) aims to recognize unseen classes by transferring semantic knowledge from seen categories. However, existing methods often struggle with the persistent semantic gap caused by limited semantic descriptors and rigid visual feature modeling. In particular, modeling pre-defined class-level attribute descriptions as ground truth hinders effective semantic-to-visual alignment to some extent. To mitigate these issues, we propose the Bilateral-guided Prototype Refinement Network (BPRN), a novel ZSL framework designed to refine dual prototypes across meta-domains of varying scales. Specifically, we first disentangle the relationships among class-level semantics and use them to generate corresponding pseudo-visual prototypes. Then, by leveraging distribution information across dual prototypes in different meta-domains, BPRN achieves bidirectional calibration between visual-to-semantic and semantic-to-visual modalities. Finally, a synthesized class-level representation derived from the refined dual prototypes is employed for inference, instead of relying on a single prototype. Extensive experiments conducted on five widely-used ZSL benchmark datasets demonstrate that BPRN consistently achieves competitive or even superior performance. Specifically, in the GZSL scenario, BPRN shows improvements of 2.1%, 7.3%, 6.1%, and 4.8% on AWA1, AWA2, SUN, and aPY, respectively, compared to existing embedding-based ZSL methods. Ablation studies and visualization analyses further validate the effectiveness of the proposed components.

Spatial transcriptomics analysis methods offer an opportunity to investigate highly diverse biological tissues. Cell-cell communication is fundamental for maintaining physiological homeostasis in organisms and coordinating complex biological processes. Identifying cell-cell interactions is critical for understanding cellular activities. The interaction of a cell with other cells depends on several factors, and most of the existing methods that consider only gene expression information of neighbouring cells and spatial location information are somewhat limited. In this paper, we propose a network architecture based on graph convolution network and long short-term memory attention module-GCNLA, which contains graph convolution layer, long short-term memory network, attention module, and residual connections. GCNLA not only learns the spatial structure of cells but also captures interaction information between distal cells, the attention module further extracting and enhancing features related to cell-cell interactions. Finally, the inner product decoding calculates the cosine similarity, which is used to infer cell-cell interactions. In addition, GCNLA is capable of reconstructing the complete cell-cell interaction network. The experimental results on seqFISH and MERFISH demonstrate that the GCNLA network structure has better robustness and noise immunity. The potential features learned by GCNLA enable other downstream analyses, including single-cell resolution cell clustering based on spatial information resolving cell heterogeneity.

The microbiome has emerged as a promising predictor of human cancers. Fungi are important components of the human microbiome and are closely associated with cancer. However, our understanding of the function and efficacy of fungal cells in tumors remains limited, making it challenging to accurately detect cancer tissues using limited tumor mycobiome data. Transfer learning has recently revolutionized the bioinformatics field by leveraging deep learning models pre-trained on large-scale general datasets, effectively addressing the predicted issue in tasks with limited data. Here, we propose an interpretable large model, MCaPred, that encodes different tumor tissue sites with microbiome features. The model was pre-trained on large-scale microbiome data from different tumor tissue sites to predict cancer in limited mycobiome data samples, thereby exploring the association between fungi and cancer-type specificity. Studies have shown that MCaPred outperforms recent deep learning models in cancer screening based on tumor mycobiome data and that mycobiome-based cancer detection is augmented by pretrained models. More importantly, MCaPred can capture key factors from mycobiome data related to specific cancer occurrences because of its high interpretability. The proposed model, analysis code, and supplementary materials used in this study are available at GitHub (https://github.com/cskyan/MCaPred.git). Contact: [email protected] or [email protected]

With the rapid advancement in multimodal fusion technology, the integration of pathological images with genomics data has achieved promising results in cancer survival prediction. However, most existing multimodal models are not pre-trained by combining pathology and genomics modalities, ignoring the inherent task-agnostic associations between different modalities. While some self-supervised methods align multimodal information through pre-training objectives such as correlation and mean square error, they lack in-depth multimodal interaction. To address these issues, we propose ContraMAE, a contrastive alignment masked autoencoder framework, to fuse pathological images and genomics data for cancer survival prediction. Concretely, we introduce a contrastive objective to align multimodality and construct their intrinsic consistency. Besides, we design two reconstruction objectives to capture the complex relationships between multi-modalities by mutually compensating for the information that each side lacks. In survival prediction, the pathology and genomics encodings from the ContraMAE encoder are concatenated as the final representation to generate a survival risk score. Experimental results demonstrate that ContraMAE outperforms existing state-of-the-art methods on five cancer datasets sourced from The Cancer Genome Atlas (TCGA). The code is available at https://github.com/SuixueWang/ContraMAE.

Cancer has emerged as a significant threat to human health, leading to numerous fatalities globally. It is important for improving the level of cancer treatment to predict the response of cancer patients to drugs,which is based on their personalized differences. Existing methods for cancer drug response prediction are difficult to effectively capture the differences and correlations between multi-omics features and extract the complex structural patterns of drug molecules, resulting in insufficient accuracy of drug response prediction. To solve these problems, we propose a graph contrastive learning method with multi-omics for cancer drug response prediction (GCLM-CDR). Firstly, we construct a multi-omics drug feature representation module to extract multi-omics features and complex structural patterns of drug molecular graphs. Specifically, for multi-omics data, we use deep neural networks to extract multi-omics features, then we construct a multi-omics neighbor interaction module to capture differences and correlations between different omics data. For drugs, the graph attention network is used to effectively extract complex structural patterns of drug molecular graphs. Secondly, we construct a graph contrastive learning module to further enhance the feature representation after the fusion of multi-omics and drug molecular graphs. In this task, the graph construction strategies of effective positive and negative sample are designed for two types of data. Finally, we construct a cancer drug response prediction module to obtain the prediction results. The experimental results on the GDSC dataset and CCLE dataset showed that the AUC were 0.8534 and AUPR were 0.5327, which were superior to existing methods.

RNASite is a comprehensive platform that integrates multiple RNA modification site databases and servers, focusing on RNA modification sites. These modification sites significantly affect RNA's structure, stability, and function, playing a crucial role in epigenetics and gene expression regulation. The website offers both datasets and online RNA modification site identification tools to enhance the recognition and understanding of these modification sites. With advancements in high-throughput sequencing technology and machine learning, RNASite leverages these innovations to provide efficient and cost-effective RNA modification site identification methods. The platform features 17 high-quality datasets covering 11 common RNA modification sites and includes 7 online identification tools. Notably, some of these tools exceed the accuracy of existing models. RNASite is designed to be a convenient and efficient resource for researchers in biochemistry and bioinformatics, facilitating progress in the study of RNA modification sites. The platform is available for free at http://www.bioai-lab.com/RNASite.

Graph dataset condensation (GDC) reduces a dataset with many graphs into a smaller dataset with fewer graphs while maintaining model training accuracy. GDC saves the storage cost and hence accelerates training. Although several GDC methods have been proposed, they are all supervised and require massive labels for the graphs, while graph labels can be scarce in many practical scenarios. To fill this gap, we propose a self-supervised graph dataset condensation method called SGDC, which does not require label information. Our initial design starts with the classical bilevel optimization paradigm for dataset condensation and incorporates contrastive learning techniques. But such a solution yields poor accuracy due to the biased gradient estimation caused by data augmentation. To solve this problem, we introduce representation matching, which conducts training by aligning the representations produced by the condensed graphs with the target representations generated by a pre-trained SSL model. This design eliminates the need for data augmentation and avoids biased gradient. We further propose a graph attention kernel, which not only improves accuracy but also reduces running time when combined with self-supervised kernel ridge regression (KRR). To simplify SGDC and make it more robust, we adopt a adjacency matrix reusing approach, which reuses the topology of the original graphs for the condensed graphs instead of repeatedly learning topology during training. Our evaluations on seven graph datasets find that SGDC improves model accuracy by up to 9.7% compared with 5 state-of-the-art baselines, even if they use label information. Moreover, SGDC is significantly more efficient than the baselines.

Spatial transcriptome technology can parse transcriptomic data at the spatial level to detect high-throughput gene expression and preserve information regarding the spatial structure of tissues. Identifying spatial domains, that is identifying regions with similarities in gene expression and histology, is the most basic and critical aspect of spatial transcriptome data analysis. Most current methods identify spatial domains only through a single view, which may obscure certain important information and thus fail to make full use of the information embedded in spatial transcriptome data. Therefore, we propose an unsupervised clustering framework based on multiview graph convolutional networks (MVST) to achieve accurate spatial domain recognition by the learning graph embedding features of neighborhood graphs constructed from gene expression information, spatial location information, and histopathological image information through multiview graph convolutional networks. By exploring spatial transcriptomes from multiple views, MVST enables data from all parts of the spatial transcriptome to be comprehensively and fully utilized to obtain more accurate spatial expression patterns. We verified the effectiveness of MVST on real spatial transcriptome datasets, the robustness of MVST on some simulated datasets, and the reasonableness of the framework structure of MVST in ablation experiments, and from the experimental results, it is clear that MVST can achieve a more accurate spatial domain identification compared with the current more advanced methods. In conclusion, MVST is a powerful tool for spatial transcriptome research with improved spatial domain recognition.

Tuberculosis has plagued mankind since ancient times, and the struggle between humans and tuberculosis continues. Mycobacterium tuberculosis is the leading cause of tuberculosis, infecting nearly one-third of the world's population. The rise of peptide drugs has created a new direction in the treatment of tuberculosis. Therefore, for the treatment of tuberculosis, the prediction of anti-tuberculosis peptides is crucial. This paper proposes an anti-tuberculosis peptide prediction method based on hybrid features and stacked ensemble learning. First, a random forest (RF) and extremely randomized tree (ERT) are selected as first-level learning of stacked ensembles. Then, the five best-performing feature encoding methods are selected to obtain the hybrid feature vector, and then the decision tree and recursive feature elimination (DT-RFE) are used to refine the hybrid feature vector. After selection, the optimal feature subset is used as the input of the stacked ensemble model. At the same time, logistic regression (LR) is used as a stacked ensemble secondary learner to build the final stacked ensemble model Hyb_SEnc. The prediction accuracy of Hyb_SEnc achieved 94.68% and 95.74% on the independent test sets of AntiTb_MD and AntiTb_RD, respectively.

Smart healthcare has emerged to provide healthcare services using data analysis techniques. Especially, clustering is playing an indispensable role in analyzing healthcare records. However, large multi-modal healthcare data imposes great challenges on clustering. Specifically, it is hard for traditional approaches to obtain desirable results for healthcare data clustering since they are not able to work for multi-modal data. This paper presents a new high-order multi-modal learning approach using multimodal deep learning and the Tucker decomposition (F- HoFCM). Furthermore, we propose an edge-cloud-aided private scheme to facilitate the clustering efficiency for its embedding in edge resources. Specifically, the computationally intensive tasks, such as parameter updating with high-order back propagation algorithm and clustering through high-order fuzzy c-means, are processed in a centralized location with cloud computing. The other tasks such as multi-modal data fusion and Tucker decomposition are performed at the edge resources. Since the feature fusion and Tucker decomposition are nonlinear operations, the cloud cannot obtain the raw data, thus protecting the privacy. Experimental results state that the presented approach produces significantly more accurate results than the existing high-order fuzzy c-means (HOFCM) on multi-modal healthcare datasets and furthermore the clustering efficiency are significantly improved by the developed edge-cloud-aided private healthcare system.

As The cornerstone of human health, physical and mental well-being are intricately linked, influencing both an individual's physical condition and their emotional state [1]. Chronic diseases such as hypertension and diabetes can have a significant impact on mental health, leading to anxiety and depression [2]. Similarly, psychological problems such as stress, anxiety, and depression can weaken the immune system, making individuals more susceptible to physical illnesses. In recent years, the rapid development of technology has brought exciting new possibilities to the field of physical and psychological health. The Internet of Things (IoT) and artificial intelligence (AI) have shown great potential in building a comprehensive health management system that empowers individuals to take a more proactive role in their well-being.

Lung cancer is a leading cause of death. An accurate early lung cancer diagnosis can improve a patient’s survival chances. Histopathological images are essential for cancer diagnosis. With the development of deep learning in the past decade, many scholars have used deep learning to learn the features of histopathological images and achieve lung cancer classification. However, deep learning requires a large quantity of annotated data to train the model to achieve a good classification effect, and collecting many annotated pathological images is time-consuming and expensive. Faced with the scarcity of pathological data, we present a meta-learning method for lung cancer diagnosis (called MLLCD). In detail, the MLLCD works in three steps. First, we preprocess all data using the bilinear interpolation method and then design the base learner which units a convolutional neural network(CNN) and transformer to distill local features and global features of pathology images with different resolutions. Finally, we train and update the base learner with a model-agnostic meta-learning (MAML) algorithm. Clinical Proteomic Tumor Analysis Consortium (CPTAC) cancer patient data demonstrate that our proposed model achieves the receiver operating characteristic (ROC) values of 0.94 for lung cancer diagnosis.

Lung cancer is one of the leading causes of cancer-related death in humans, with very high morbidity and mortality. Accurately identifying the lung tumor markers can significantly improve the diagnostic capability of lung cancer, which helps clinicians detect potential lung cancer lesions earlier and take a more timely and effective treatment regime, thus improving the survival rate and quality of life of cancer patients. Recently, there has been growing interest in biomarker discovery based on multi-omics and multimodal data, which explore high-dimensional, high-throughput, and multi-scale biomedical data of lung cancer patients from various perspectives, including molecular, histopathology, radiology, and clinical records. Additionally, machine learning (ML) techniques can handle complex multi-omics and multimodal data more efficiently than traditional biomarker discovery methods, so ML has been widely adopted in finding lung cancer biomarkers. This paper reviews biomarker discovery using multimodal data with the potential application in lung tumor diagnosis. Specifically, we comprehensively summarize and analyze the current research status of biomarker discovery from three aspects, including multimodal data feature extraction and selection, multimodal integration methods, and biomarker discovery methods. Furthermore, we highlight some of the limitations of existing machine learning-based biomarker discovery methods using multi-omics and multimodal data for lung tumor diagnosis and outline our future directions.

Recently, the development of automatic question answering in the field of smart medicine has attracted much attention, especially in the field of Chinese medicine. Many methods have been presented for Chinese medical question answering. However, due to redundant information in sentences, existing Chinese medical question answering methods, like BRCNN, are difficult to choose the best answer for patients’ questions. Aiming at the problem, this paper proposes a BiGRU model based on attention mechanism, and adds it to BRCNN. Specifically, our algorithm first inputs the word vector generated by BERT into the attention module to get the attention of the sentence. Secondly, the attention and word vector of the sentence are input into BiGRU together, and the sequence information of the sentence is obtained through the interaction of attention and word vector. Thirdly, the sequence information of sentences is used as the input of multi-scale CNNs to obtain n-gram information of different sizes of sentences. Finally, the best answer to the question is selected by calculating the cosine similarity between the question and the answer. We conducted some experiments to verify the proposed model by comparing it with the baseline model on the cMedQA and cMedQA2 data sets. The results show that the accuracy of the proposed model is 0.5% and 0.2% higher than that of the baseline model, respectively.

Accurate cancer survival prediction enables clinicians to tailor treatment regimens based on individual patient prognoses, effectively mitigating over-treatment and inefficient medical resource allocation. Recently, the integration of histopathological images and multi-omics data, together with deep learning, has become increasingly applied to predict cancer survival. However, current deep learning-based integration methods ignore the spatial relationships across various fields of view within gigapixel histopathological images, since they mainly focus on a specific field of view. Inspired by the hierarchical image pyramid transformer (HIPT), we propose a hierarchical cross-attention masked autoencoder (HC-MAE) to integrate histopathological images and multi-omics data for cancer survival prediction. Specifically, HC-MAE aggregates the representations learned from different fields of view, effectively capturing the fine-grained details and the spatial relationships within histopathological images. We conduct experiments to compare the HC-MAE method with current state-of-the-art methods on six cancer datasets sourced from The Cancer Genome Atlas (TCGA). The experimental results demonstrate that HC-MAE achieves superior performance on five out of six cancer datasets, significantly outperforming the compared methods. The code is available at https://github.com/SuixueWang/HC-MAE.

Cell type deconvolution is a critical mission in spatial transcriptomics. We propose an adaptive dual contrastive learning framework (ADCL) based on MHGAT and VAE for cell type deconvolution. For spatial transcriptomic data, we construct unsupervised contrast learning module based on the multi-head graph attention networks (MHGAT) to encode gene expression profiles and spatial location information. For scRNA-seq data, we utilize a variational autoencoder (VAE) to reconstruct gene expression matrix and reduce the impact of noise. Finally, we construct contrastive learning deconvolution module to learn probability matrix to achieve cell type deconvolution. Experiments on two real datasets and one simulated dataset prove that ADCL outperforms current state-of-the-art approaches.

This research presents a music theory-inspired acoustic representation (hereafter, MTAR) to address improved speech emotion recognition. The recognition of emotion in speech and music is developed in parallel, yet a relatively limited understanding of MTAR for interpreting speech emotions is involved. In the present study, we use music theory to study representative acoustics associated with emotion in speech from vocal emotion expressions and auditory emotion perception domains. In experiments assessing the role and effectiveness of the proposed representation in classifying discrete emotion categories and predicting continuous emotion dimensions, it shows promising performance compared with extensively used features for emotion recognition based on the spectrogram, Mel-spectrogram, Mel-frequency cepstral coefficients, VGGish, and the large baseline feature sets of the INTERSPEECH challenges. This proposal opens up a novel research avenue in developing a computational acoustic representation of speech emotion via music theory.

Various stream learning methods are emerging in an endless stream to provide a wealth of solutions for artificial intelligence in streaming data scenarios. However, when each data stream is oriented to a different target space, it forces stream learning approaches oriented to the same task to be no longer applicable. Due to inconsistent target spaces for different tasks, the previous approaches fail on the new streaming tasks or it is impracticable to be trained from scratch with few labeled samples at the beginning. To this end, we have proposed an adaptive learning scheme for few-shot streaming tasks with the contributions of tensor and meta-learning. This adaptive scheme is conducive to mitigating the domain shift when a new task has few labeled samples. We elaborate a novel tensor-empowered attention mechanism derived from nonlocal neural networks, which enables to capture long-range dependency and preserve the high-dimensional structure to refine the global features of streaming tasks. Furthermore, we develop a fine-grained similarity computing approach, which is prone to better characterize the difference across few-shot streaming tasks. To show the superiority of our method, we have carried out extensive experiments on three popular few-shot datasets to simulate streaming tasks and evaluate the performance of adaptation. The results show that our proposed method has achieved competitive performance for few-shot streaming tasks compared with the state-of-the-art (SOTA).

Automatically generating radiology reports given radiographs has considerable promise for easing clinical workflows, reducing diagnostic errors, and ultimately streamlining clinical care. Therefore, this work has attracted much attention and early studies mainly employed cutting-edge techniques from computer vision and natural language generation to improve the readability of generated reports. However, those methods often fail to accurately convey critical information such as the presence status of disease, which is the first priority of medical image-to-text generation. Additionally, the class imbalance issues are frequently found in a number of radiographic datasets, resulting in missed diagnosis of some uncommon diseases. In this manuscript, we propose a clinically coherent radiology report generation method that introduces an additional memory-enhanced feature-wise affine transformation (MFAT) layer and an imbalance-aware clinically accurate reward (ICAR) to tackle the aforementioned issues. We evaluate the proposed method by comparing with state-of-the-art methods over both natural language generation (NLG) metrics and clinical efficacy results on two datasets, namely IU X-Ray and MIMIC-CXR. The results demonstrate that our method achieves significantly higher clinical efficacy accuracy for both common or uncommon disease annotations while still maintaining acceptably high NLG metrics for readability.

Sepsis is an emergency that usually causes a high mortality rate in intensive care units. A dynamic personalized optimal treatment strategy is necessary to develop for each patient of sepsis since even the same treatment strategy has different effects on different patients, which imposes a high challenge on the treatment of sepsis. Recently, deep reinforcement learning (DRL) achieves encouraging results in discovering an adaptive individual treatment regime for sepsis. However, traditional deep reinforcement learning models have limited ability to feature learning for high-dimensional heterogeneous data, thus to limit its effectiveness in learning dynamic treatment policy for sepsis. In this work, we present a deep reinforcement computation model to deduce dynamic treatment strategy for septic patients. The presented model combines the stacked tensor auto-encoders with Q-learning to improve DRL for high-dimensional heterogenous data learning. Furthermore, we present a training approach to update the parameters of the deep reinforcement computation model using the experience replay strategy. Finally, we verify the treatment policies learned by the presented model on a simulated set of patients by comparing with the policies given by sepsis specialists and learned by deep reinforcement learning models. The results justify that the presented model could learn the considerable level of sepsis experts in dynamic treatment policies and thus reduce mortality rate significantly on simulated patients. Therefore, the presented model is promising to contribute to smart medicine by providing the computer-aided dynamic treatment regimes for sepsis in intensive care units.

Emotion recognition is an important component of human-computer interaction (HCI) systems. However, current emotion recognition methods have some drawbacks such as inconsistency in brain network size, lack of effective mining of features in different dimensions. In this paper, we propose a multidimensional feature extraction method based on MSTBN and EEMD-WPT for emotion recognition. Firstly, the wavelet packet transform (WPT) is utilized to decompose the pre-processed electroencephalography (EEG) signals into four frequency bands ($\theta,\alpha,\beta$, and $\gamma$), and phase locking value (PLV) is used to construct multi-band connectivity matrix. Secondly, to remove redundant information, the minimum spanning tree based brain network (MSTBN) is established and MSTBN features are extracted including global features and local features. Thirdly, ensemble empirical mode decomposition (EEMD) and WPT (EEMD-WPT) are applied to EEG signals for a more refined decomposition of modes and bands. Then, the modified multi-scale sample entropy (MMSE) and fractal dimension (FD) are extracted to capture the neural activity processes in the brain. Finally, the MSTBN features are fused with the nonlinear features MMSE and FD, which are input into random forest (RF) to identify emotions. Experimental results on DEAP dataset indicate that the accuracy is 87.24% and 89.84% for valance and arousal. Experimental analysis reveals that MSTBN of negative emotions is more divergent and emotional information is transmitted more rapidly in the brain. Women are more susceptible to emotional perception than men. The proposed multidimensional feature extraction method has potential to be applied to HCI systems.

As one important technique of fuzzy clustering in data mining and pattern recognition, the possibilistic c-means algorithm (PCM) has been widely used in image analysis and knowledge discovery. However, it is difficult for PCM to produce a good result for clustering big data, especially for heterogenous data, since it is initially designed for only small structured dataset. To tackle this problem, the paper proposes a high-order PCM algorithm (HOPCM) for big data clustering by optimizing the objective function in the tensor space. Further, we design a distributed HOPCM method based on MapReduce for very large amounts of heterogeneous data. Finally, we devise a privacy-preserving HOPCM algorithm (PPHOPCM) to protect the private data on cloud by applying the BGV encryption scheme to HOPCM, In PPHOPCM, the functions for updating the membership matrix and clustering centers are approximated as polynomial functions to support the secure computing of the BGV scheme. Experimental results indicate that PPHOPCM can effectively cluster a large number of heterogeneous data using cloud computing without disclosure of private data.

Smart healthcare systems are generating a large scale of heterogenous high-dimensional data with complex relationships. It is hard for current methods to analyze such high-dimensional healthcare data. Specifically, the traditional data reduction methods can not keep the correlation among different modalities of data objects, while the latest methods based on tensor singular value decomposition are not effective for data reduction, although they can keep the correlation. This article presents a tensor train-tensor singular value decomposition (TT-TSVD) algorithm for data reduction. Particularly, the presented algorithm balances the correlation-preservation ability of modalities and data reduction ability by combining the advantages of the train structure of the tensor train decomposition and the association relationship between the tensor singular value decomposition retention mode. Extensive experiments are conducted on the convolutional neural network and the results clearly show that the presented algorithm performs effectively for data reduction with a low-loss classification accuracy; what is more, classification accuracy on medical image dataset has been improved a little.

Summary Cloud computing is significantly contributing to the development of smart Chinese medicine. The diagnosis and treatment of spleen and stomach diseases has been arousing great interest in smart Chinese medicine with cloud computing since many persons are suffering from spleen and stomach diseases. Currently, spleen and stomach diseases present some new characteristics with the dramatic changes in natural climate, social environment, and human living habits. Recently, deep learning, together with cloud computing techniques, has successfully used in medical image analysis and therefore it is the most promising model for diagnosing spleen and stomach disease in smart Chinese medicine. In this paper, we present a survey on deep learning models in medical image analysis for computer‐aided diagnosis in modern medicine. Afterwards, we summarize the syndrome types of spleen and stomach diseases and furthermore analyze the causes and pathogenesis for each syndrome. Finally, we discuss the open challenges and research directions of deep learning models applicable to the computer‐aided diagnosis of spleen and stomach diseases, which is expected to contribute to the development of smart Chinese medicine with cloud computing.

Video surveillance techniques like scene segmentation are playing an increasingly important role in multimedia Internet-of-Things (IoT) systems. However, existing deep learning-based methods face challenges in both accuracy and memory when deployed on edge computing devices with limited computing resources. To address these challenges, a tensor-train video scene segmentation scheme that compares the local background information in regional scene boundary boxes in adjacent frames is proposed. Compared to the existing methods, the proposed scheme can achieve competitive performance in both segmentation accuracy and parameter compression rate. In detail, first, an improved faster region convolutional neural network (faster-RCNN) model is proposed to recognize and generate a large number of region boxes with foreground and background to achieve boundary boxes. Then, the foreground boxes with sparse objects are removed and the rest are considered as optional background boxes used to measure the similarity between two adjacent frames. Second, to accelerate the training efficiency and reduce memory size, a general and efficient training way using tensor-train decomposition to factor the input-to-hidden weight matrix is proposed. Finally, experiments are conducted to evaluate the performance of the proposed scheme in terms of accuracy and model compression. Our results demonstrate that the proposed model can improve the training efficiency and save the memory space for the deep computation model with good accuracy. This work opens the potential for the use of artificial intelligence methods in edge computing devices for multimedia IoT systems.

Smart Chinese medicine has emerged to contribute to the evolution of healthcare and medical services by applying machine learning together with advanced computing techniques like cloud computing to computer-aided diagnosis and treatment in the health engineering and informatics. Specifically, smart Chinese medicine is considered to have the potential to treat difficult and complicated diseases such as diabetes and cancers. Unfortunately, smart Chinese medicine has made very limited progress in the past few years. In this paper, we present a unified smart Chinese medicine framework based on the edge-cloud computing system. The objective of the framework is to achieve computer-aided syndrome differentiation and prescription recommendation, and thus to provide pervasive, personalized, and patient-centralized services in healthcare and medicine. To accomplish this objective, we integrate deep learning and deep reinforcement learning into the traditional Chinese medicine. Furthermore, we propose a multi-modal deep computation model for syndrome recognition that is a crucial part of syndrome differentiation. Finally, we conduct experiments to validate the proposed model by comparing with the staked auto-encoder and multi-modal deep learning model for syndrome recognition of hypertension and cold.

Medical Internet of Things are generating a big volume of data to enable smart medicine that tries to offer computer-aided medical and healthcare services with artificial intelligence techniques like deep learning and clustering. However, it is a challenging issue for deep learning, and clustering algorithms to analyze large medical data because of their high computational complexity, thus hindering the progress of smart medicine. In this article, we present an incremental high-order possibilistic c-means algorithm (IHoPCM) on a cloud-edge computing system to achieve medical data coclustering of multiple hospitals in different locations. Specifically, each hospital employs the deep computation model to learn a feature tensor of each medical data object on the local edge computing system, and then uploads the feature tensors to the cloud computing platform. The high-order possibilistic c-means algorithm is performed on the cloud system for medical data clustering on uploaded feature tensors. Once the new medical data feature tensors are arriving at the cloud computing platform, the incremental high-order possibilistic c-means algorithm (IHoPCM) is performed on the combination of the new feature tensors and the previous clustering centers to obtain clustering results for the feature tensors received to date. In this way, repeated clustering on the previous feature tensors is avoided to improve the clustering efficiency. In the experiments, we compare different algorithms on two medical datasets regarding clustering accuracy and clustering efficiency. Results show that the presented IHoPCM method achieves great improvements over the compared algorithms in clustering accuracy and efficiency.

Supervised machine learning algorithms, especially classification algorithms, have been widely used in data analysis of industrial big data. Among them, the support vector machine (SVM) has achieved great success in the binary classification of some areas like image processing, computer vision, and pattern recognition. However, an SVM cannot achieve the desirable classification results for heterogeneous and high-dimensional data generated from thousands of industrial sensors in physical environments, because the traditional vector-based and feature-aligned SVM algorithm may result in loss of structural information and rich context information. Although the support tensor machine (STM) has extended the traditional vector-based SVM to tensor space, it fails to deal with multiple classification problems. Therefore, designing a general multiple classification algorithm for heterogeneous and high-dimensional data is a challenging but promising topic. To achieve this goal, this article presents a support multimode tensor machine (SMTM) algorithm by applying the multimode product to generalize the formulation of the STM. Furthermore, this article presents an efficient algorithm to train the parameters. Experiments conducted on various data sets validate the better performance of the SMTM over other algorithms in the multiple classification and imply the potential of the proposed model for multiple classification on industrial big data.

Big data feature learning is a crucial issue for the service management for Internet of Things. However, big data collected from Internet of Things is of dynamic nature at a high speed, which poses an important challenge on wireless big data learning models, especially the deep computation model. In this paper, an incremental deep computation model is proposed for wireless big data feature learning in Internet of Things. First, two incremental tensor auto-encoders (ITAE) are developed by devising two incremental learning algorithms, namely parameter-based incremental learning algorithm (PI-TAE) and structure-based incremental learning algorithm (SI-TAE), when new wireless samples are available. PI-TAE only updates the network parameters while SI-TAE simultaneously adjusts the structure and updates the parameters to adapt to the new arriving wireless big data. Furthermore, an incremental deep computation model is constructed by stacking several ITAEs. Experiments are conducted to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two representative incremental learning algorithms, i.e., OANN and PIE. Results demonstrate that the presented model can modify the network in an incremental manner for new arriving data learning efficiently with preserving the prior knowledge for the previous data learning, proving its potential for dynamic wireless big data learning in Internet of Things.

In this article, a high-order possibilistic c-means algorithm (HOPCM) based on the double-layer deep computation model (DCM) is proposed for big data clustering. Specifically, an asymmetric tensor autoencoder is presented to efficiently train the double-layer DCM for big data feature learning. Furthermore, an edge-cloud computing system is developed to improve the clustering efficiency. In the edge-cloud system, the computation-intensive tasks including the parameters' training and clustering are offloaded to the cloud while the task of feature learning is performed at the edge of network. Finally, we conduct extensive experiments to evaluate the performance of the presented algorithm by comparing it with other two representative big data clustering algorithms, i.e., the standard HOPCM and the HOPCM based on deep learning. Results demonstrate that the presented algorithm achieves higher accuracy than the two compared algorithms and furthermore the clustering efficiency are significantly improved by the developed edge-cloud computing system.

Energy efficiency and autonomous power management are extremely important for mobile-edge computing. Reducing energy consumption of a number of applications running concurrently in mobile devices while maintaining performance poses a challenge to energy optimization due to the limited capacity of the embedded battery. To extend battery life and offer a long-lasting working energy, dynamic voltage and frequency scaling (DVFS) has been widely used in mobile devices for energy consumption minimization. However, most conventional DVFS techniques scale operating frequency based on static policies, and thus, they are difficult to be adapted to systems of varied conditions. In order to improve adaptivity, in this article, we proposed a Double-Q power management approach to scale operating frequency based on learning. The Double-Q method stores two Q tables and two corresponding update functions. In each decision point, either of Q tables is randomly chosen and updated, while the other is used for the measurement. This mechanism reduces the overestimation in Q values, consequently enhancing the accurateness of frequency predictions. To evaluate the effectiveness of our proposed approach, a Double-Q governor is implemented in the Linux kernel. Our approach is computationally light, and experimental results indicate that it achieves at least 5-18% total energy saving compared to on-demand and conservative governors, as well as Q learning-based method.

With the ongoing development of sensor devices and network techniques, big data are being generated from the cyber-physical systems. Because of sensor equipment occasional failure and network transmission unreliability, a large number of low-quality data, such as noisy data and incomplete data, is collected from the cyber-physical systems. Low-quality data pose a remarkable challenge on deep learning models for big data feature learning. As a novel deep learning model, the deep computation model achieves superior performance for big data feature learning. However, it is difficult for the deep computation model to learn dependable features for low-quality data, since it uses the nonlinear function as the encoder. In this article, a dependable deep computation model is proposed for feature learning on low-quality big data in cyber-physical systems. Specially, a regularity is added into the objective function of the deep computation model to obtain reliable features in the intermediate-level representation space. Furthermore, a learning algorithm based on the back-propagation strategy is devised to train the parameters of the proposed model. Finally, experiments are conducted on three representative datasets and a real dataset to evaluate the effectiveness of the dependable deep computation model for low-quality big data feature learning. Results show that the proposed model achieves a remarkable result for the tasks of classification, restoration, and prediction, proving the potential of this work for practical applications in cyber-physical systems.

The deep convolutional computation model (DCCM) enabled remarkable progress in feature learning of industrial big data in Internet of Things. However, as a typical static deep learning model, it is difficult to learn features for incremental industrial big data. To solve this problem, we propose an incremental DCCM by developing two incremental algorithms, i.e., parameter-incremental algorithm and structure-incremental algorithm. The parameter-incremental algorithm aims to incrementally train the fully connected layers together with fine tuning for incorporating the new knowledge into the prior one. Then, the structure-incremental algorithm is used to transfer the previous knowledge by introducing an updating rule of the tensor convolutional, pooling, and fully connected layers. Furthermore, the dropout strategy is extended into the tensor fully connected layer to improve the robustness of the proposed model. Finally, extensive experiments are carried out on the representative datasets including CIFRA and CUAVE to justify the proposed model in terms of adaption, preservation, and convergence efficiency.

Deep computation, as an advanced machine learning model, has achieved the state-of-the-art performance for feature learning on big data in industrial Internet of Things (IoT). However, the current deep computation model usually suffers from overfitting due to the lack of public available labeled training samples, limiting its performance for big data feature learning. Motivated by the idea of active learning, an adaptive dropout deep computation model (ADDCM) with crowdsourcing to cloud is proposed for industrial IoT big data feature learning in this paper. First, a distribution function is designed to set the dropout rate for each hidden layer to prevent overfitting for the deep computation model. Furthermore, the outsourcing selection algorithm based on the maximum entropy is employed to choose appropriate samples from the training set to crowdsource on the cloud platform. Finally, an improved supervised learning from multiple experts scheme is presented to aggregate answers given by human workers and to update the parameters of the ADDCM simultaneously. Extensive experiments are conducted to evaluate the performance of the presented model by comparing with the dropout deep computation model and other state-of-the-art crowdsourcing algorithms. The results demonstrate that the proposed model can prevent overfitting effectively and aggregate the labeled samples to train the parameters of the deep computation model with crowdsouring for industrial IoT big data feature learning.

Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average 2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.

Energy saving is a critical and challenging issue for real-time systems in embedded devices because of their limited energy supply. To reduce the energy consumption, a hybrid dynamic voltage and frequency scaling (DVFS) scheduling based on Q-learning (QL-HDS) was proposed by combining energy-efficient DVFS techniques. However, QL-HDS discretizes the system state parameters with a certain step size, resulting in a poor distinction of the system states. More importantly, it is difficult for QL-HDS to learn a system for various task sets with a Q-table and limited training sets. In this paper, an energy-efficient scheduling scheme based on deep Q-learning model is proposed for periodic tasks in real-time systems (DQL-EES). Specially, a deep Q-learning model is designed by combining a stacked auto-encoder and a Q-learning model. In the deep Q-learning model, the stacked auto-encoder is used to replace the Q-function for learning the Q-value of each DVFS technology for any system state. Furthermore, a training strategy is devised to learn the parameters of the deep Q-learning model based on the experience replay scheme. Finally, the performance of the proposed scheme is evaluated by comparison with QL-HDS on different simulation task sets. Results demonstrated that the proposed algorithm can save average$4.2\%$energy than QL-HDS.

Recent years have witness a considerable advance of Internet of Things with the tremendous progress of communication theories and sensing technologies. A large number of data, usually referring to big data, have been generated from Internet of Things. In this paper, we present a double-projection deep computation model (DPDCM) for big data feature learning, which projects the raw input into two separate subspaces in the hidden layers to learn interacted features of big data by replacing the hidden layers of the conventional deep computation model (DCM) with double-projection layers. Furthermore, we devise a learning algorithm to train the DPDCM. Cloud computing is used to improve the training efficiency of the learning algorithm by crowdsourcing the data on cloud. To protect the private data, a privacy-preserving DPDCM (PPDPDCM) is proposed based on the BGV encryption scheme. Finally, experiments are carried on Animal-20 and NUS-WIDE-14 to estimate the performance of DPDCM and PPDPDCM by comparing with DCM. Results demonstrate that DPDCM achieves a higher classification accuracy than DCM. More importantly, PPDPDCM can effectively improve the efficiency for training parameters, proving its potential for big data feature learning.

Currently, a large number of industrial data, usually referred to big data, are collected from Internet of Things (IoT). Big data are typically heterogeneous, i.e., each object in big datasets is multimodal, posing a challenging issue on the convolutional neural network (CNN) that is one of the most representative deep learning models. In this paper, a deep convolutional computation model (DCCM) is proposed to learn hierarchical features of big data by using the tensor representation model to extend the CNN from the vector space to the tensor space. To make full use of the local features and topologies contained in the big data, a tensor convolution operation is defined to prevent overfitting and improve the training efficiency. Furthermore, a high-order backpropagation algorithm is proposed to train the parameters of the deep convolutional computational model in the high-order space. Finally, experiments on three datasets, i.e., CUAVE, SNAE2, and STL-10 are carried out to verify the performance of the DCCM. Experimental results show that the deep convolutional computation model can give higher classification accuracy than the deep computation model or the multimodal model for big data in IoT.

The deep computation model has been proved to be effective for big data hierarchical feature and representation learning in the tensor space. However, it requires expensively computational resources including high-performance computing units and large memory to train a deep computation model with a large number of parameters, limiting its effectiveness and efficiency for industry informatics big data feature learning. In this paper, a tensor-train deep computation model is presented for industry informatics big data feature learning. Specially, the tensor-train network is used to compress the parameters significantly by converting the dense weight tensors into the tensor-train format. Furthermore, a learning algorithm is implemented based on gradient descent and back-propagation to train the parameters of the presented tensor-train deep computation model. Extensive experiments are carried on STL-10, CUAVE, and SNAE2 to evaluate the presented model in terms of the approximation error, classification accuracy drop, parameters reduction, and speedup. Results demonstrate that the presented model can improve the training efficiency and save the memory space greatly for the deep computation model with small accuracy drops, proving its potential for industry informatics big data feature learning.

Deep learning, as the most important architecture of current computational intelligence, achieves super performance to predict the cloud workload for industry informatics. However, it is a nontrivial task to train a deep learning model efficiently since the deep learning model often includes a great number of parameters. In this paper, an efficient deep learning model based on the canonical polyadic decomposition is proposed to predict the cloud workload for industry informatics. In the proposed model, the parameters are compressed significantly by converting the weight matrices to the canonical polyadic format. Furthermore, an efficient learning algorithm is designed to train the parameters. Finally, the proposed efficient deep learning model is applied to the workload prediction of virtual machines on cloud. Experiments are conducted on the datasets collected from PlanetLab to validate the performance of the proposed model by comparing with other machine-learning-based approaches for workload prediction of virtual machines. Results indicate that the proposed model achieves a higher training efficiency and workload prediction accuracy than state-of-the-art machine-learning-based approaches, proving the potential of the proposed model to provide predictive services for industry informatics.

Deep computation models achieve super performance for big data feature learning. However, training a deep computation model poses a significant challenge since a deep computation model typically involves a large number of parameters. Specially, it needs a high-performance computing server with a large-scale memory and a powerful computing unit to train a deep computation model, making it difficult to increase the size of a deep computation model further for big data feature learning on low-end devices such as conventional desktops and portable CPUs. In this paper, we propose an improved deep computation model based on the canonical polyadic decomposition scheme to compress the parameters and to improve the training efficiency. Furthermore, we devise a learning algorithm based on the back-propagation strategy to train the parameters of the proposed model. The learning algorithm can be directly performed on the compressed parameters to improve the training efficiency. Finally, we carry on the experiments on three representative datasets, i.e., CUAVE, SNAE2, and STL-10, to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two improved deep computation models based on the Tucker decomposition and the tensor-train network. Results demonstrate that the proposed model can compress parameters greatly and improve the training efficiency significantly with a low classification accuracy drop.

With the rapid advances of sensing technologies and wireless communications, large amounts of dynamic data pertaining to industrial production are being collected from many sensor nodes deployed in the industrial Internet of Things. Analyzing those data effectively can help to improve the industrial services and mitigate the system unprepared breakdowns. As an important technique of data analysis, clustering attempts to find the underlying pattern structures embedded in unlabeled information. Unfortunately, most of the current clustering techniques that could only deal with static data become infeasible to cluster a significant volume of data in the dynamic industrial applications. To tackle this problem, an incremental clustering algorithm by fast finding and searching of density peaks based on k-mediods is proposed in this paper. In the proposed algorithm, two cluster operations, namely cluster creating and cluster merging, are defined to integrate the current pattern into the previous one for the final clustering result, and k-mediods is employed to modify the clustering centers according to the new arriving objects. Finally, experiments are conducted to validate the proposed scheme on three popular UCI datasets and two real datasets collected from industrial Internet of Things in terms of clustering accuracy and computational time.

Recently, the deep computation model, as a tensor deep learning model, has achieved super performance for multimedia feature learning. However, the conventional deep computation model involves a large number of parameters. Typically, training a deep computation model with millions of parameters needs high-performance servers with large-scale memory and powerful computing units, limiting the growth of the model size for multimedia feature learning on common devices such as portable CPUs and conventional desktops. To tackle this problem, this article proposes a Tucker deep computation model by using the Tucker decomposition to compress the weight tensors in the full-connected layers for multimedia feature learning. Furthermore, a learning algorithm based on the back-propagation strategy is devised to train the parameters of the Tucker deep computation model. Finally, the performance of the Tucker deep computation model is evaluated by comparing with the conventional deep computation model on two representative multimedia datasets, that is, CUAVE and SNAE2, in terms of accuracy drop, parameter reduction, and speedup in the experiments. Results imply that the Tucker deep computation model can achieve a large-parameter reduction and speedup with a small accuracy drop for multimedia feature learning.

To improve the efficiency of big data feature learning, the paper proposes a privacy preserving deep computation model by offloading the expensive operations to the cloud. Privacy concerns become evident because there are a large number of private data by various applications in the smart city, such as sensitive data of governments or proprietary information of enterprises. To protect the private data, the proposed model uses the BGV encryption scheme to encrypt the private data and employs cloud servers to perform the high-order back-propagation algorithm on the encrypted data efficiently for deep computation model training. Furthermore, the proposed scheme approximates the Sigmoid function as a polynomial function to support the secure computation of the activation function with the BGV encryption. In our scheme, only the encryption operations and the decryption operations are performed by the client while all the computation tasks are performed on the cloud. Experimental results show that our scheme is improved by approximately 2.5 times in the training efficiency compared to the conventional deep computation model without disclosing the private data using the cloud computing including ten nodes. More importantly, our scheme is highly scalable by employing more cloud servers, which is particularly suitable for big data.

Clustering is a commonly used technique for multimedia data analysis and management. In this article, we propose a high-order clustering algorithm by fast search and find of density peaks (HOCFS) by extending the traditional clustering scheme by fast search and find of density peaks (CFS) algorithm from the vector space to the tensor space for multimedia data clustering. Furthermore, we propose a privacy preserving HOCFS algorithm (PPHOCFS) which improves the efficiency of the HOCFS algorithm by using the cloud computing to perform most of the clustering operations. To protect the private data in the multimedia data sets during the clustering process on the cloud, the raw data is encrypted by the Brakerski-Gentry-Vaikun-tanathan (BGV) strategy before being uploaded to the cloud for performing the HOCFS clustering algorithm efficiently. In the proposed method, the client is required to only execute the encryption/decryption operations and the cloud servers are employed to perform all the computing operations. Finally, the performance of our scheme is evaluated on two representative multimedia data sets, namely NUS-WIDE and SNAE2, in terms of clustering accuracy, execution time, and speedup in the experiments. The results demonstrate that the proposed PPHOCFS scheme can save at least 40% running time compared with HOCFS, without disclosing the private data on the cloud, making our scheme securely suitable for multimedia big data clustering.

Deep learning has been successfully applied to feature learning in speech recognition, image classification and language processing. However, current deep learning models work in the vector space, resulting in the failure to learn features for big data since a vector cannot model the highly non-linear distribution of big data, especially heterogeneous data. This paper proposes a deep computation model for feature learning on big data, which uses a tensor to model the complex correlations of heterogeneous data. To fully learn the underlying data distribution, the proposed model uses the tensor distance as the average sum-of-squares error term of the reconstruction error in the output layer. To train the parameters of the proposed model, the paper designs a high-order back-propagation algorithm (HBP) by extending the conventional back-propagation algorithm from the vector space to the high-order tensor space. To evaluate the performance of the proposed model, we carried out the experiments on four representative datasets by comparison with stacking auto-encoders and multimodal deep learning models. Experimental results clearly demonstrate that the proposed model is efficient to perform feature learning when evaluated using the STL-10, CUAVE, SANE and INEX datasets.