Transformer-based semantic segmentation methods have demonstrated outstanding performance by leveraging global self-attention to effectively capture long-range dependence. However, there still exist two issues in existing works: (1) Most of them utilize the full-rank weight matrix to support the self-attention mechanism and feed-forward network in modelling long-range dependence between patches/pixels, resulting in a high computational cost during both training and inference. (2) Most of them ignore information interactions between high-level semantics and low-level structures during the image resolution recovery, which leads to the performance degradation in segmenting objects with complex boundaries. To tackle these challenges, a lightweight Kolmogorov-Arnold Transformer model (LKAFormer) is proposed for the image semantic segmentation, containing a two-stream lightweight Transformer encoder and a graph feature pyramid aggregation KAN-decoder. The former constructs a hierarchical feature cross-scale fusion pipeline to obtain sufficient semantics containing comprehensive multi-scale information via setting coarse-grained and fine-grained streams with different-size patches of images. In that pipeline, feature lightweight focusing modules model complex and long-range dependence across patches/pixels to refine image semantics with less computational costs by lightweight multi-head self-attention and lightweight feed-forward network designs. The latter leverages the learnable nonlinear transformation mechanism of the Kolmogorov-Arnold Transformer architecture to adaptively capture spatial structure dependence of distinct sub-regions of images. And then, it jointly performs the intra-scale graph fusion and cross-scale graph fusion during the image resolution recovery to enhance information interactions between high-level semantics and low-level structures, which achieves the robust boundary localization and texture refinement of segmentation objects. Finally, plentiful experiments are conducted on three challenging datasets, and the results show LKAFormer sets a new baseline in the image segmentation task in comparison with 11 methods.

Accurate continuous blood glucose prediction is an effective and direct method for treating type 2 diabetes mellitus. However, current methods are commonly single-domain single-scale blood glucose prediction models. That is, they only learn time correlations within constant time steps of continuous blood glucose data, to mine fluctuation patterns, which limits the model effectiveness and robustness. To this end, a novel dynamic-static feature fusion with multi-scale attention method (DSfusion) is proposed for accurately predicting continuous blood glucose. Specifically, DSfusion designs the cross-domain complementary augmentation via Fourier cycle transformation for learning inherent frequency features. Then, DSfusion devises the multi-scale dependence aggregation based on the attention mechanism with various scale receptive fields to capture time correlations and feature correlations across different time steps. Meanwhile, DSfusion utilizes the static feature enhancement to integrate multi-domain information, i.e., static and dynamic features, into a unified prediction architecture, which greatly boosts the model performance. Finally, comprehensive experiments are performed on the real-world blood glucose dataset, and the results confirm the excellence of DSfusion.

Multi-view clustering aims to divide samples into several clusters, by mining and utilizing the consistency and complementarity of multi-view data. Recent years, numerous deep contrastive multi-view clustering methods have been proposed to address the false negative issue by using self-supervised information. However, the quality of these self-supervised information was rarely taken into consideration, and using these information without discrimination can compromise training, leading to sub optimal performance. To tackle this issue, we propose Hard Sample Aware Robust Contrastive Learning for Multi-View Clustering(HearMVC). Concretely, we use self-supervised information and similarity to determine hard samples. The model focuses on these hard samples by assigning higher weights to enhance discriminative capability. Moreover, we utilize the confidence of self-supervised cluster assignment as weights, to strengthen the learning to confident samples and weaken the influence of unconfident samples. By simultaneously considering the weighting of hardness and confidence, our method can achieve best robustness and strongest discriminative capability. Extensive experiments on public datesets verify the effectiveness of our method.

Deep multi-view clustering has attracted increasing attention in the pattern mining of data. However, most of them perform self-learning mechanisms in a single space, ignoring the fruitful structural information hidden in different-level feature spaces. Meanwhile, they conduct the reconstruction constraint to learn generalized representations of samples, failing to explore the discriminative ability of complementary and consistent information. To address the challenges, a multi-granularity invariant structure clustering scheme (MASTER) is proposed to define a bottom-up process that extracts multi-level information in sample, neighborhood, and category granularities from low-level, high-level, and semantics feature space, respectively. Specifically, it leverages the self-learning reconstruction with information-theoretic overclustering to capture invariant sample structure in the low-level feature space. Then, it models data diffusion of the clustering process in the reliable neighborhood to capture invariant local structure in the high-level feature space. Meanwhile, it defines dual divergences induced by the space geometry to capture invariant global structure in the semantics space. Finally, extensive experiments on 8 real-world datasets show that MASTER achieves state-of-the-art performance compared to 11 baselines.

Internet of Things (IoT) in extreme environments inevitably produces incomplete multi-view data, presenting challenges to the existing data analysis methods. Although incomplete multi-view clustering methods have the potential to mine patterns of incomplete IoT data, they are still confronted with two challenges. 1) They ignore shifts of semantics caused by missing data in aggregating consistent and complementary information of incomplete data, degrading the robustness of models in pattern mining. 2) Most of them rely on the instances with complete views as pairwise supervision to capture correlations among views, failing to mine inherent patterns of data in the extreme view missing scenario where multi-view instances are only with an available view. To this end, a deep incomplete multi-view clustering network (DIMC) is proposed via defining dual consistencies within the information bottleneck framework to mine accurate patterns of incomplete data. Specifically, an unsupervised multi-view information bottleneck (MIB) is formulated to model dependencies of data, which remedies shifts of semantics via within-view intrinsic knowledge learning, consistent semantics sharing, and consistent structure aligning. Meanwhile, dual consistencies are designed to implement MIB, which builds invariant transformations to mine correlations between views without the help of complete instances. Finally, extensive experiments on four benchmark incomplete datasets demonstrate the superiority of DIMC. Especially, DIMC surpasses the state-of-the-art methods by 0.2048 in accuracy under extreme view missing scenarios.

Multiview clustering has attracted significant attention in various fields, due to the superiority in mining patterns of multiview data. However, previous methods are still confronted with two challenges. First, they do not fully consider the semantic invariance of multiview data in aggregating complementary information, degrading semantic robustness of fusion representations. Second, they rely on predefined clustering strategies to mine patterns, lacking adequate explorations of data structures. To address the challenges, deep multiview adaptive clustering via semantic invariance (DMAC-SI) is proposed, which learns an adaptive clustering strategy on semantics-robust fusion representations to fully explore structures in mining patterns. Specifically, a mirror fusion architecture is devised to explore interview invariance and intrainstance invariance hidden in multiview data, which captures invariant semantics of complementary information to learn semantics-robust fusion representations. Then, a Markov decision process of multiview data partitions is proposed within the reinforcement learning framework, which learns an adaptive clustering strategy on semantics-robust fusion representations to guarantee the structure explorations in mining patterns. The two components seamlessly collaborate in an end-to-end manner to accurately partition multiview data. Finally, extensive experiment results on five benchmark datasets demonstrate that DMAC-SI outperforms the state-of-the-art methods.

With the wide deployment of the Internet of Things (IoT), large volumes of incomplete multiview data that violates data integrity is generated by various applications, which inevitably produces negative impacts on the quality of service of IoT systems. Incomplete multiview clustering (IMC), as an essential technique of data processing, has the potential for mining patterns of incomplete IoT data. However, previous methods utilize notion-strong distances that can only measure differences between distributions at the overlap of data manifolds in fusing complementary information of data for pattern mining. They may suffer from biased estimation and information loss in capturing intrinsic structures of incomplete multiview data. To address these challenges, a semidiscrete multiview optimal transport (SD-MOT) is defined for IMC, which utilizes distances with weak notions to capture intrinsic structures of incomplete multiview data. Specifically, IMC is recast as an equivalent optimal transport between continuous incomplete multiview data and discrete clustering centroids, to avoid the strict assumption on overlap between manifolds in pattern mining. Then, SD-MOT is instantiated as a deep incomplete contrastive clustering network to remedy biased estimation and information loss on intrinsic structures of incomplete multiview data. Afterwards, a variational solution to SD-MOT is derived to effectively train the network parameters for pattern mining. Finally, extensive experiments on four representative incomplete multiview datasets verify the superiority of SD-MOT in comparison with nine baseline methods.

Chronic Obstructive Pulmonary Disease (COPD) severely affects people’s health. With this in mind, we propose a novel Multi-Stage Composite Ensemble Learning Framework (MSCELF) that can diagnose COPD without utilising pulmonary function tests data. Our method explores 12 features from the patients’ baseline data, medical history, blood tests, and arterial blood gas analysis. In the first stage of our approach, three different ensemble learning methods are employed. The second stage involves the utilization of two machine learning methods. Finally, the Murphy’s method is integrated in the final stage to combine the outputs, with weights being assigned based on their information quantity and credibility. We evaluate our method on a clinical dataset of 329 patients and show that it outperforms existing methods in terms of accuracy, AUC, sensitivity, specificity, PPV, NPV, and F1 score, which are 0.7980, 0.8082, 0.8551, 0.6835, 0.8570, 0.6531, 0.8560.

Medical image segmentation plays a vital role in clinical diagnosis. However, previous methods cannot handle intrinsic ambiguities in extracting deep semantics of medical images. Moreover, they neglect fruitful semantic information in segmentation maps. To address the challenges, a latent adversarial Cauchy-Schwarz autoencoder is proposed, which defines image segmentation as a cross-modal translation task from medical images to segmentation maps. Specifically, a probabilistic graph model is defined to fit the conditional distribution of the image translation between modalities, which leverages the Cauchy-Schwarz divergence to alleviate approximation errors caused by ambiguities in extracting deep semantics. Then, a novel numerical solution is derived to optimize the probabilistic graph model, which explores semantics in segmentation maps to facilitate the segmentation. Afterwards, a dual-flow architecture is proposed with an adversarial encoding-decoding paradigm to implement the numerical solution. Finally, extensive experiments in two medical scenarios illustrate that the proposed method achieves the state-of-the-art performance compared with nine baseline methods.

Nowadays, much research leverages the clustering to mine commercial patterns from data in enterprise systems. However, previous methods cannot fully consider local structures and global topology of data, which may cause the degradation of clustering performance. To address the challenges, a deep multimodal adversarial cycle-consistent network (DMACCN) is proposed to mine intrinsic patterns of data, which can capture the local structures from instance reconstructions and the global topology from adversarial games. Specifically, DMACCN is designed as an adversarial encoding-decoding architecture composed of the modality specific-encoder, the modality-common fusion network, the cycle-consistent modality-specific generator, and the modality-fusion discriminator, which can fully fuse complementary information of data. Then, an adversarial cycle-consistent loss is devised to guide the clustering pattern mining from complementary information of data, which can align semantics between modalities and capture clustering structures of instances. The two components collaborate in a seamless manner to capture accurate commercial patterns. Finally, extensive experimental results on four datasets show DMACCN greatly outperforms the comparison methods.

Deep clustering has attracted plentiful attention in various domains owning to the superior performance. However, the previous deep clustering methods are guided by pre-specified clustering strategies that lack sustained explorations of data structures, degrading recognition of intrinsic patterns hidden in data. To address this challenge, deep reinforcement clustering (DRC) is proposed to learn an adaptive partition policy for pattern mining, which can fully explore structure knowledge of data in an adaptive manner. DRC is defined as a Markov decision process of data partitions, which chooses the optimal cluster prototype for data via maximizing the cumulative reward in state transition of environment. To implement the definition, a Bernoulli action prototype is devised to capture decision distributions in the transition of states, where the heavy-tailed Cauchy distribution precisely measures the structure divergences of data. Furthermore, a reward maximizing policy is designed to guide sustained explorations of data structures, which ensures intra-cluster compactness and inter-cluster separation of data partitions. Finally, extensive experiments are conducted on eight benchmark datasets, and the results demonstrate that DRC outperforms the state-of-the-art baseline methods.

As one important technique of fuzzy clustering in data mining and pattern recognition, the possibilistic c-means algorithm (PCM) has been widely used in image analysis and knowledge discovery. However, it is difficult for PCM to produce a good result for clustering big data, especially for heterogenous data, since it is initially designed for only small structured dataset. To tackle this problem, the paper proposes a high-order PCM algorithm (HOPCM) for big data clustering by optimizing the objective function in the tensor space. Further, we design a distributed HOPCM method based on MapReduce for very large amounts of heterogeneous data. Finally, we devise a privacy-preserving HOPCM algorithm (PPHOPCM) to protect the private data on cloud by applying the BGV encryption scheme to HOPCM, In PPHOPCM, the functions for updating the membership matrix and clustering centers are approximated as polynomial functions to support the secure computing of the BGV scheme. Experimental results indicate that PPHOPCM can effectively cluster a large number of heterogeneous data using cloud computing without disclosure of private data.

Summary Cloud computing is significantly contributing to the development of smart Chinese medicine. The diagnosis and treatment of spleen and stomach diseases has been arousing great interest in smart Chinese medicine with cloud computing since many persons are suffering from spleen and stomach diseases. Currently, spleen and stomach diseases present some new characteristics with the dramatic changes in natural climate, social environment, and human living habits. Recently, deep learning, together with cloud computing techniques, has successfully used in medical image analysis and therefore it is the most promising model for diagnosing spleen and stomach disease in smart Chinese medicine. In this paper, we present a survey on deep learning models in medical image analysis for computer‐aided diagnosis in modern medicine. Afterwards, we summarize the syndrome types of spleen and stomach diseases and furthermore analyze the causes and pathogenesis for each syndrome. Finally, we discuss the open challenges and research directions of deep learning models applicable to the computer‐aided diagnosis of spleen and stomach diseases, which is expected to contribute to the development of smart Chinese medicine with cloud computing.

Smart Chinese medicine has emerged to contribute to the evolution of healthcare and medical services by applying machine learning together with advanced computing techniques like cloud computing to computer-aided diagnosis and treatment in the health engineering and informatics. Specifically, smart Chinese medicine is considered to have the potential to treat difficult and complicated diseases such as diabetes and cancers. Unfortunately, smart Chinese medicine has made very limited progress in the past few years. In this paper, we present a unified smart Chinese medicine framework based on the edge-cloud computing system. The objective of the framework is to achieve computer-aided syndrome differentiation and prescription recommendation, and thus to provide pervasive, personalized, and patient-centralized services in healthcare and medicine. To accomplish this objective, we integrate deep learning and deep reinforcement learning into the traditional Chinese medicine. Furthermore, we propose a multi-modal deep computation model for syndrome recognition that is a crucial part of syndrome differentiation. Finally, we conduct experiments to validate the proposed model by comparing with the staked auto-encoder and multi-modal deep learning model for syndrome recognition of hypertension and cold.

Generating pictures from text is an interesting, classic, and challenging task. Benefited from the development of generative adversarial networks (GAN), the generation quality of this task has been greatly improved. Many excellent cross modal GAN models have been put forward. These models add extensive layers and constraints to get impressive generation pictures. However, complexity and computation of existing cross modal GANs are too high to be deployed in mobile terminal. To solve this problem, this paper designs a compact cross modal GAN based on canonical polyadic decomposition. We replace an original convolution layer with three small convolution layers and use an autoencoder to stabilize and speed up training. The experimental results show that our model achieves 20% times of compression in both parameters and FLOPs without loss of quality on generated images.

With the wide deployments of heterogeneous networks, huge amounts of data with characteristics of high volume, high variety, high velocity, and high veracity are generated. These data, referred to multimodal big data, contain abundant intermodality and cross-modality information and pose vast challenges on traditional data fusion methods. In this review, we present some pioneering deep learning models to fuse these multimodal big data. With the increasing exploration of the multimodal big data, there are still some challenges to be addressed. Thus, this review presents a survey on deep learning for multimodal data fusion to provide readers, regardless of their original community, with the fundamentals of multimodal deep learning fusion method and to motivate new multimodal data fusion techniques of deep learning. Specifically, representative architectures that are widely used are summarized as fundamental to the understanding of multimodal deep learning. Then the current pioneering multimodal data fusion deep learning models are summarized. Finally, some challenges and future topics of multimodal data fusion deep learning models are described.

Big data feature learning is a crucial issue for the service management for Internet of Things. However, big data collected from Internet of Things is of dynamic nature at a high speed, which poses an important challenge on wireless big data learning models, especially the deep computation model. In this paper, an incremental deep computation model is proposed for wireless big data feature learning in Internet of Things. First, two incremental tensor auto-encoders (ITAE) are developed by devising two incremental learning algorithms, namely parameter-based incremental learning algorithm (PI-TAE) and structure-based incremental learning algorithm (SI-TAE), when new wireless samples are available. PI-TAE only updates the network parameters while SI-TAE simultaneously adjusts the structure and updates the parameters to adapt to the new arriving wireless big data. Furthermore, an incremental deep computation model is constructed by stacking several ITAEs. Experiments are conducted to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two representative incremental learning algorithms, i.e., OANN and PIE. Results demonstrate that the presented model can modify the network in an incremental manner for new arriving data learning efficiently with preserving the prior knowledge for the previous data learning, proving its potential for dynamic wireless big data learning in Internet of Things.

With the rapid industrialization and urbanization, pattern mining of soil contamination of heavy metals is attracting increasing attention to control soil contamination. However, the correlation over various heavy metals and the high-dimension representation of heavy metal data pose vast challenges on the accurate mining of patterns over heavy metals of soil contamination. To solve those challenges, a multiview Gaussian mixture model is proposed in this paper, to naturally capture complicated relationships over multiviews on the basis of deep fusion features of data. Specifically, a deep fusion feature architecture containing modality-specific and modality-common stacked autoencoders is designed to distill fusion representations from the information of all views. Then, the Gaussian mixture model is extended on the fusion representations to naturally recognize the accurate patterns of the intra- and inter-views. Finally, extensive experiments are conducted on the representative datasets to evaluate the performance of the multiview Gaussian mixture model. Results show the outperformance of the proposed methods.

With the ongoing development of sensor devices and network techniques, big data are being generated from the cyber-physical systems. Because of sensor equipment occasional failure and network transmission unreliability, a large number of low-quality data, such as noisy data and incomplete data, is collected from the cyber-physical systems. Low-quality data pose a remarkable challenge on deep learning models for big data feature learning. As a novel deep learning model, the deep computation model achieves superior performance for big data feature learning. However, it is difficult for the deep computation model to learn dependable features for low-quality data, since it uses the nonlinear function as the encoder. In this article, a dependable deep computation model is proposed for feature learning on low-quality big data in cyber-physical systems. Specially, a regularity is added into the objective function of the deep computation model to obtain reliable features in the intermediate-level representation space. Furthermore, a learning algorithm based on the back-propagation strategy is devised to train the parameters of the proposed model. Finally, experiments are conducted on three representative datasets and a real dataset to evaluate the effectiveness of the dependable deep computation model for low-quality big data feature learning. Results show that the proposed model achieves a remarkable result for the tasks of classification, restoration, and prediction, proving the potential of this work for practical applications in cyber-physical systems.

The deep convolutional computation model (DCCM) enabled remarkable progress in feature learning of industrial big data in Internet of Things. However, as a typical static deep learning model, it is difficult to learn features for incremental industrial big data. To solve this problem, we propose an incremental DCCM by developing two incremental algorithms, i.e., parameter-incremental algorithm and structure-incremental algorithm. The parameter-incremental algorithm aims to incrementally train the fully connected layers together with fine tuning for incorporating the new knowledge into the prior one. Then, the structure-incremental algorithm is used to transfer the previous knowledge by introducing an updating rule of the tensor convolutional, pooling, and fully connected layers. Furthermore, the dropout strategy is extended into the tensor fully connected layer to improve the robustness of the proposed model. Finally, extensive experiments are carried out on the representative datasets including CIFRA and CUAVE to justify the proposed model in terms of adaption, preservation, and convergence efficiency.

Deep computation, as an advanced machine learning model, has achieved the state-of-the-art performance for feature learning on big data in industrial Internet of Things (IoT). However, the current deep computation model usually suffers from overfitting due to the lack of public available labeled training samples, limiting its performance for big data feature learning. Motivated by the idea of active learning, an adaptive dropout deep computation model (ADDCM) with crowdsourcing to cloud is proposed for industrial IoT big data feature learning in this paper. First, a distribution function is designed to set the dropout rate for each hidden layer to prevent overfitting for the deep computation model. Furthermore, the outsourcing selection algorithm based on the maximum entropy is employed to choose appropriate samples from the training set to crowdsource on the cloud platform. Finally, an improved supervised learning from multiple experts scheme is presented to aggregate answers given by human workers and to update the parameters of the ADDCM simultaneously. Extensive experiments are conducted to evaluate the performance of the presented model by comparing with the dropout deep computation model and other state-of-the-art crowdsourcing algorithms. The results demonstrate that the proposed model can prevent overfitting effectively and aggregate the labeled samples to train the parameters of the deep computation model with crowdsouring for industrial IoT big data feature learning.

Reducing energy consumption is a vital and challenging problem for the edge computing devices since they are always energy-limited. To tackle this problem, a deep Q-learning model with multiple DVFS (dynamic voltage and frequency scaling) algorithms was proposed for energy-efficient scheduling (DQL-EES). However, DQL-EES is highly unstable when using a single stacked auto-encoder to approximate the Q-function. Additionally, it cannot distinguish the continuous system states well since it depends on a Q-table to generate the target values for training parameters. In this paper, a double deep Q-learning model is proposed for energy-efficient edge scheduling (DDQ-EES). Specially, the proposed double deep Q-learning model includes a generated network for producing the Q-value for each DVFS algorithm and a target network for producing the target Q-values to train the parameters. Furthermore, the rectified linear units (ReLU) function is used as the activation function in the double deep Q-learning model, instead of the Sigmoid function in QDL-EES, to avoid gradient vanishing. Finally, a learning algorithm based on experience replay is developed to train the parameters of the proposed model. The proposed model is compared with DQL-EES on EdgeCloudSim in terms of energy saving and training time. Results indicate that our proposed model can save average 2%-2.4% energy and achieve a higher training efficiency than QQL-EES, proving its potential for energy-efficient edge scheduling.

Energy saving is a critical and challenging issue for real-time systems in embedded devices because of their limited energy supply. To reduce the energy consumption, a hybrid dynamic voltage and frequency scaling (DVFS) scheduling based on Q-learning (QL-HDS) was proposed by combining energy-efficient DVFS techniques. However, QL-HDS discretizes the system state parameters with a certain step size, resulting in a poor distinction of the system states. More importantly, it is difficult for QL-HDS to learn a system for various task sets with a Q-table and limited training sets. In this paper, an energy-efficient scheduling scheme based on deep Q-learning model is proposed for periodic tasks in real-time systems (DQL-EES). Specially, a deep Q-learning model is designed by combining a stacked auto-encoder and a Q-learning model. In the deep Q-learning model, the stacked auto-encoder is used to replace the Q-function for learning the Q-value of each DVFS technology for any system state. Furthermore, a training strategy is devised to learn the parameters of the deep Q-learning model based on the experience replay scheme. Finally, the performance of the proposed scheme is evaluated by comparison with QL-HDS on different simulation task sets. Results demonstrated that the proposed algorithm can save average$4.2\%$energy than QL-HDS.

Recent years have witness a considerable advance of Internet of Things with the tremendous progress of communication theories and sensing technologies. A large number of data, usually referring to big data, have been generated from Internet of Things. In this paper, we present a double-projection deep computation model (DPDCM) for big data feature learning, which projects the raw input into two separate subspaces in the hidden layers to learn interacted features of big data by replacing the hidden layers of the conventional deep computation model (DCM) with double-projection layers. Furthermore, we devise a learning algorithm to train the DPDCM. Cloud computing is used to improve the training efficiency of the learning algorithm by crowdsourcing the data on cloud. To protect the private data, a privacy-preserving DPDCM (PPDPDCM) is proposed based on the BGV encryption scheme. Finally, experiments are carried on Animal-20 and NUS-WIDE-14 to estimate the performance of DPDCM and PPDPDCM by comparing with DCM. Results demonstrate that DPDCM achieves a higher classification accuracy than DCM. More importantly, PPDPDCM can effectively improve the efficiency for training parameters, proving its potential for big data feature learning.

Currently, a large number of industrial data, usually referred to big data, are collected from Internet of Things (IoT). Big data are typically heterogeneous, i.e., each object in big datasets is multimodal, posing a challenging issue on the convolutional neural network (CNN) that is one of the most representative deep learning models. In this paper, a deep convolutional computation model (DCCM) is proposed to learn hierarchical features of big data by using the tensor representation model to extend the CNN from the vector space to the tensor space. To make full use of the local features and topologies contained in the big data, a tensor convolution operation is defined to prevent overfitting and improve the training efficiency. Furthermore, a high-order backpropagation algorithm is proposed to train the parameters of the deep convolutional computational model in the high-order space. Finally, experiments on three datasets, i.e., CUAVE, SNAE2, and STL-10 are carried out to verify the performance of the DCCM. Experimental results show that the deep convolutional computation model can give higher classification accuracy than the deep computation model or the multimodal model for big data in IoT.

The deep computation model has been proved to be effective for big data hierarchical feature and representation learning in the tensor space. However, it requires expensively computational resources including high-performance computing units and large memory to train a deep computation model with a large number of parameters, limiting its effectiveness and efficiency for industry informatics big data feature learning. In this paper, a tensor-train deep computation model is presented for industry informatics big data feature learning. Specially, the tensor-train network is used to compress the parameters significantly by converting the dense weight tensors into the tensor-train format. Furthermore, a learning algorithm is implemented based on gradient descent and back-propagation to train the parameters of the presented tensor-train deep computation model. Extensive experiments are carried on STL-10, CUAVE, and SNAE2 to evaluate the presented model in terms of the approximation error, classification accuracy drop, parameters reduction, and speedup. Results demonstrate that the presented model can improve the training efficiency and save the memory space greatly for the deep computation model with small accuracy drops, proving its potential for industry informatics big data feature learning.

Deep learning, as the most important architecture of current computational intelligence, achieves super performance to predict the cloud workload for industry informatics. However, it is a nontrivial task to train a deep learning model efficiently since the deep learning model often includes a great number of parameters. In this paper, an efficient deep learning model based on the canonical polyadic decomposition is proposed to predict the cloud workload for industry informatics. In the proposed model, the parameters are compressed significantly by converting the weight matrices to the canonical polyadic format. Furthermore, an efficient learning algorithm is designed to train the parameters. Finally, the proposed efficient deep learning model is applied to the workload prediction of virtual machines on cloud. Experiments are conducted on the datasets collected from PlanetLab to validate the performance of the proposed model by comparing with other machine-learning-based approaches for workload prediction of virtual machines. Results indicate that the proposed model achieves a higher training efficiency and workload prediction accuracy than state-of-the-art machine-learning-based approaches, proving the potential of the proposed model to provide predictive services for industry informatics.

Deep computation models achieve super performance for big data feature learning. However, training a deep computation model poses a significant challenge since a deep computation model typically involves a large number of parameters. Specially, it needs a high-performance computing server with a large-scale memory and a powerful computing unit to train a deep computation model, making it difficult to increase the size of a deep computation model further for big data feature learning on low-end devices such as conventional desktops and portable CPUs. In this paper, we propose an improved deep computation model based on the canonical polyadic decomposition scheme to compress the parameters and to improve the training efficiency. Furthermore, we devise a learning algorithm based on the back-propagation strategy to train the parameters of the proposed model. The learning algorithm can be directly performed on the compressed parameters to improve the training efficiency. Finally, we carry on the experiments on three representative datasets, i.e., CUAVE, SNAE2, and STL-10, to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two improved deep computation models based on the Tucker decomposition and the tensor-train network. Results demonstrate that the proposed model can compress parameters greatly and improve the training efficiency significantly with a low classification accuracy drop.

With the rapid advances of sensing technologies and wireless communications, large amounts of dynamic data pertaining to industrial production are being collected from many sensor nodes deployed in the industrial Internet of Things. Analyzing those data effectively can help to improve the industrial services and mitigate the system unprepared breakdowns. As an important technique of data analysis, clustering attempts to find the underlying pattern structures embedded in unlabeled information. Unfortunately, most of the current clustering techniques that could only deal with static data become infeasible to cluster a significant volume of data in the dynamic industrial applications. To tackle this problem, an incremental clustering algorithm by fast finding and searching of density peaks based on k-mediods is proposed in this paper. In the proposed algorithm, two cluster operations, namely cluster creating and cluster merging, are defined to integrate the current pattern into the previous one for the final clustering result, and k-mediods is employed to modify the clustering centers according to the new arriving objects. Finally, experiments are conducted to validate the proposed scheme on three popular UCI datasets and two real datasets collected from industrial Internet of Things in terms of clustering accuracy and computational time.

Recently, the deep computation model, as a tensor deep learning model, has achieved super performance for multimedia feature learning. However, the conventional deep computation model involves a large number of parameters. Typically, training a deep computation model with millions of parameters needs high-performance servers with large-scale memory and powerful computing units, limiting the growth of the model size for multimedia feature learning on common devices such as portable CPUs and conventional desktops. To tackle this problem, this article proposes a Tucker deep computation model by using the Tucker decomposition to compress the weight tensors in the full-connected layers for multimedia feature learning. Furthermore, a learning algorithm based on the back-propagation strategy is devised to train the parameters of the Tucker deep computation model. Finally, the performance of the Tucker deep computation model is evaluated by comparing with the conventional deep computation model on two representative multimedia datasets, that is, CUAVE and SNAE2, in terms of accuracy drop, parameter reduction, and speedup in the experiments. Results imply that the Tucker deep computation model can achieve a large-parameter reduction and speedup with a small accuracy drop for multimedia feature learning.