Cloud segmentation is fundamental in obtaining many parameters of clouds. However, traditional cloud segmentation performs far from satisfactory, due to the fuzzy boundaries and complex textures of clouds. Although deep learning methods have shown superior performance in cloud segmentation, they are constrained by limited labels in ground-based cloud image data sets. This letter established a new Ground-Based Cloud Segmentation (GBCS) data set with 1742 accurately labeled images. Then to evaluate how well deep learning models perform in cloud segmentation, 12 state-of-the-art semantic segmentation networks are selected, among which DeepLabV3+ outperformed all others. Since 1742 images are not enormous, a novel Transfer learning (TL)-DeepLabV3+ model was developed by TL: DeepLabV3+ network was trained with the PASCAL VOC 2012 data set, then retrained in GBCS. TL-DeepLabV3+ showed a high ability of cloud segmentation, scoring the Mean Intersection-over-Union (MIoU) of 91.05% in GBCS and further verified in the UTILITY data set and the Cirrus Cumulus Stratus Nimbus (CCSN) data set.

Recent years have witnessed a widespread increase of interest in network representation learning (NRL). By far most research efforts have focused on NRL for homogeneous networks like social networks where vertices are of the same type, or heterogeneous networks like knowledge graphs where vertices (and/or edges) are of different types. There has been relatively little research dedicated to NRL for bipartite networks. Arguably, generic network embedding methods like node2vec and LINE can also be applied to learn vertex embeddings for bipartite networks by ignoring the vertex type information. However, these methods are suboptimal in doing so, since real-world bipartite networks concern the relationship between two types of entities, which usually exhibit different properties and patterns from other types of network data. For example, E-Commerce recommender systems need to capture the collaborative filtering patterns between customers and products, and search engines need to consider the matching signals between queries and webpages. This work addresses the research gap of learning vertex representations for bipartite networks. We present a new solution BiNE, short forBipartiteNetworkEmbedding, which accounts for two special properties of bipartite networks: long-tail distribution of vertex degrees and implicit connectivity relations between vertices of the same type. Technically speaking, we make three contributions: (1) We design a biased random walk generator to generate vertex sequences that preserve the long-tail distribution of vertices; (2) We propose a new optimization framework by simultaneously modeling the explicit relations (i.e., observed links) and implicit relations (i.e., unobserved but transitive links); (3) We explore the theoretical foundations of BiNE to shed light on how it works, proving that BiNE can be interpreted as factorizing multiple matrices. We perform extensive experiments on five real datasets covering the tasks of link prediction (classification) and recommendation (ranking), empirically verifying the effectiveness and rationality of BiNE. Our experiment codes are available at:https://github.com/clhchtcjj/BiNE.

As a fundamental component in map service, map matching is of great importance for many trajectory-based applications, e.g., route optimization, traffic scheduling, and fleet management. In practice, Hidden Markov Model and its variants are widely used to provide accurate and efficient map matching service. However, HMM-based methods fail to utilize the knowledge (e.g., the mobility pattern) of enormous trajectory big data, which are useful for intelligent map matching. Furthermore, with many following-up works, they are still easily influenced by the common noisy and sparse records in the reality. In this paper, we revisit the map matching task from the data perspective and propose to utilize the great power of massive data and deep learning to solve these problems. Based on the seq2seq learning framework, we build a trajectory2road model with attention mechanism to map the sparse and noisy trajectory into the accurate road network. Different from previous algorithms, our deep learning based model complete the map matching in the latent space, which provides the high tolerance to the noisy trajectory and also enhances the matching with the knowledge of mobility pattern. Extensive experiments demonstrate that the proposed model outperforms the widely used HMM-based methods by more than 10 percent (absolute accuracy) in various situations especially the noisy and sparse settings.

Class imbalance is one of the most basic and important problems of web data. The key to overcoming the class imbalance problems is to increase the effective instances of the minority, that is, data augmentation. Generative Adversarial Networks (GANs), which have recently been successfully applied in the field of image generation, can be used for data augmentation because they can learn the data distribution given ample training data instances and generate more data. However, learning the distributions from the imbalanced data can make GANs easily get stuck in a local optimum. In this work, we propose a new training strategy called Annealing Genetic GAN (AGGAN), which incorporates simulated annealing genetic algorithm into the training process of GANs. And this can help GANs avoid the local optimum trapping problem, which easily occurs when the training set is imbalanced. Unlike existing GANs, which use a fixed adversarial learning objective alternately training a generator, we use multiple adversarial learning objectives to train a set of generators and use the Metropolis criterion in simulated annealing to decide whether the generator should update. More specifically, the Metropolis criterion accepts worse solutions with a certain probability, so it can make our AGGAN escape from the local optimum and find a better solution. Theory and mathematical analysis provide strong theoretical support for the proposed training strategy. And experiments on several datasets demonstrate that AGGAN achieves convincing ability to solve the class imbalanced problem and reduces the training problems inherent in existing GANs.

Cross-modality data processing is critical for the Internet-of-Things (IoT) deployment in healthcare. It can convert the innumerable raw day-to-day medical big data from massive IoT-based medical devices to diagnostic valuable data so that they can be feed to clinical routine. In this article, we propose a novel spatiotemporal two-streams generative adversarial network (SpGAN) as a cross-modality data processing approach to deploy the medical IoT in infarction learning. Our SpGAN remotely converts diagnostic valuable contrast-enhanced images (the “gold standard” for infarction learning, but it requires the injection of contrast agents) directly from raw nonenhanced cine MR images. This converting allows physicians to remotely perform infarction observation and analysis to break through the limitations of time and space by building a cloud computing platform of IoT-based MRI devices. Importantly, this converting offers a low-risk IoT-based manner to eliminate the potential fatal risk caused by contrast agent injection in the current infarction learning workflow. Specifically, SpGAN consists of: 1) a spatiotemporal two-stream framework as an encoding–decoding model to achieve data converting and 2) a spatiotemporal pyramid network enhances those features that are responsible to the infarction learning during encoding to improve decoding performance. Real IoT-based remote diagnosis experiments performed on 230 patients demonstrate that SpGAN provides high-quality converted images for infarction learning and promotes the in-depth application and deployment of IoT in the medical field.

Advances in Internet of Things (IoT) and cloud/edge computing systems could precisely monitor the meteorological elements and environmental conditions. Remote automated observation system (RAOS) makes the full use of IoT to communicate with other sensors, enabling the active responses from passive devices for smart weather. Cloud observation and classification have been regarded as a successful application that could automatically perform emergency tasks in RAOS. However, with the increasing growth of resource exploitation, the performance of communications among the automatic observation platforms, and the efficiency of task allocation among them has become a critical challenge. In this article, an ensemble learning method and resource allocation scheme are proposed to realize the cloud observation and classification with the help of reliable and controllable infrastructures. On the one hand, several ensemble methods, like Bagging, AdaBoost, and Snapshot are selected as a base classifier to capture the cross-semantic and structure features of cloud, while applying them to the ensemble using convolutional neural networks with different base learners and residual neural networks with different depths. on the other hand, a particular cloud-edge distributed framework is proposed for cloud classification approach based on the intelligent network, to overcome the difficulty in the massive data transmission. The experimental results verify that the proposed ensemble approach achieves high accuracy of cloud classification, and effectively improves the number of allocated tasks. Ensemble methods can generate a more accurate prediction than any single classifier or the majority algorithms. It consistently yields lower error rates than single state-of-the-art models at no additional training cost.

Recently, convolution neural network (CNN)-based hyperspectral image (HSI) classification has enjoyed high popularity due to its appealing performance. However, using 2-D or 3-D convolution in a standalone mode may be suboptimal in real applications. On the one hand, the 2-D convolution overlooks the spectral information in extracting feature maps. On the other hand, the 3-D convolution suffers from heavy computation in practice and seems to perform poorly in scenarios having analogous textures along with consecutive spectral bands. To solve these problems, we propose a mixed CNN with covariance pooling for HSI classification. Specifically, our network architecture starts with spectral-spatial 3-D convolutions that followed by a spatial 2-D convolution. Through this mixture operation, we fuse the feature maps generated by 3-D convolutions along the spectral bands for providing complementary information and reducing the dimension of channels. In addition, the covariance pooling technique is adopted to fully extract the second-order information from spectral-spatial feature maps. Motivated by the channel-wise attention mechanism, we further propose two principal component analysis (PCA)-involved strategies, channel-wise shift and channel-wise weighting, to highlight the importance of different spectral bands and recalibrate channel-wise feature response, which can effectively improve the classification accuracy and stability, especially in the case of limited sample size. To verify the effectiveness of the proposed model, we conduct classification experiments on three well-known HSI data sets, Indian Pines, University of Pavia, and Salinas Scene. The experimental results show that our proposal, although with less parameters, achieves better accuracy than other state-of-the-art methods.

Smart manufacturing, which is increasingly popular worldwide, is aided by time-series forecasting. As the volume of historical data increases, powerful forecasting techniques that reveal unknown relationships between past and future values are required to provide accurate forecasts of production and sales. Thus, in this article, a composite gate recurrent unit (GRU)-Prophet model with an attention mechanism was constructed to predict sales volume. In this composite model, Prophet model and GRU model with attention mechanism were used to capture linear and nonlinear features, respectively. The composite model was experimentally determined to be more applicable and to provide more accurate predictions than did recurrent neural network, long short-term memory, gate recurrent unit, Prophet, and autoregressive integrated moving average models. This article's composite model is suited to rapid changes in market demand and helps enterprises be more competitive in the field of smart manufacturing.

A large amount of labeled data are important to enhance the performance of deep-learning-based methods in the area of fault diagnosis. Because it is difficult to obtain high-quality samples in real industrial applications, federated learning is an effective framework for solving the problem of sparse samples by using the distributed data. Its global model is updated by the local client without sharing data at each round. Considering computing resources and communication loss of multiple clients, an efficient method based on stacked sparse autoencoders (SSAEs) and Siamese networks is proposed to detect interturn short-circuit (ITSC) faults in permanent magnet synchronous motors. In this article, to achieve an accurate ITSC fault detection, an SSAE was employed to extract sparse features in a limited number of samples, and Siamese networks were used to determine the similarity between the given samples. The problem of fault diagnosis is transformed into a classification problem under few-shot learning. Furthermore, the proposed method is trained successfully in the frameworks of centralized learning and decentralized structure. The experimental results indicate that the proposed method achieved high fault diagnosis accuracy. Moreover, it is suitable for deployment in smart manufacturing systems.

Numerous surveillance data processing is crucial in the Internet-of-Things systems with pervasive edge computing. In this process, salient object detection from surveillance videos plays an important role because it provides the human-concerned semantic cue for various industrial tasks. However, it is still challenging for the existing studies with two aspects. The first one is the redundant saliency information from moving background to disturb the detection of salient objects. The second one is the difficulty to model the spatiotemporal saliency uncertainty. To overcome these challenges. In this article, an intelligent approach is proposed for surveillance saliency detection. It enables a region-proposal-based optical flow strategy to suppress the saliency enhancement of non-salient regions due to the moving background. Besides, it develops the bidirectional Bayesian state transition strategy to model the motion uncertainty for refining the spatiotemporal saliency feature. Extensive experiments have been performed on two datasets (the increase of Fβis larger than 0.01 for DAVIS, and larger than 0.015 for UVSD), and the comparison with seven methods to evaluate the effectiveness of the proposed approach.

With the wide use of visual sensors in the Internet of Things (IoT) in the past decades, huge amounts of images are captured in people's daily lives, which poses challenges to traditional deep-learning-based image retrieval frameworks. Most such frameworks need a large amount of annotated training data, which are expensive. Moreover, machines still lack human intelligence, as illustrated by the fact that they pay less attention to the interesting regions that humans generally focus on when searching for images. Hence, this paper proposes a novel unsupervised framework that focuses on the instance object in the image and integrates human intelligence into the deep-learning-based image retrieval. This framework is called adversarial instance-level image retrieval (AILIR). We incorporate adversarial training and an attention mechanism into this framework that considers human intelligence with artificial intelligence. The generator and discriminator are redesigned to guarantee that the generator retrieves similar images while the discriminator selects unmatched images and creates an adversarial reward for the generator. A minimax game is conducted by the adversarial reward retrieval mechanism until the discriminator is unable to judge whether the image sequence retrieved matches the query. Comparison and ablation experiments on four benchmark datasets prove that the proposed adversarial training framework indeed improves instance retrieval and outperforms the state-of-the-art methods focused on instance retrieval.

The unmanned surface vehicles (USVs) have been regarded as a promising paradigm to automatically perform emergency tasks in a dynamic maritime traffic environment. However, the performance of maritime communication between USVs and offshore platforms becomes a critical challenge, and the efficiency of task allocation for USVs in the smart ocean is low. In this article, a novel task allocation scheme for USVs in the smart ocean Internet of Things (IoT) is proposed to improve the efficiency of task allocation. First, the offshore platform is developed to provide maritime communication for USVs in the smart ocean IoT. Second, the network resource allocation process between USVs and offshore platforms is modeled as the second price sealed auction game, where the optimal bidding strategy of USV is derived by the Q-learning to maximize the utilities of USVs and offshore platforms. Third, the task allocation scheme is proposed to improve the number of allocated tasks. Finally, the performance of the proposed scheme is conducted based on extensive simulations. The simulation results show that the proposed scheme can significantly improve the number of allocated tasks compared with the conventional schemes.

Light field image quality assessment (LFI-QA) is a significant and challenging research problem. It helps to better guide light field acquisition, processing and applications. However, only a few objective models have been proposed and none of them completely consider intrinsic factors affecting the LFI quality. In this paper, we propose a No-Reference Light Field image Quality Assessment (NR-LFQA) scheme, where the main idea is to quantify the LFI quality degradation through evaluating the spatial quality and angular consistency. We first measure the spatial quality deterioration by capturing the naturalness distribution of the light field cyclopean image array, which is formed when human observes the LFI. Then, as a transformed representation of LFI, the Epipolar Plane Image (EPI) contains the slopes of lines and involves the angular information. Therefore, EPI is utilized to extract the global and local features from LFI to measure angular consistency degradation. Specifically, the distribution of gradient direction map of EPI is proposed to measure the global angular consistency distortion in the LFI. We further propose the weighted local binary pattern to capture the characteristics of local angular consistency degradation. Extensive experimental results on four publicly available LFI quality datasets demonstrate that the proposed method outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.