Multimodal sentiment analysis (MSA) has emerged as a powerful approach to better understand human emotions by integrating information from multiple modalities such as text, audio, and visual data. However, challenges arise from semantic discrepancies and modality conflicts during multimodal fusion, as well as complex sentiment scenes. This study proposes a novel framework for addressing these issues, referred to as Context-aware Multimodal Sentiment Analysis (CMSA), which incorporates dynamic correction mechanisms and context-aware adaptation to enhance sentiment analysis accuracy. Specifically, we introduce a Text-guided Multimodal Alignment and Correction module that leverages text as the dominant modality to guide the correction of auxiliary modalities (audio and visual), reducing the semantic gap between them. This correction process is further enhanced by a cross-modal alignment mechanism, ensuring that conflicting information across modalities is resolved effectively. Additionally, CMSA incorporates Context-aware mechanisms to adjust the model’s behavior based on contextual variations by utilizing Context-aware Mixture-of-Experts (CMoE), optimizing performance across diverse scenarios. Experimental results on several benchmark datasets demonstrate the superiority of our approach. CMSA outperforms state-of-the-art methods in terms of both sentiment classification accuracy and robustness to modality conflicts, confirming its effectiveness in multimodal sentiment analysis tasks. The proposed model’s ability to adapt to different contexts and dynamically refine modality contributions presents a promising direction for future research in multimodal emotion recognition and sentiment analysis.

Computer-aided diagnosis greatly enhances personalized treatment planning and diagnostic efficiency by providing accurate dental anatomy through teeth segmentation. However, it still constrained by the scarcity of high-quality annotated dental datasets. To address this issue, this paper presents a dataset combining both 2D panoramic X-rays with over 6,500 images and 3D CBCT with over 580 volumes (88,500+ slices) to support the Semi-supervised Teeth Segmentation (STS) Challenge, which includes partially meticulous annotations and covers all age groups. Moreover, multi-phase semi-supervised teeth segmentation algorithms and high-confidence pseudo-labels refinement strategies were proposed by competitors during this challenge. Algorithms were verified on this proposed dataset and good segmentation performance were achieved, over 93+ and 80+ Dice score were obtained for top three 2D and 3D participants, demonstrating the high quality of this proposed dataset. This paper also summarizes the diverse methods employed by the top-ranking teams in the MICCAI 2023 STS Challenge. Our dataset is publicly accessible through Zenodo ( https://zenodo.org/records/10597292 ), and the participants’ code is hosted on GitHub ( https://github.com/ricoleehduu/STS-Challenge ).

Fully inductive knowledge graph completion (KGC) aims to predict triplets involving both unseen entities and relations. Recent several approaches transform paths between entities into descriptions and modeling semantic correlations between paths using pre-trained language models (PLMs), have emerged as a promising solution for fully inductive reasoning . However, these methods often adopt a simplistic concatenation strategy for path-to-sentence transformation, which impedes PLMs’ ability to capture subtle nuances in context, resulting in sub-optimal path context embeddings. Furthermore, they ignore the high-order semantics underlying the complete context, which can provide richer information for inductive reasoning . To address these issues, we propose a Multi-Granularity Contextual Semantic (MGCS) modeling framework, utilizing a Path Modeling Network (PMN) and a Subgraph Modeling Network (SMN) to extract two granularity levels of contextual semantics from single paths and complete subgraphs, for fully inductive KGC. The PMN extracts paths between head and tail entities and employs reasoning patterns from similar cases to filter out unreliable paths. Then two innovative path conversion strategies are designed to significantly enhance the pre-trained language model’s understanding of specific path contexts. The SMN employs a neighbor interactive graph neural network to extract high-order semantics from the complete subgraph context with a concept-enhanced relation encoding, and optimizes it through a contrastive learning method. Finally, the confidence of the triples is evaluated from the perspective of global complete context by comparing the semantics between the subgraphs surrounding the target triplet and the subgraphs surrounding similar cases. Experimental results on benchmark datasets demonstrate the effectiveness of MGCS.

Stealthy False Data Injection Attack (FDIA) that intentionally modifies measurement data of smart grid meters to bypass the traditional bad data detection module is one of menacing cyber attacks in smart grid. Due to requiring no costly labeling abnormal measurement data, deep neural networks (DNNs) based unsupervised FDIA detection has attracted great attentions. However, the existing schemes have two weaknesses. First, most schemes didn’t take into account the inherent spatial relationships between measurements in the grid. Second, for practical usage, the robustness and generalization of the trained FDIA detection scheme will be influenced by potential noisy measurement data. To address the issues above, based on spatial Graph Neural Network (GNN) architecture, a novel FDIA detection and localization scheme is proposed, named as Recursive Variational Graph Autoencoder (ReVGAE). Specifically, our contributions are following. The VGAE module in our proposed ReVGAE innovatively plays dual roles: data and topology reconstructor, and denoising module. The first role aims to simultaneously reconstruct both nodes’ temporal measurements and topological relationship between nodes. In the second role, the outputs of VGAE as the reconstructor (i.e., the reconstructed temporal measurements) are intentionally used as the artificially noisy samples, and recursively fed into VGAE as input to improve the model’s robustness. Then the residual between the finally reconstructed and the observed measurement data on each node is viewed as anomaly score to judge whether FDIA temporally happens on each node. Thorough experiments on a real grid system demonstrate that the proposed ReVGAE outperforms other VAE and GNN based FDIA anomaly detection schemes.

Accurately forecasting wind speed is crucial for efficiently utilizing the renewable energy, stabilizing the energy system and advancing the progress of the decarbonization of our society. However, due to its inherently temporal volatility and intermittency, accurate wind speed forecasting in a wind farm is challenging. Recently, Large Language Models (LLMs) have demonstrated notable performance in abundant natural language processing and computer vision tasks. However, the conventional LLMs fail to learn the complex spatial and temporal correlations of the wind speed data at multiple turbines in a wind farm, which makes wind speed forecasting cant fully benefit from the significant breakthroughs of LLM. To fill in this gap, we propose a novel spatial-temporal cross-attention LLM framework for wind speed forecasting, namely STCA-LLM, composed of alignment phase, and fine-tuning phase. In detail, our contributions are given as follows. First, the alignment phase aligns the general-purpose LLM with task-specific data, i.e., training the LLM with wind speed data. Second, in the fine-tuning phase, two representation learning modules, i.e., convolution network and graph neural network (GNN) are respectively used to extract temporal features of intra time series in each turbine, and the correlation of inter time-series at multiple turbines in a wind farm. Moreover, the cross-attention module is innovatively proposed to establish the connections between spatial and temporal embeddings. Then, the spatial-temporal representation modules and the aligned LLM are fine-tuned in two-stage way. Finally, thorough experiments on real wind speed dataset demonstrate that our proposed STCA-LLM outperforms state-of-the-art time series forecasting models including Transformer-based models, spatial-temporal GNN-based models, and pertained LLM-based models. Our code is available at: https://github.com/Justinzzcj/STCA-LLM

The rapid development and usage of digital technologies in modern intelligent systems and applications bring critical challenges on data security and privacy. It is essential to allow cross-organizational data sharing to achieve smart service provisioning, while preventing unauthorized access and data leak to ensure end users’ efficient and secure collaborations. Federated Learning (FL) offers a promising pathway to enable innovative collaboration across multiple organizations. However, more stringent security policies are needed to ensure authenticity of participating entities, safeguard data during communication, and prevent malicious activities. In this paper, we propose a Decentralized Federated Graph Learning (FGL) with Lightweight Zero Trust Architecture (ZTA) model, named DFGL-LZTA, to provide context-aware security with dynamic defense policy update, while maintaining computational and communication efficiency in resource-constrained environments, for highly distributed and heterogeneous systems in next-generation networking. Specifically, with a re-designed lightweight ZTA, which leverages adaptive privacy preservation and reputation-based aggregation together to tackle multi-level security threats (e.g., data-level, model-level, and identity-level attacks), a Proximal Policy Optimization (PPO) based Deep Reinforcement Learning (DRL) agent is introduced to enable the real-time and adaptive security policy update and optimization based on contextual features. A hierarchical Graph Attention Network (GAT) mechanism is then improved and applied to facilitate the dynamic subgraph learning in local training with a layer-wise architecture, while a so-called sparse global aggregation scheme is developed to balance the communication efficiency and model robustness in a P2P manner. Experiments and evaluations conducted based on two open-source datasets and one synthetic dataset demonstrate the usefulness of our proposed model in terms of training performance, computational and communication efficiency, and model accuracy, compared with other four state-of-the-art methods for next-generation networking security in modern distributed learning systems.

Recently, machine/deep learning techniques are achieving remarkable success in a variety of intelligent control and management systems, promising to change the future of artificial intelligence (AI) scenarios. However, they still suffer from some intractable difficulty or limitations for model training, such as the out-of-distribution (OOD) issue, in modern smart manufacturing or intelligent transportation systems (ITSs). In this study, we newly design and introduce a deep generative model framework, which seamlessly incorporates the information theoretic learning (ITL) and causal representation learning (CRL) in a dual-generative adversarial network (Dual-GAN) architecture, aiming to enhance the robust OOD generalization in modern machine learning (ML) paradigms. In particular, an ITL- and CRL-enhanced Dual-GAN (ITCRL-DGAN) model is presented, which includes an autoencoder with CRL (AE-CRL) structure to aid the dual-adversarial training with causality-inspired feature representations and a Dual-GAN structure to improve the data augmentation in both feature and data levels. Following a newly designed feature separation strategy, a causal graph is built and improved based on the information theory, which can enhance the causally related factors among the separated core features and further enrich the feature representation with the counterfactual features via interventions based on the refined causal relationships. The ITL is incorporated to improve the extraction of low-dimensional feature representations and learn the optimized causal representations based on the idea of "information flow." A dual-adversarial training mechanism is then developed, which not only enables the generator to expand the boundary of feature distribution in accordance with the optimized feature representation from AE-CRL, but also allows the discriminator to further verify and improve the quality of the augmented data for OOD generalization. Experiment and evaluation results based on an open-source dataset demonstrate the outstanding learning efficiency and classification performance of our proposed model for robust OOD generalization in modern smart applications compared with three baseline methods.

The accurate power load forecasting is beneficial in reasonably arranging the power supply load, and greatly promoting the safety and smooth operation of smart grid. However, on one hand, high volatility exists in electricity load, especially, in which a few spike loads suddenly occurred has significantly affected the accuracy of short term load forecasting. On the other hand, most existing forecasting schemes follows one-size-fits-all paradigm, i.e., train and utilize a unique model to forecast the whole loads, irrespective of the spike or normal loads. Based on the above observation, we propose a novel short-term load forecasting scheme assisted by cooopeative Multi-task learning (MTL)-based spike occurrence prediction, LFAS, which is composed of two stages. At the first stage, a MTL-based spike occurrence prediction is explicitly proposed to forecast whether the future loads would be spike or not. Then, instead of one-size-fits-all scheme, the second stage intentionally selects the suitable data pre-processing technologies and deep neural network (DNN) prediction models for respectively forecasting spike and normal loads. The experimental results on the Europe electricity load dataset from ENTSO-E Transparency Platform demonstrate that our proposed scheme increases the accuracy of spike occurrence prediction, significantly improves both the forecasting accuracy of spike and normal electricity loads, in comparison with the existing one-size-fits-all load forecasting schemes.

Due to the increasingly irregular lifestyles, the health status of youth is gradually deteriorating. It is important to understand the risk factors that contribute to this deterioration. In this study, we focus on analyzing lifestyles and behaviors with causal discovery to clarify the relationships between them and youth health. We analyze objective wearable device data and subjective self-assessment health data using NOTEARS, a causal discovery algorithm, and construct causal graphs. Experiment results show that there are causal relationships between exercise factors and stress degree, and between sleep quality and self-assessment score. Unhealthy habits like alcohol consumption and staying up late negatively affect youth health. Notably, females and individuals of higher BMI are more prone to low mood and stress.

In recent years, with the widespread of the Internet and further big data, E-Commerce (EC) has emerged as a popular medium for users to engage in online transactions of products and services. Generally, Service Providers (SPs) of EC collect users' personal information and utilize advanced big data technologies to enhance their services. However, the price discrimination problem may also arise based on personalized information, where malicious SPs analyze users' historical orders to provide the same products or services at varying prices depending on their characteristics. In this paper, we propose a price auditing system called E-Commerce Price Auditing System (ECPAS) to resolve this problem. ECPAS consists of four smart contracts: User Registration Contract, Product Registration Contract, Insurance Purchasing Contract, and Price Auditing Contract, which realize EC price auditing and financial compensation for price discrimination based on a private blockchain. Meanwhile, ECPAS utilizes InterPlanetary File System (IPFS) to efficiently store product data. Experimental results demonstrate that ECPAS achieves a higher processing speed of 5 million price auditing per day while maintaining low gas and on-chain storage costs based on the IPFS.

With the flourish of digital technologies and rapid development of 5G and beyond networks, Metaverse has become an increasingly hotly discussed topic, which offers users with multiple roles for diversified experience interacting with virtual services. How to capture and model users’ multi-platform or cross-space data/behaviors become essential to enrich people with more realistic and immersed experience in Metaverse-enabled smart applications over 5G and beyond networks. In this study, we propose a Personalized Federated Learning with Model-Contrastive Learning (PFL-MCL) framework, which may efficiently enhance the communication and interaction in human-centric Metaverse environments by making use of the large-scale, heterogeneous, and multi-modal Metaverse data. Differing from the conventional Federated Learning (FL) architecture, a multi-center aggregation structure to learn multiple global models based on the changes of dynamically updated local model weights, is developed in global, while a hierarchical neural network structure which includes a personalized module and a federated module to tackle both issues on data heterogeneity and model heterogeneity, is designed in local, so as to enhance the performance of PFL with unique characteristics of Metaverse data. In particular, a two-stage iterative clustering algorithm with a more precise initialization is developed to facilitate the personalized global aggregation with dynamically updated multiple aggregation centers. A personalized multi-modal fusion network is constructed to greatly reduce the computational cost and feature dimensions from the high-dimensional heterogeneous inputs for more efficient cross-modal fusion, based on a hierarchical shift-window attention mechanism and a newly designed bridge attention mechanism. A MCL scheme is then incorporated to speed up the model convergence with less communication overload between the local federated module and global model, while an embedding layer which effectively enables the delivered global model to better adapt to the local personality in each client is further integrated. Compared with five baseline methods, experiment and evaluation results based on two different real-world datasets demonstrate the excellent performance of our proposed PFL-MCL model in a fine-grain personalized training strategy, toward more efficient communication and networking among human-centric Metaverse enabled smart applications.

Eyelid malignant melanoma (MM) is a rare disease with high mortality. Accurate diagnosis of such disease is important but challenging. In clinical practice, the diagnosis of MM is currently performed manually by pathologists, which is subjective and biased. Since the heavy manual annotation workload, most pathological whole slide image (WSI) datasets are only partially labeled (without region annotations), which cannot be directly used in supervised deep learning. For these reasons, it is of great practical significance to design a laborsaving and high data utilization diagnosis method. In this paper, a self-supervised learning (SSL) based framework for automatically detecting eyelid MM is proposed. The framework consists of a self-supervised model for detecting MM areas at the patch-level and a second model for classifying lesion types at the slide level. A squeeze-excitation (SE) attention structure and a feature-projection (FP) structure are integrated to boost learning on details of pathological images and improve model performance. In addition, this framework also provides visual heatmaps with high quality and reliability to highlight the likely areas of the lesion to assist the evaluation and diagnosis of the eyelid MM. Extensive experimental results on different datasets show that our proposed method outperforms other state-of-the-art SSL and fully supervised methods at both patch and slide levels when only a subset of WSIs are annotated. It should be noted that our method is even comparable to supervised methods when all WSIs are fully annotated. To the best of our knowledge, our work is the first SSL method for automatic diagnosis of MM at the eyelid and has a great potential impact on reducing the workload of human annotations in clinical practice.

With the explosion of information, personalized recommender plays a vital role in almost all economic platforms. Usually, the recommender exploits user–item (UI) interactive data to learn users’ latent interests, and then correspondingly conducts recommendations. To address the problem of sparse interactions, graph neural networks (GNNs) have been used to efficiently learn the latent representations of users and items, through structurally modeling the inter-relationships among users, items, and their attributes as graphs. However, most of the existing GNN-based methods ignore the issue of the irrelevant attributes, which means that some attributes of an item are irrelevant to a specific user’s preference. Incorporating them into a recommendation scheme may introduce noise and decrease recommendation accuracy. Thus, to address the issue above, this article proposes a novel personalized recommender based on learnable attribute sampling and heterogeneous graph neural network (LASGRec) to improve the recommender’s performance. The work’s contributions are mainly threefold. First, based on the user’s interactive history with items, the heterogeneous user–item–attribute (UIA) graph is constructed, and attributes of the items are sampled with a learnable neural network to alleviate the issue of irrelevant attributes. Second, using the pruned UIA, the heterogeneous GNN model is appropriately used to learn representations of users and items. Novelly, the learnt user’s embedding first aggregates the sampled attributes of items interactive with the user, and then aggregates these items. The learnt embedding of each item incorporates two relationships: the interacted users and its sampled attributes. Finally, comprehensive experiments on multiple real-world datasets demonstrate the superiority of the proposed LASGRec over the state-of-the-art deep neural network (DNN-) and GNN-based recommendation schemes.

The proliferation of Internet-of-Things (IoT) technologies in modern smart society enables massive data exchange for offering intelligent services. It becomes essential to ensure secure communications while exchanging highly sensitive IoT data efficiently, which leads to high demands for lightweight models or algorithms with limited computation capability provided by individual IoT devices. In this study, a graph representation learning model, which seamlessly incorporates graph neural network (GNN) and knowledge distillation (KD) techniques, named reconstructed graph with global-local distillation (RG-GLD), is designed to realize the lightweight anomaly detection across IoT communication networks. In particular, a new graph network reconstruction strategy, which treats data communications as nodes in a directed graph while edges are then connected according to two specifically defined rules, is devised and applied to facilitate the graph representation learning in secure and efficient IoT communications. Both the structural and traffic features are then extracted from the graph data and flow data respectively, based on the graph attention network (GAT) and multilayer perceptron (MLP) techniques. These can benefit the GNN-based KD process in accordance with the more effective feature fusion and representation, considering both structural and data levels across the dynamic IoT networks. Furthermore, a lightweight local subgraph preservation mechanism improved by the graph attention mechanism and downsampling scheme to better utilize the topological information, and a so-called global information alignment defined based on the self-attention mechanism to effectively preserve the global information, are developed and incorporated in a refined graph attention based KD scheme. Compared with four different baseline methods, experiments and evaluations conducted based on two public datasets demonstrate the usefulness and effectiveness of our proposed model in improving the efficiency of knowledge transfer with higher classification accuracy but lower computational load, which can be deployed for lightweight anomaly detection in sustainable IoT computing environments.

Abstract Previous studies have shown that about 90% of traffic accidents are due to human error, which means that human factors may affect a driver's braking behaviors and thus their driving safety, especially when the driver makes a braking motion. However, most studies have mounted sensors on the brake pad, ignoring to some extent an analysis of the driver's behavior before the brake pad is pressed (pre-braking). Therefore, to determine the effect of different human factors on drivers' pre-braking behaviors, this study focused on analyzing drivers' local joints (knee, ankle, and toe) by a motion capture device. A Hilbert–Huang Transform (HHT)-based local human body movement analysis method was used to decompose the realistic complex pre-braking actions into sub-actions such as intrinsic mode functions (IMF1, IMF2, etc.). Analysis of the results showed that IMF1 is a common and necessary action when pre-braking for all drivers, and IMF2 may be the safety assurance action that allows right-foot transverse movement at the beginning part of the pre-braking process. We also found that the experienced, male, and Phys.50 groups may have consistent characteristics in the HHT scheme, which could mean that such drivers would have better performance and efficiency during the pre-braking process. The results of this study will be useful in decomposing and discerning the specific actions that lead to accidents, providing insights into driver training for novice drivers, and guiding the construction of daily automated driver assistance or accident prevention systems (advanced driver assistance systems, ADASs).

Abstract As a distributed learning framework, Federated Learning (FL) allows different local learners/participants to collaboratively train a joint model without exposing their own local data, and offers a feasible solution to legally resolve data islands. However, among them, the data privacy and model security are two challenges. The former means that, if original data are used for trained FL models, various methods can be used to deduce the original data samples, thereby causing the leakage of data. The latter implies that unreliable/malicious participants may affect or destroy the joint FL model, through uploading wrong local model parameters. Therefore, this paper proposes a novel distributed FL training framework, namely LDP‐Fed+, which takes into account differential privacy protection and model security defense. Specifically, firstly, a local perturbation module is added at the local learner side, which perturbs the original data of local learners through feature extraction, binary encoding and decoding, and random response. Then, through using the perturbed data, local neural network model is trained to obtain the network parameters that meet local differential protection, to effectively deal with model inversion attacks. Secondly, a security defense module is added on the server side, which uses the auxiliary model and differential index mechanism to select an appropriate number of local disturbance parameters for aggregation to enhance model security defense and deal with membership inference attacks. The experimental results show that, compared with other federated learning models based on differential privacy, LDP‐Fed+ has stronger robustness for model security and higher accuracy for model training while ensuring strict privacy protection.

Summary The power of artificial intelligence of things (AIoT) stems from adapting machine learning (ML) and artificial intelligence (AI) models into abundant intelligent IoT fields, based on a large data stream with different formats, sizes, and timestamps generated by massive numbers of heterogeneous sensors. On the one hand, data acquisition is the fundamental basis for any AIoT systems, but data sensed by massive IoT devices may be noisy and even contain adversarial samples. On the other hand, ensuring the efficiency and robustness in data acquisition is vitally important for data‐driven ML and AI. Recently, besides perceiving ability, the literature has witnessed great development of empowering things with learning and reasoning ability through deep learning models, including recurrent neural networks (RNNs) and/or convolutional neural network (CNNs). However, the existing works have one significant weakness: fail to explicitly leverage the geospatial implications and latent connections among sensors for high‐quality data acquisition and quality control. Graphs are intrinsically suitable for representing the dependencies and inter‐relationships between AIoT data sensing devices. Due to the ability of capturing the complex interactive relationships between nodes and producing high‐level representations of the graph input, graph neural networks (GNNs) have exploded onto various ML and AI fields, to learn from graph‐structured data. Our review covers the latest progresses in GNN for the fundamental atomic task of data acquisition in AIoT. Instead of surveying the abundant GNN schemes in vertically various IoT sensing applications, this paper systematically reviews the horizontal infrastructure that all AIoT fields should have, that is, AIoT data acquisition, based on GNN and other related emerging AI factors. Our contributions include the following aspects: Provide the latest progresses in GNN for the horizontal task of data acquisition in AIoT, propose the unified GNN pipeline based on encoder–decoder paradigm, and systematically categorize and summarize the emerging technologies helpful to address the issues in AIoT data acquisition, especially the noisy and adversarial data, and point out some future directions about GNN‐based AIoT data acquisition.

Accurate measurements of ophthalmic parameters and postoperative appearance prediction are essential for the diagnosis and treatment of many ophthalmic diseases. Nevertheless, it remains challenging due to (1) inconsistent ophthalmic image sampling standards, including ocular-camera distance, facial angle, and patient number, (2) complicated ocular morphology, such as subconjunctival hemorrhage, ocular movements, lighting effects, and morphological aging. It is difficult for a model to measure parameters and make predictions in variable sampling methods and morphology conditions. Therefore, the Global attention-based Ophthalmic Image Measurement and Postoperative Appearance Prediction System (GOMPS) is proposed, which quantifies ophthalmic image parameters to diagnose disease and simultaneously predict postoperative appearance of blepharoptosis. By perceiving the global structure of the ophthalmic image, GOMPS makes logical inference predictions of the sclera and cornea morphology, to overcome the above difficulties. Concretely, a global attention unit (GAU) and a novel global attention structure-aware network (GASA-Net) are designed to enhance GOMPS’s global structure awareness ability to perform logical reasoning. Extensive experimental results on our collected ophthalmic dataset for diagnosis & prediction (OD2P) demonstrate that GOMPS surpasses the state-of-the-art methods in segmentation accuracy and achieves the current optimal performance in measurement and postoperative prediction under many clinical scenes.

Group behavior modeling is an important research topic in the field of social network analysis . Existing methods regarding this topic can only learn the static group preference, ignoring the multiple characteristics of the group behavior , so they cannot model the group behavior in a complete way. In this paper, we propose a Dynamic Multi-view Group Preference Learning (DMGPL) model for group behavior prediction in social networks. Firstly, an area-aware user dynamic preference extracting module is developed concerning user representation learning , which can integrate the dynamic user behavior preference and the corresponding group structural information into group members’ representations. Then, to simultaneously obtain stability, propensity, potentiality, and heterogeneity of the group behavior, Multi-view Group Preference Aggregation (MGA) is designed for group behavior modeling. In MGA, various aggregation strategies are used to capture different kinds of group behavior properties, which can achieve the effect of highlighting different properties in different scenarios. Moreover, considering the influence of the member’s size on group behavior, we define the group scaling degree and perform group representation scaling to make MGA more flexible. Finally, extensive experiments are performed on two real-world datasets, and the results show that the accuracy of group behavior prediction by DMGPL improved by about 10% compared to the baseline models .

The Internet of Things (IoT) has become a focus of information infrastructure development in recent years. The smart blockchain can provide various solutions for trust, security, and privacy (TSP) challenges to protect IoT data, and smart contracts are the foundation of blockchain intelligence, and greatly enhance the ability of smart blockchain to solve TSP problems. So, the security of smart contracts must be addressed. We propose an efficient smart contract vulnerability detector to improve the safety of smart contracts. It comprises a graph extraction method and a complete vulnerability detection process. The graph extraction method consists of vulnerability pattern extraction and a graph generation process. The vulnerability detection process first uses the approximate graph matching algorithm to select representative SCGraphs from the data set to build vulnerability SCGraph libraries. Second, determine whether the contract contains vulnerabilities by calculating the similarity between the SCGraphs generated from the contracts to be detected and the SCGraphs in the vulnerability library. Experiments show that our approach achieves an inspiring high detection rate and is the fastest among existing vulnerability detection tools, which indicates that it can provide good vulnerability detection for smart contracts.

Nowadays, the concept of Internet of Everything (IoE) is becoming a hotly discussed topic, which is playing an increasingly indispensable role in modern intelligent applications. These applications are known for their real-time requirements under limited network and computing resources, thus it becomes a highly demanding task to transform and compute tremendous amount of raw data in a cloud center. The edge–cloud computing infrastructure allows a large amount of data to be processed on nearby edge nodes and then only the extracted and encrypted key features are transmitted to the data center. This offers the potential to achieve an end–edge–cloud-based big data intelligence for IoE in a typical two-stage data processing scheme, while satisfying a data security constraint. In this study, a deep-reinforcement-learning-enhanced two-stage scheduling (DRL-TSS) model is proposed to address the NP-hard problem in terms of operation complexity in end–edge–cloud Internet of Things systems, which is able to allocate computing resources within an edge-enabled infrastructure to ensure computing task to be completed with minimum cost. A presorting scheme based on Johnson’s rule is developed and applied to preprocess the two-stage tasks on multiple executors, and a DRL mechanism is developed to minimize the overall makespan based on a newly designed instant reward that takes into account the maximal utilization of each executor in edge-enabled two-stage scheduling. The performance of our method is evaluated and compared with three existing scheduling techniques, and experimental results demonstrate the ability of our proposed algorithm in achieving better learning efficiency and scheduling performance with a 1.1-approximation to the targeted optimal IoE applications.

Artificial intelligence (AI) continues demonstrating its positive impact on society and successful adoptions in data-rich domains including social computing systems. There are serious ethical and legal concerns about AI’s ability to make decisions in a responsible way. Many principles and guidelines for responsible AI (RAI) have been issued by governments, research organizations, and enterprises. For instance, the Institute for Ethical Machine Learning provides various RAI resources[1], including higher level guidelines and frameworks, tools, standards, regulations, course, and so on. However, high-level principles are far from ensuring the trustworthiness of AI systems.

The proliferation in embedded and communication technologies made the concept of the Internet of Medical Things (IoMT) a reality. Individuals’ physical and physiological status can be constantly monitored, and numerous data can be collected through wearable and mobile devices. However, the silo of individual data brings limitations to existing machine learning approaches to correctly identify a user’s health status. Distributed machine learning paradigms, such as federated learning, offer a potential solution for privacy-preserving knowledge sharing without sending raw personal data. However, federated learning is vulnerable to harmful participants that can degrade the overall model quality by sharing low-quality data. Therefore, it is critical to select suitable participants to ensure the accuracy and efficiency of federated learning. In this article, a unique clustering-based approach is proposed to use social context data for participant selection. Different edge participant groups will be established, and group-specific federated learning will be performed. The models of various edge groups will be further aggregated to strengthen the robustness of the global model. The experimental results demonstrated that through participant selection, clustering-based hierarchical federated learning can achieve better results with less participants in two different IoMT applications for ECG and human motion monitoring. This shows the efficacy of the proposed method in improving federated learning performance and efficiency in various IoMT applications.

Safety production surveillance is of great significance to industrial operation management. While augmented intelligence of things is demonstrating tremendous potential in industrial applications, the analyzed information offers lots of benefits to the higher level planning in the enterprise management systems, to further improve the operational efficiency. In this article, a video surveillance system with augmented intelligence of things is considered as a promising solution to enhance the operational efficiency of enterprises. However, the challenge is to process the surveillance video streams as soon as possible without ignoring any emergencies. This issue can be formulated as a two-stage scheduling problem, which is an NP-hard problem that can be integrated with higher level enterprise systems for operational efficiency improvement. An improved Deep Q-Network (DQN) model with a newly designed prioritized replay scheme, named Bi-Dueling DQN with Prioritized Replay, is proposed to solve this two-stage scheduling problem in a smart enterprise management system. A dense reward function based on a concrete state representation is designed to tackle the sparse reward challenge and to speed up the convergence in actual large-scale task scheduling process. A prioritized replay scheme is then developed to improve the sampling efficiency, so as to effectively reduce the training time in deep reinforcement learning for the optimal two-stage scheduling. The experiment results demonstrated that the proposed approach is able to provide an efficient scheduling policy to resolve the two-stage scheduling problem, while at the same time offering insight information to improve the performance of higher level smart enterprise management systems.

Modern manufacturing process is commonly composed of multiple automated devices working together efficiently. Cloud-based manufacturing aims to achieve better efficiency by allowing the collaborative manufacturing across a group of automated robots. Cooperations between multiple robots can accomplish more complicated tasks that is beyond the capability of any individual one. However, it is of critical importance to control robots with different capabilities to work in harmony while ensuring safety and reliability during this process. In this paper, a double constrained containment mechanism is proposed to dispatch heterogeneous robots in a distributed containment control framework for smart manufacturing. Following a three-layer control framework, a cloud decision-making center is designed to realize the cloud-based collaborative manufacturing, which is more cost-effective than other commonly used containment mechanisms that only rely on information exchange among leader-robots. A projection-based containment control scheme is developed, which not only consider nonlinearities induced by position constraints and velocity constraints, but also can tackle dynamically changing communication topologies with uncertain communication delay, to efficiently navigate all follower-robots into a safe working zone formed by leaders. A theoretical stability analysis is conducted to prove the proposed mechanism can ensure all followers enter the target area while their positions and velocities remaining in the corresponding constraint sets. Experiment evaluation results under three application scenarios demonstrate the advantage of our method that can offer a more practical solution to other existing multi-robot containment control for cloud-based smart manufacturing, in considering both position and velocity constraints combined with switching topologies and communication delays.

The impact of Internet of Things (IoT) has become increasingly significant in smart manufacturing, while deep generative model (DGM) is viewed as a promising learning technique to work with large amount of continuously generated industrial Big Data in facilitating modern industrial applications. However, it is still challenging to handle the imbalanced data when using conventional Generative Adversarial Network (GAN) based learning strategies. In this article, we propose a distribution bias aware collaborative GAN (DB-CGAN) model for imbalanced deep learning in industrial IoT, especially to solve limitations caused by distribution bias issue between the generated data and original data, via a more robust data augmentation. An integrated data augmentation framework is constructed by introducing a complementary classifier into the basic GAN model. Specifically, a conditional generator with random labels is designed and trained adversarially with the classifier to effectively enhance augmentation of the number of data samples in minority classes, while a weight sharing scheme is newly designed between two separated feature extractors, enabling the collaborative adversarial training among generator, discriminator, and classifier. An augmentation algorithm is then developed for intelligent anomaly detection in imbalanced learning, which can significantly improve the classification accuracy based on the correction of distribution bias using the rebalanced data. Compared with five baseline methods, experiment evaluations based on two real-world imbalanced datasets demonstrate the outstanding performance of our proposed model in tackling the distribution bias issue for multiclass classification in imbalanced learning for industrial IoT applications.

Precise and rapid categorization of images in the B-scan ultrasound modality is vital for diagnosing ocular diseases. Nevertheless, distinguishing various diseases in ultrasound still challenges experienced ophthalmologists. Thus a novel contrastive disentangled network (CDNet) is developed in this work, aiming to tackle the fine-grained image categorization (FGIC) challenges of ocular abnormalities in ultrasound images, including intraocular tumor (IOT), retinal detachment (RD), posterior scleral staphyloma (PSS), and vitreous hemorrhage (VH). Three essential components of CDNet are the weakly-supervised lesion localization module (WSLL), contrastive multi-zoom (CMZ) strategy, and hyperspherical contrastive disentangled loss (HCD-Loss), respectively. These components facilitate feature disentanglement for fine-grained recognition in both the input and output aspects. The proposed CDNet is validated on our ZJU Ocular Ultrasound Dataset (ZJUOUSD), consisting of 5213 samples. Furthermore, the generalization ability of CDNet is validated on two public and widely-used chest X-ray FGIC benchmarks. Quantitative and qualitative results demonstrate the efficacy of our proposed CDNet, which achieves state-of-the-art performance in the FGIC task.

Summary Behavior is autonomous, convergent, and uncertain, which brings challenges to the modeling of social network behavior spread. In this article, we propose a behavior spread model based on group cohesion under uncertain environments. First, for behavioral convergence, we define group cohesion to quantify the convergent effects of group. Second, based on the game theory to model the autonomy of behavior, according to the characteristics of the game payoffs changing with time and the depth of spread, and integrating group cohesion, a dynamic game payoffs calculation method is designed. Finally, aiming at the uncertainty of behavior, a group behavior spread model based on random utility theory is established. Experiments on multiple real social network behavior spread datasets demonstrate the effectiveness of the proposed model in modeling and predicting behavior spread processes under uncertain environments.

Summary The spread of rumors has a major negative impact on social stability. Traditional rumor spreading models are mostly based on infectious disease models and do not consider the influence of individual differences and the network structure on rumor spreading. In this paper, we propose a rumor Fick‐spreading model that integrates information decay in social networks. The dissemination of rumors in social networks is random and uncertain and is affected by the dissemination capabilities of individuals and the network environment. The rumor Fick‐transition coefficient and Fick‐transition gradient are defined to determine the influence of the individual transition capacity and the network environment on rumor propagation, respectively. The Fick‐state transition probability is used to describe the probability of change of an individual's state. Moreover, an information decay function is defined to characterize the self‐healing probability of individuals. According to the different roles and reactions of users during rumor dissemination, the user state and the rumor dissemination rules among users are refined, and the influence of the network structure on the rumor dissemination is ascertained. The experimental results demonstrate that the proposed model outperforms other rumor spread models.

The smart grid is now increasingly dependent on smart devices to operate, which leaves space for cyber attacks. Especially, the intentionally designed false data injection attack (FDIA) can successfully bypass the traditional measurement residual-based bad data detection scheme. Considering that the smart grid data naturally contain linear and nonlinear components, inspired by parallel ensemble learning, especially by the stacking method, this article presents an effective two-level learner-based FDIA detection scheme using the Kalman filter and recurrent neural network (KFRNN). The first level includes two base learners, in which the Kalman filter is used for state prediction to fit linear data, and the recurrent neural network is used to fit the nonlinear data feature. The second-level learner uses the fully connected layer and backpropagation (BP) module to adaptively combine the results of two base learners. Then, through fitting Weibull distribution of the sum of square errors (SSEs) between the observed measurements and the predicted measurements, the dynamic threshold is obtained to judge whether FDIA occurs or not. Comprehensive simulation results show that our scheme has better performance than other neural network-based and ensemble learning-based FDIA detection schemes.

Couplet is a unique art form in Chinese traditional culture. The development of deep neural network (DNN) technology makes it possible for computers to automatically generate couplets. Especially, Transformer is a DNN-based “Encoder–Decoder” framework, and widely used in natural language processing (NLP). However, the existed Transformer mechanism cannot fully exploit the essential linguistic knowledge in Chinese, including the special format and requirements of Chinese couplets. Therefore, this article adapts the Transformer mechanism to generate meaningful Chinese couplets. Specifically, the contributions of our work are threefold. First, considering the fact that the words in the corresponding positions of the antecedent clause and the subsequent clause in a Chinese couplet always have same part-of-speech (pos, i.e., word class), pos information is intentionally added into the Transformer to improve the accuracy of the conceived couplet. Second, to deal with the large number of unregistered and low-frequency words in Chinese couplet, a specific unregistered/low-frequency word processing mechanism (UWP) is designed and combined with the Transformer model. Third, to further improve the coherence of couplets, we incorporate the polish mechanisms (PMs) into Transformer model. In terms of three evaluation criteria including bilingual evaluation understudy (BLEU), perplexity, and human evaluation, the experimental results demonstrate the effectiveness of our designed Chinese couplet generation system.

Recently, along with several technological advancements in cyber-physical systems, the revolution of Industry 4.0 has brought in an emerging concept named digital twin (DT), which shows its potential to break the barrier between the physical and cyber space in smart manufacturing. However, it is still difficult to analyze and estimate the real-time structural and environmental parameters in terms of their dynamic changes in digital twinning, especially when facing detection tasks of multiple small objects from a large-scale scene with complex contexts in modern manufacturing environments. In this article, we focus on a small object detection model for DT, aiming to realize the dynamic synchronization between a physical manufacturing system and its virtual representation. Three significant elements, including equipment, product, and operator, are considered as the basic environmental parameters to represent and estimate the dynamic characteristics and real-time changes in building a generic DT system of smart manufacturing workshop. A hybrid deep neural network model, based on the integration of MobileNetv2, YOLOv4, and Openpose, is constructed to identify the real-time status from physical manufacturing environment to virtual space. A learning algorithm is then developed to realize the efficient multitype small object detection based on the feature integration and fusion from both shallow and deep layers, in order to facilitate the modeling, monitoring, and optimizing of the whole manufacturing process in the DT system. Experiments and evaluations conducted in three different use cases demonstrate the effectiveness and usefulness of our proposed method, which can achieve a higher detection accuracy for DT in smart manufacturing.

Accurate evaluation of the treatment result on X-ray images is a significant and challenging step in root canal therapy since the incorrect interpretation of the therapy results will hamper timely follow-up which is crucial to the patients’ treatment outcome. Nowadays, the evaluation is performed in a manual manner, which is time-consuming, subjective, and error-prone. In this article, we aim to automate this process by leveraging the advances in computer vision and artificial intelligence, to provide an objective and accurate method for root canal therapy result assessment. A novel anatomy-guided multi-branch Transformer (AGMB-Transformer) network is proposed, which first extracts a set of anatomy features and then uses them to guide a multi-branch Transformer network for evaluation. Specifically, we design a polynomial curve fitting segmentation strategy with the help of landmark detection to extract the anatomy features. Moreover, a branch fusion module and a multi-branch structure including our progressive Transformer and Group Multi-Head Self-Attention (GMHSA) are designed to focus on both global and local features for an accurate diagnosis. To facilitate the research, we have collected a large-scale root canal therapy evaluation dataset with 245 root canal therapy X-ray images, and the experiment results show that our AGMB-Transformer can improve the diagnosis accuracy from 57.96% to 90.20% compared with the baseline network. The proposed AGMB-Transformer can achieve a highly accurate evaluation of root canal therapy. To our best knowledge, our work is the first to perform automatic root canal therapy evaluation and has important clinical value to reduce the workload of endodontists.

Recently, on one hand, human activity recognition (HAR) has witnessed great application on portable smart devices (e.g., smart phones and wearables, etc.) as they are widely used around the world. On the other hand, HAR methods based on deep learning have attracted much attention, for they possess excellent performance due to their strength on extracting virtual features automatically and hierarchically. However, to establish a personalized deep learning based HAR scheme based on smart devices, insufficient records from target users and heavy computation cost on training from scratch are two challenges. Considering that, in transfer learning, the knowledge learnt in the source domain could be appropriately transferred to help accomplish tasks in the target domain, this paper proposes a personalized HAR scheme through exploiting virtual feature adaptation based on transfer learning (i.e., VFAT) to achieve high recognition accuracy with low computation time. VFAT is composed of pre-training phase on sufficient labeled records in source-domain, and adaption phase on target-domain that uses the few labeled records available. Specifically, VFAT scheme pre-trains the LSTM-based feature extraction component in the pre-training phase and then introduces domain loss in the adaptation phase to minimize the similarity between target-domain virtual features and source-domain activity patterns (i.e., virtual features averaged by activity labels). The HAR scheme applied to the MotionSense dataset and results demonstrate the effectiveness of our proposed VFAT scheme. Moreover, we also investigate the impact of domain division on the performance of transfer learning based HAR.

Feature selection (FS) is one of the fundamental data processing techniques in various machine learning algorithms, especially for classification of healthcare data. However, it is a challenging issue due to the large search space. Binary Particle Swarm Optimization (BPSO) is an efficient evolutionary computation technique, and has been widely used in FS. In this paper, we proposed a Confidence-based and Cost-effective feature selection (CCFS) method using BPSO to improve the performance of healthcare data classification. Specifically, first, CCFS improves search effectiveness by developing a new updating mechanism that designs the feature confidence to explicitly take into account the fine-grained impact of each dimension in the particle on the classification performance. The feature confidence is composed of two measurements: the correlation between feature and categories, and historically selected frequency of each feature. Second, considering the fact that the acquisition costs of different features are naturally different, especially for medical data, and should be fully taken into account in practical applications, besides the classification performance, the feature cost and the feature reduction ratio are comprehensively incorporated into the design of fitness function. The proposed method has been verified in various UCI public datasets and compared with various benchmark schemes. The thoroughly experimental results show the effectiveness of the proposed method, in terms of accuracy and feature selection cost.

Pneumothorax is a common pulmonary disease that can lead to dyspnea and can be life-threatening. X-ray examination is the main means to diagnose this disease. Computer-aided diagnosis of pneumothorax on chest X-ray, as a prerequisite for a timely cure, has been widely studied, but it is still not satisfactory to achieve highly accurate results. In this paper, an image classification algorithm based on the deep convolutional neural network (DCNN) is proposed for high-resolution medical image analysis of pneumothorax X-rays, which features a Network In Network (NIN) for cleaning the data, random histogram equalization data augmentation processing, and a DCNN. The experimental results indicate that the proposed method can effectively increase the correct diagnosis rate of pneumothorax, and the Area under Curve (AUC) of the test verified in the experiment is 0.9844 on ZJU-2 test data and 0.9906 on the ChestX-ray14, respectively. In addition, a large number of atmospheric pleura samples are visualized and analyzed based on the experimental results and in-depth learning characteristics of the algorithm. The analysis results verify the validity of feature extraction for the network. Combined with the results of these two aspects, the proposed X-ray image processing algorithm can effectively improve the classification accuracy of pneumothorax photographs.

The papers in this special section focus on machine learning for artificial intelligent-enhances healthcare and medical services. These services are always among the top concerns for humans, especially under the special situation of COVID-19 pandemic, started from early 2020. In the field of computational biology and bioinformatics, scientists seek various possibilities using computer technologies, especially artificial intelligence (AI) enhanced methods, for healthcare services and medical diagnoses. For example, over the past few years, scientists have been working hard to identify the internal relationships between gene microarrays, cells, tissues, organisms, diseases, etc., and apply the AI, machine learning and deep learning technologies looking for more innovative solutions for new diseases, such as COVID-19. In fact, nowadays, AI technology, such as the convolutional neural network (CNN), is considered has one of the most important computer technologies and has been widely applied in the fields of healthcare engineering, medical research, disease diagnosis, cancer/tumor analysis and etc.

The unprecedented development of the Internet of Things (IoT), artificial intelligence (AI), and Big Data has stimulated a boom of social networks such as Twitter, WeChat, Facebook, etc., generating a huge amount of social data that are worth further analysis. Social computing has an important focus on mining the deep relationships between social organizations, networks, and media. The increasing volumes and complexities make big social data mining more and more difficult. Hybrid Human–Artificial Intelligence (H-AI) is an approach combining both human intelligence and AI, so as to handle demanding problems in a harmonious way. By adopting H-AI in social computing, it would provide more possibilities for social data analysis, relationship discovery, outlier detection, and prediction, and is proving to be an emerging and promising direction for AI and big data research.

With the increasing population of Industry 4.0, industrial big data (IBD) has become a hotly discussed topic in digital and intelligent industry field. The security problem existing in the signal processing on large scale of data stream is still a challenge issue in industrial internet of things, especially when dealing with the high-dimensional anomaly detection for intelligent industrial application. In this article, to mitigate the inconsistency between dimensionality reduction and feature retention in imbalanced IBD, we propose a variational long short-term memory (VLSTM) learning model for intelligent anomaly detection based on reconstructed feature representation. An encoder-decoder neural network associated with a variational reparameterization scheme is designed to learn the low-dimensional feature representation from high-dimensional raw data. Three loss functions are defined and quantified to constrain the reconstructed hidden variable into a more explicit and meaningful form. A lightweight estimation network is then fed with the refined feature representation to identify anomalies in IBD. Experiments using a public IBD dataset named UNSW-NB15 demonstrate that the proposed VLSTM model can efficiently cope with imbalance and high-dimensional issues, and significantly improve the accuracy and reduce the false rate in anomaly detection for IBD according to F1, area under curve (AUC), and false alarm rate (FAR).

With the increasing population of Industry 4.0, both AI and smart techniques have been applied and become hotly discussed topics in industrial cyber-physical systems (CPS). Intelligent anomaly detection for identifying cyber-physical attacks to guarantee the work efficiency and safety is still a challenging issue, especially when dealing with few labeled data for cyber-physical security protection. In this article, we propose a few-shot learning model with Siamese convolutional neural network (FSL-SCNN), to alleviate the over-fitting issue and enhance the accuracy for intelligent anomaly detection in industrial CPS. A Siamese CNN encoding network is constructed to measure distances of input samples based on their optimized feature representations. A robust cost function design including three specific losses is then proposed to enhance the efficiency of training process. An intelligent anomaly detection algorithm is developed finally. Experiment results based on a fully labeled public dataset and a few labeled dataset demonstrate that our proposed FSL-SCNN can significantly improve false alarm rate (FAR) and F1 scores when detecting intrusion signals for industrial CPS security protection.

Coronavirus disease 2019 (COVID-19) is one of the most destructive pandemic after millennium, forcing the world to tackle a health crisis. Automated lung infections classification using chest X-ray (CXR) images could strengthen diagnostic capability when handling COVID-19. However, classifying COVID-19 from pneumonia cases using CXR image is a difficult task because of shared spatial characteristics, high feature variation and contrast diversity between cases. Moreover, massive data collection is impractical for a newly emerged disease, which limited the performance of data thirsty deep learning models. To address these challenges, Multiscale Attention Guided deep network with Soft Distance regularization (MAG-SD) is proposed to automatically classify COVID-19 from pneumonia CXR images. In MAG-SD, MA-Net is used to produce prediction vector and attention from multiscale feature maps. To improve the robustness of trained model and relieve the shortage of training data, attention guided augmentations along with a soft distance regularization are posed, which aims at generating meaningful augmentations and reduce noise. Our multiscale attention model achieves better classification performance on our pneumonia CXR image dataset. Plentiful experiments are proposed for MAG-SD which demonstrates its unique advantage in pneumonia classification over cutting-edge models. The code is available at https://github.com/JasonLeeGHub/MAG-SD.

The driver's driving experience is one of the important factors affecting his or her behaviors. Prior studies have noted the effect of driving experiences on drivers' eye-movements when right-turning. However, related studies usually focused on the accident scenarios, for drivers' eye-movements on daily driving when facing right-turn, the effects of drivers' experience remain unknown. Therefore, according to design and apply a set of experiments, this paper focused on the analysis of driver's eye-movements during the daily right-turn task to compare the differences between Experienced and Novice drivers. A total of 10 drivers were invited and be classified into two groups (Experienced and Novice) to participate in the experiments. All of the drivers are driving on the right-hand side of the road, and the steering wheel is on the left side of the vehicle. The aimed data were collected by a set of glasses type eye-tracker and be further classified into four driving vision-based AOI (Areas of Interest). The results of Mann-Whitney U-tests showed that Novice drivers have a disordered line of sight and tend to spend more attention on their right view and switch their line of sight back and forth between the AOIs. Moreover, Experienced drivers more tend to keep their view directly in front of their heads instead of using the peripheral vision. The results of this study may provide guidelines to prevent accidents in Advanced Driver Assistance Systems (ADAS) and offers useful insights for the training of new drivers.

Along with the advancement of several emerging computing paradigms and technologies, such as cloud computing, mobile computing, artificial intelligence, and big data, Internet of Things (IoT) technologies have been applied in a variety of fields. In particular, the Internet of Healthcare Things (IoHT) is becoming increasingly important in human activity recognition (HAR) due to the rapid development of wearable and mobile devices. In this article, we focus on the deep-learning-enhanced HAR in IoHT environments. A semisupervised deep learning framework is designed and built for more accurate HAR, which efficiently uses and analyzes the weakly labeled sensor data to train the classifier learning model. To better solve the problem of the inadequately labeled sample, an intelligent autolabeling scheme based on deep Q-network (DQN) is developed with a newly designed distance-based reward rule which can improve the learning efficiency in IoT environments. A multisensor based data fusion mechanism is then developed to seamlessly integrate the on-body sensor data, context sensor data, and personal profile data together, and a long short-term memory (LSTM)-based classification method is proposed to identify fine-grained patterns according to the high-level features contextually extracted from the sequential motion data. Finally, experiments and evaluations are conducted to demonstrate the usefulness and effectiveness of the proposed method using real-world data.

The crowdsourcing system is a distributed problem-solving platform, in which tasks are delivered to the crowd (i.e., crowdworkers) in the form of an open call. Usually, large-scale crowdsourcing systems contain abundant microtasks, and the overhead of a crowdworker spending on searching the appropriate task may be comparable to the cost of completing the task. Therefore, task recommendation is necessary. However, existing work ignores the dynamics in crowdsourcing system, i.e., new tasks continually arrive, which leads to the issues of task cold-start. To overcome the challenge of the new coming task recommendation, this article proposes a feature-based Bayesian task recommendation (BTR) scheme. The key idea to deal with the dynamics of the crowdsourcing system lies in that the BTR learns the latent factor of the task through the task features instead of task ID and then learns the user's preference according to their historical behaviors. Specifically, based on task features and the user's historical behavior records, BTR can not only timely provide crowdworkers with personalized task recommendations but also solve the task cold-start problem. The simulations based on the real crowdsourced data set demonstrate that BTR performs better than other typical schemes that target at recommending the newly arrived tasks to crowdworkers.

Deep neural network (DNN) learning has witnessed significant applications in various fields, especially for prediction and classification. Frequently, the data used for training are provided by crowdsourcing workers, and the training process may violate their privacy. A qualified prediction model should protect the data privacy in training and classification/prediction phases. To address this issue, we develop a differential privacy (DP)-enabled DNN learning framework, DNN-DP, that intentionally injects noise to the affine transformation of the input data features and provides DP protection for the crowdsourced sensitive training data. Specifically, we correspondingly estimate the importance of each feature related to target categories and follow the principle that less noise is injected into the more important feature to ensure the data utility of the model. Moreover, we design an adaptive coefficient for the added noise to accommodate the heterogeneous feature value ranges. Theoretical analysis proves that DNN-DP preserves ε-differentially private in the computation. Moreover, the simulation based on the US Census data set demonstrates the superiority of our method in predictive accuracy compared with other existing privacy-aware machine learning methods.

The papers in this special section focus on new methodologies and applications in artificial intelligence (AI) and machine learning (ML). With the recent development of machine learning (ML), artificial intelligence (AI) and cyber technologies in the field of industrial informatics, it is important to migrate the traditional businesses and services in the physical world to the digital cyber-enabled world. Cyber intelligence technologies, such as the fifth-generation (5G) mobile communication network, big data, Internet of Things (IoT), cloud computing, cognitive computing, ubiquitous computing and blockchains, enable goods, houses, services, information, and capitalization to be shared through the Internet of the Appendix]. Industrial applications, including various mechanical systems, utilities, supply chains, energy systems, power grids, infrastructures, manufactures, traffics, healthcare, and environmental issues, are partially operated or managed remotely with the influence of AI and cyber intelligence technology.

Crowdsourcing has proved to be a splendid tool to aggregate the knowledge from a pool of individuals in order to perform abundant microtasks efficiently. Recently, with the explosive growth of online social network, Word of Mouth (WoM)-based crowdsourcing systems have emerged, in which besides conducting the tasks by themselves, participants simultaneously recruit other individuals through exploiting their social networks to help solve crowdsourced tasks. This crowdsourcing paradigm can greatly facilitate to grow the pool of crowdworkers. However, there exist two conflicting challenges in designing an effective WoM-based incentive mechanism: 1) sybil attack and 2) heterogeneous effect of participants. That is, intuitively, incentivizing (usually compensating for) common-ability individuals will inevitably stimulate the behavior of sybil attack (i.e., some individuals create multiple sybils, and split the total efforts into those sybils to expect more compensation). This paper proposes a novel biased contest-based crowdsourcing incentive mechanism through exploiting social networks (BciNet), aiming to balance those two conflicting objectives. BciNet is composed of two phases. First, based on spreading activation model, an enhanced geometric virtual point dissemination mechanism is able to provide sybil-proof property and accommodate the realistic social network structure. Second, based on participants' virtual points, a biased contest gives more reward to less able participants. Through carefully calibrating the bias factor, simulation results based on the real dataset show that BciNet can greatly improve the amount of participants' effort levels, and actually be robust against the sybil attack. In brief, for a practical incentive mechanism, the methodology to address conflicting goals is to put rational individuals into dilemma: to be sybil or not to be, it is the problem, i.e., the potential gain from the sybils in the second phase may be offset by the loss in the first phase.

Pulse diagnosis is one of the diagnostic methods in traditional Chinese medicine (TCM). Such diagnoses are made subjectively by a TCM doctor, who requires expert knowledge. If pulse diagnosis could be automated, it would be beneficial for health management. In our previous study, we showed that pulse diagnosis might be related to personal health data, such as step count and sleep score. In this study, we propose a new approach to classifying pulse diagnoses based on a combination of features from pulse and health data. Pulse characteristics are extracted from electronically recorded pulse shapes, and health data feature analysis is augmented by considering the periodicity of daily health metrics. Using these features, we perform both single- and multi-label classifications, and investigate the possibility to improve classification accuracy. We further adopt two classification methods for multi-label classification: random forests and deep learning. Our results show that our approach improves classification accuracy for pulse diagnoses.

BACKGROUND: Cost-sensitive algorithm is an effective strategy to solve imbalanced classification problem. However, the misclassification costs are usually determined empirically based on user expertise, which leads to unstable performance of cost-sensitive classification. Therefore, an efficient and accurate method is needed to calculate the optimal cost weights. RESULTS: In this paper, two approaches are proposed to search for the optimal cost weights, targeting at the highest weighted classification accuracy (WCA). One is the optimal cost weights grid searching and the other is the function fitting. Comparisons are made between these between the two algorithms above. In experiments, we classify imbalanced gene expression data using extreme learning machine to test the cost weights obtained by the two approaches. CONCLUSIONS: Comprehensive experimental results show that the function fitting method is generally more efficient, which can well find the optimal cost weights with acceptable WCA.

The Internet of Things (IoT)-Cloud combines the IoT and cloud computing, which not only enhances the IoT's capability but also expands the scope of its applications. However, it exhibits significant security and efficiency problems that must be solved. Internal attacks account for a large fraction of the associated security problems, however, traditional security strategies are not capable of addressing these attacks effectively. Moreover, as repeated/similar service requirements become greater in number, the efficiency of IoT-Cloud services is seriously affected. In this paper, a novel architecture that integrates a trust evaluation mechanism and service template with a balance dynamics based on cloud and edge computing is proposed to overcome these problems. In this architecture, the edge network and the edge platform are designed in such a way as to reduce resource consumption and ensure the extensibility of trust evaluation mechanism, respectively. To improve the efficiency of IoT-Cloud services, the service parameter template is established in the cloud and the service parsing template is established in the edge platform. Moreover, the edge network can assist the edge platform in establishing service parsing templates based on the trust evaluation mechanism and meet special service requirements. The experimental results illustrate that this edge-based architecture can improve both the security and efficiency of IoT-Cloud systems.

Emerging cyber technology permeates not only our working environment but also our daily life. We enjoy various cyber-enabled online services, e.g., using smartphones for navigation and friendships via social network services (SNS) and doing business through the cyberspace, where a large amount of human-centric data are produced in both the man–man and man–machine systems. This issue aims to enhance the next generation of human-centered social computing and provide relevant theoretical and algorithmic support for the application of social networks. Accepted articles included in this special issue cover topics on human-centric big data analysis, social influence analysis, data-driven interdisciplinary modeling and analysis, user behavior modeling and analysis, cyber-enabled and artificial intelligent (AI)-enhanced healthcare services, and so on. These contents may draft promising perspectives for future research and demonstrate a variety of issues and solutions related to social computing. With the AI-enhanced technical analysis, these human-centric data can be effectively used to better serve humanity.

With the rapid development of Internet of Things (IOT) technologies, modern power systems have become complex cyber-physical systems. A large number of smart devices have promoted efficient generation, transmission and distribution in the smart grid. State estimation (SE) is one of fundamental components in smart grid that evaluates the operation state of a grid by using a set of sensor measurements and grid topologies. A major issue is the authenticity of the measurements collected by the sensors. Specifically, the false data injection attack (FDIA) aims to temper the information that reflect the grid operation state. In this paper, we propose an efficient prediction-based FDIA detection and location scheme, PDL, in which the state vector of smart grid can be represented as multivariate time series, and can be predicted by vector autoregressive processes (VAR) through intentionally exploiting the temporal and spatial correlations of states. Different from most previous works which assumed the state transfer matrix constant and diagonal, a time-varying and non-diagonal matrix is adopted in this scheme. Then, the consistency between the predicted measurements and the observed measurements is utilized to detect and locate abnormal data. Besides, the detected abnormal data can be replaced with the predicted data, which simplifies the calibration process. Extensive simulation results verify the performance of the proposed scheme.

While social computing related research has focused mostly on how to provide users with more precise and direct information, or on recommending new search methods to find requested information rapidly, the authors believe that network users themselves could be viewed as an important social resource. This study concentrates on analyzing potential and dynamic user correlations, based on topic-aware similarity and behavioral influence, which may help us to discover communities in social networking sites. The dynamically socialized user networking (DSUN) model is extended and refined to represent implicit and explicit user relationships in terms of topic-aware features and social behaviors. A set of measures is defined to describe and quantify interuser correlations, relating to social behaviors. Three types of ties are proposed to describe and discover communities according to influence-based user relationships. Results of the experiment with Twitter data are used to show the discovery of three types of communities, based on the presented model. Comparison with six different schemes and two existing methods demonstrates that the proposed method is effective in discovering influence-based communities. Finally, the scenario-based simulation of collective decision-making processes demonstrates the practicability of the proposed model and method in social interactive systems.

Summary Crowdsourcing is the generalized act of outsourcing tasks, traditionally performed by an employee or contractor, to a large group of Internet population through an open call. With the great development of smartphones with rich built‐in sensors and multiple ratio interfaces, mixing smartphone‐based mobile technologies and crowdsourcing offers significant flexibilities and leads to a new paradigm called mobile crowdsourcing (MCS), which can be fully explored for real‐time and location‐sensitive crowdsourced tasks. In this paper, we present a taxonomy for the MCS applications, which are explicitly divided as using human as sensors, and exploiting the wisdom of crowd (i.e., human intelligence). Moreover, two paradigms for mobilizing users in MCS are outlined: direct mode and word of mouth mode. A comprehensive MCS framework and typical workflow of MCS applications are proposed, which consist of nine functional modules, pertaining to three stakeholders in MCS: crowdsourcer, crowdworkers, and crowdsourcing platform. Then, we elaborate the MCS challenges including task management, incentives, security and privacy, and quality control, and summarize the corresponding solutions. Especially, from the viewpoints of various stakeholders, we propose the desired properties that an ideal MCS system should satisfy. The primary goal of this paper is to comprehensively classify and provide a summary on MCS framework, challenges, and possible solutions to highlight the MCS related research topics and facilitate to develop and deploy interesting MCS applications. Copyright © 2016 John Wiley & Sons, Ltd.

Nowadays, most popular social networking services adopt centralized architecture, in which continual Internet connectivity is prerequisite for each user to exploit those services, and centralized servers are used for storage and processing of all application/context data, even though mobile users are within proximity area (like campus, event spot, and community), and can directly exchange media through various wireless technologies (e.g., Bluetooth, Wi-Fi Direct, etc.). On one hand, the omniscient centralized server may cause serious privacy concern, due to the fact that it collects and stores all users' data (messages, profiles, location, relations, etc.), On the other hand, transmitting a large amount of media generated by users in proximity through Internet, would not only bring a lot of pressure to the network infrastructure and service provider, but incur heavy data traffic cost to users. In this paper, our contributions are twofold. First, we propose a Wi-Fi Direct based P2P social networking framework, which enables direct data exchange among users without using infrastructure network when users are located in proximity, and provide solutions to two core problems in this framework, i.e., discoverability and privacy. Second, the prototype of this framework is preliminarily implemented in Android, which is composed of the following functions: localization based on Google Geocoding, Chat, and file-sharing component supporting intermittent transmission.

Sharing resources and information on the Internet has become an important activity for education. In distance learning, instructors can benefit from resources, also known as Learning Objects (LOs), to create plenteous materials for specific learning purposes. Our repository (called the MINE Registry) has been developed for storing and sharing learning objects, around 22,000 in total, in the past few years. To enhance reusability, one significant concept named Reusability Tree was implemented to trace the process of changes. Also, weighting and ranking metrics have been proposed to enhance the searchability in the repository. Following the successful implementation, this study goes further to investigate the relationships between LOs from a perspective of social networks. The LONET (Learning Object Network), as an extension of Reusability Tree, is newly proposed and constructed to clarify the vague reuse scenario in the past, and to summarize collaborative intelligence through past interactive usage experiences. We define a social structure in our repository based on past usage experiences from instructors, by proposing a set of metrics to evaluate the interdependency such as prerequisites and references. The structure identifies usage experiences and can be graphed in terms of implicit and explicit relations among learning objects. As a practical contribution, an adaptive algorithm is proposed to mine the social structure in our repository. The algorithm generates adaptive routes, based on past usage experiences, by computing possible interactive input, such as search criteria and feedback from instructors, and assists them in generating specific lectures.

Identity fraud has become one of major concerns for broad communities. The new information era needs a new digital identity infrastructure to support next generation Internet. This article suggests a hierarchical structure for digital identities. We define and classify all digital identities into three broad categories: Object, People and Organization. This paper is the first to systematically address the classification of digital identities. More and more individuals and communities heavily rely on the network. Many countries are trying their own digital identity initiatives, like E-passport, national smart card, etc. This article intends to initiate a discussion on a universal digital identity infrastructure for our future. We believe that in the near future all paper-based identities will be replaced by digital identities. A robust digital identity infrastructure will take a vital role in the future information age.

Service computing aims to provide IT and computing resources to users in a way that it simply serves them. However, many issues remain in, such as portability, interoperability, and heterogeneity of diverse services, in addition to service modeling, creation, deployment, discovery, recommendation, composition, and delivery, in order to provide a service that best fits user needs and contexts at the time. In this study, we propose a new model of flowable services. Its primary purpose is to realize seamless integration and provision of diverse services in an intuitive "flowable" way maximally close to the "flow" of users' activities and thoughts, and as pertinent as possible to their needs, thus gain the most satisfaction. In this paper, we propose and discuss a human-centric framework of flow able services toward service assurance. We further describe the evaluation of the model, and an application scenario for adaptive delivery of personalized learning service under the proposed framework.

Context is the information, which is created and obtained from the surrounding environment for the interaction between humans and computational services. A generic model is a key accessor to the context in any context-aware applications for ubiquitous computing. In the past decades, a number of context modeling techniques have been proposed e.g. markup scheme based, logic-based, graphical, and ontology-based. Since ontology in its nature is a promising tool to specify concepts and interrelations, it has been widely adopted in context modeling. However, in the rapid changing environments, semantics may vary according to the time factors and dynamic group of users. In this paper, we propose an ontology-based context model with temporal vector space in order to complement this deficiency.

We proposed a collaborative information sharing environment based on P2P networking technology, to support communication among special groups with given tasks, ensure fast information exchange, increase the productivity of working groups, and reduce maintenance and administration costs in our previous work. However, for a social growing community, not only the information exchange/sharing functions are necessary, but also solutions to support users with idea and knowledge publication tools for private purpose or public use are essential. Some private message (personal idea and experience) posting tools (e.g., Weblog) and group collaborative knowledge editing tools (e.g., Wikis) are used in practice; the merits of these tools have been recognized. In this paper, we propose a scalable information sharing solution, which integrates decentralized P2P networking with centralized personal/group media tools. This solution combines the effective tools, such as Weblog and Wiki, into P2P-based collaborative groupware system, to facilitate infinite, growing and scalable information management and sharing for individuals and groups.

This paper describes the computer supported ubiquitous learning system and the social interaction between learners. We define the ubiquitous learning with five prime attributes, and present a generalized social interaction support model for the ubiquitous learning. Moreover, in order to support learners with increasing social skill, a solution for constructing social interaction in ubiquitous learning environment is designed, which includes three major functions; encounter, communication and collaboration support functions.

This paper outlines a new model of ubiquitous computing technology support learning (uLearning). In the uLearning, in order to facilitate the awareness of other helpful learners, increase the availability of the mutual and friendly support in the learning process, and enhance learners’ communication skills, the authors offer a serial of services. These services are designed as dynamic group construction service, social intercommunion facilitation service, service of navigating learner to get out of difficulty, etc.

This paper presents a multiagent system to assist a teacher managing his/her course contents placed on Web servers. In this system there is a set of agents and every agent may work independently from or collaboratively with others. Once generated, an agent can reside in a teacher's daily working computer (called administration host) or a proxy host, and can move between the two hosts. Each agent is devoted to one piece of job and all of them, as a whole, coordinately conduct a sequence of management work during the entire process of teaching a course. A teacher may administrate agents via a specific system shell on his/her administration host, or a usual Web browser on another computer/PDA/mobile phone. The system has been carefully modularized, and thus a new type of agent, if necessary, can be relatively easily developed and quickly incorporated into the system to further enhance or extend its management capability.

In this study, we are trying to develop a general-purpose groupware system for uniform open groups that integrate in itself all main tools necessary for collaboration and cooperation online to achieve common goals at shorter times, ensure fast information transfer and decrease the amount of necessary administration. The system consists of a number of tools that reflect actions of one member at all the nodes of the system, giving awareness for collaboration and increasing its effectiveness. In comparison with other applications that have been designed for knowledge sharing, our system pushes necessary information through the network so that no manual search and download are necessary if one of the group members possesses it. Thus the speed of training, knowledge and information sharing is accelerated and participation of the group members will be increased. By developing the system, we are trying to provide shared workspace for all main platforms as well.

In this paper, the authors introduce a framework for every-netizen online learning communities that widely open to anyone who is willing to learn and share knowledge with others across the networks. They discuss how to provide individualized collaboration support for e-learning in the community, and demonstrate a prototype of implementation in a Web-based social virtual environment with two supporting tools.

In recent years, we have encountered a serious problem that a huge amount of information is flooding on the Internet. There is a growing desire to have an effective way to utilize information on the Internet. We propose a framework for collecting, organizing and reusing information through group collaboration: collaborative information browser. We further discuss the design and implemention of a prototype for a collaborative information browser.