Multimedia content offers additional context for recommender systems to better understand user interests. Existing studies on multimodal recommendation primarily focus on constructing item-item semantic graphs. However, most of these methods capture only shallow semantic structures based on feature similarity and struggle to model more complex or cross-entity semantic relationships (e.g., user-item). Moreover, in these methods, collaborative signals often dominate and suppress semantic knowledge, which limits its role in representation learning. To address these issues, we propose SCALE, a novel framework that combines subspace-aware graph construction and contrastive alignment for multimodal recommendation with large language models. Specifically, we first use large language models and encoders to extract user and item features. Following the subspace clustering assumption, we apply the Orthogonal Matching Pursuit algorithm to mine complex semantic structures within the item-item, user-user, and user-item spaces, and integrate them into a unified semantic graph. We then perform graph convolution on both the semantic and interaction graphs, and aggregate the results for recommendation. Furthermore, contrastive losses are employed to enhance semantic fusion and alignment. Extensive experiments on five real-world datasets demonstrate that SCALE significantly outperforms state-of-the-art multimodal recommendation models, highlighting its effectiveness in modeling complex relationships and integrating semantic knowledge with collaborative signals.

Vision-Language Models (VLMs) have demonstrated remarkable capabilities in visual classification tasks. Existing methods for enhancing VLMs on this task often rely heavily on direct category-to-image matching, which limits generalization and results in suboptimal performance. In addition, these methods provide no understanding of why a specific category is chosen. To address these limitations, we introduce a new deliberative visual classification task that decomposes the classification process into multiple deliberative steps and leverages Large Language Models (LLMs) to perform explicit reasoning before the final decision. Specifically, we propose a Retrieval-driven Reasoning model (RdR) with two components, i.e., retrieval database construction and deliberative category prediction. The first component leverages LLMs to extract category-relevant descriptors and constructs a retrieval database for effective image–descriptor matching. The second component facilitates multiple deliberative steps and performs explicit reasoning based on the retrieved descriptors to augment the category prediction. Extensive experiments on multiple datasets demonstrate that RdR consistently outperforms strong baselines, highlighting its robustness and generalization ability.

With the rapid growth of multi-modal content on the Web, robust vision-language models are essential for semantic understanding and classification of web images under diverse and dynamic contexts, supporting Web applications such as multimedia search and recommendation. Prompt learning has proven effective for enhancing vision-language models in semantic image classification tasks. However, previous methods often suffer from poor generalization: the learned prompts tend to overfit the base classes seen during training, leading to poor performance on unseen classes and under distribution shifts. This issue is especially challenging in Web-scale data, where new classes emerge and distributions shift dynamically. To address these limitations, we propose PLIKD, a novel prompt learning method that integrates instance-aware knowledge distillation for robust Web-scale semantic image classification. Specifically, PLIKD introduces an instance-aware knowledge extraction module, which leverages multi-modal large language models through a step-by-step strategy to extract external knowledge for each image instance. To incorporate this extracted knowledge, PLIKD further introduces an instance-aware knowledge distillation module, which consists of two key steps: (1) a dual-teacher strategy for robust and informative knowledge distillation, and (2) fine-grained cross-modal alignment via Smooth and Sparse Optimal Transport. Extensive experiments demonstrate that PLIKD significantly improves generalization to both seen and unseen classes, and remains robust under distribution shifts, outperforming existing state-of-the-art methods on Web-scale semantic image classification.

IIoT sensors collect large volumes of vibration time-series data at high sampling rates. These data are nonstationary and multi-scale and are essential for condition monitoring across different devices and operating conditions. Deep compression sensing reduces data volume while preserving critical information, enabling efficient and low-cost data processing in IIoT systems. However, existing methods for industrial time-series data processing struggle to preserve features effectively under stringent bandwidth and storage constraints. Moreover, most deep compression sensing models overlook computational limitations and robustness demands in noisy industrial settings. Therefore, we propose an Attention-Based Deep Compressed Sensing with Multi-Loss Optimization for IIoT Vibration Data (DCS-AMTD). The model achieves efficient compression and high-quality reconstruction while improving both resource efficiency and noise robustness. Specifically, we design a dual-path convolutional module that incorporates dilated convolutions to capture multi-scale local and global features. We design a one-dimensional convolutional block attention module (CBAM1D) for industrial vibration signals to dynamically reweight multi-scale features and enhance discriminative representations. Furthermore, we design a joint time-frequency loss with multi-domain constraints to improve reconstruction quality under strict bandwidth and computational constraints. Experiments on the Case Western Reserve University (CWRU) and Paderborn (PB) datasets demonstrate that our method achieves superior reconstruction performance across various compression ratios, outperforming existing approaches.

With the rapid development of ubiquitous networks and unmanned devices, air quality monitoring data is increasingly collected via wireless networks from sensors at multiple monitoring stations. However, the complexity of these data introduces significant challenges. Existing spatiotemporal methods for air quality prediction often struggle with issues such as inadequate handling of spatial relationships, difficulties in modeling long-term temporal dependencies, and limited generalization capabilities. To address these challenges, this paper proposes a novel spatiotemporal modeling approach—Spatiotemporal λ-Bernstein Graph Convolutional Network with Transformer for Multi-Site Air Quality Prediction (ST-BernT). This method constructs a graph structure based on spatiotemporal correlations, integrating a Gaussian-weighted adjacency matrix derived from geographic distances with an adjacency matrix capturing the temporal correlations of pollutant concentration time series, thereby precisely modeling spatial dependencies. Subsequently, a dynamic filter adjusted by λ-Bernstein polynomials is proposed to adaptively process spatiotemporal data characterized by the coexistence of low- and high-frequency components on the graph. Furthermore, a hierarchical generative transformer (GPHT) is introduced to enhance the model’s ability to capture long-term temporal patterns, such as periodicity and seasonality, while supporting parallel prediction across multiple sites, significantly improving computational efficiency and accuracy. Experimental results demonstrate that ST-BernT exhibits strong accuracy, adaptability, and generalization capability in multi-site air quality prediction tasks, particularly showing enhanced robustness in large-scale long-term forecasting scenarios.

Deep learning has shown great potential in assisting medical experts in diagnosing cerebral infarction through electroencephalogram (EEG) analysis. Classifying EEG abnormality levels is a critical component of this process. However, the inherent complexity of raw EEG signals makes reliable feature extraction challenging, while the scarcity of severe cases leads to substantial class imbalance. The limited size of available datasets further exacerbates these issues, as excessive dependence on generative data may cause classifier overfitting. To address these challenges, we propose a novel framework that integrates a diffusion-based generative model with a hybrid embedding-based few-shot learning strategy for EEG abnormality classification. The framework first extracts a comprehensive set of entropy-based and statistical features from EEG waveforms. The diffusion model incorporates class labels into the noise prediction mechanism, embedding label information into the timestep representation to generate high-fidelity, class-consistent synthetic feature vectors. The hybrid embedding-based few-shot classifier is trained using an episodic learning strategy that optimizes latent-space feature distances under limited data. During inference, the classifier employs a two-step decision procedure in which a preclassifier first estimates the two most probable candidate classes. The prototypical module then performs refined metric-based discrimination within this reduced candidate space, which improves both robustness and generalization. In addition, combining EEG abnormality levels with quantitative clinical scale scores enables more rapid and reliable auxiliary assessment of cerebral infarction. Experimental results demonstrate that the proposed framework yields substantial improvements in EEG abnormality classification accuracy and provides meaningful support for clinical decision-making.

Multimodal emotion recognition (MER) has been widely adopted in affective computing and human–computer interaction. However, real-world multimodal streams frequently suffer from missing or incomplete modalities due to sensor failures, privacy constraints, and heterogeneous acquisition costs, which often causes severe performance degradation for models trained under complete inputs. To address this issue, we propose hierarchical VAE–diffusion for emotion reconstruction (HVDER), a latent-space generative framework for incomplete MER. HVDER first employs modality-specific variational autoencoders (VAEs) to project language, visual, and audio features into a unified low-dimensional latent space with KL-regularized structure. Then, a two-stage coarse-to-fine conditional diffusion module completes missing modality latents by recovering global emotion semantics followed by refining local discriminative details. Finally, the completed and observed modality representations are fused for emotion prediction, optimized end-to-end with a multiobjective loss integrating reconstruction, diffusion matching, cross-modal alignment, and classification supervision. Extensive experiments on CMU-MOSEI and CMU-MOSI validate the effectiveness and robustness of HVDER. Under fixed modality-missing settings, HVDER achieves average ACC2/F1 of 76.8/75.4 on CMU-MOSEI and 73.4/72.9 on CMU-MOSI, outperforming representative baselines across all modality combinations. Under random missing with a high missing rate of MR = 0.7, HVDER maintains ACC2/F1 of 75.4/72.5 on CMU-MOSEI and 68.0/67.2 on CMU-MOSI, indicating a stronger performance lower bound in severely incomplete scenarios. Ablation studies further confirm that latent-space modeling and the coarse-to-fine diffusion design jointly contribute to the main performance gains.

Scene graph generation (SGG) plays an important role in the intelligence of social things (IoST) framework by extracting structured semantic representations from social device data, thereby supporting advanced scene understanding and behavioral-cultural modeling. However, the intrinsic long-tail nature of real-world social device data, coupled with the semantic entanglement between head and tail categories (e.g., “on” versus “standing on”), presents significant challenges for fine-grained SGG. This often results in biased models and suboptimal generalization to rare but semantically informative relations. To address these issues, we propose a novel cooperative dual classifier (CDC) framework for fine-grained SGG in IoST-driven social systems. CDC introduces a cooperative learning mechanism that combines two classifiers. The frozen prototype classifier is designed with maximum interclass margins to alleviate class imbalance. In parallel, a learnable classifier dynamically adjusts decision boundaries to improve discriminative precision. To further enhance the integration between the two classifiers, we introduce a weight knowledge transfer (WKT) module and a collaborative constraint term, facilitating robust adaptation to tail categories. Extensive experiments on the Visual Genome and GQA datasets demonstrate that CDC outperforms state-of-the-art SGG methods, particularly in modeling fine-grained relations under long-tail distributions. These results highlight the capability of CDC to advance semantic understanding of complex behavioral and cultural patterns within computational social systems.

Manual annotation for crowd counting remains labor-intensive and costly. Although existing semi-supervised methods partially alleviate this burden, they still face significant challenges regarding the quality of generated pseudo-labels and the utilization of unlabeled data. To address these issues, we propose a novel semi-supervised crowd counting framework, called Point-Adaptive Teacher (PAT). This framework integrates Adaptive Soft Threshold (AST) and contrastive learning to enhance pseudo-label quality and effectively leverage unlabeled data. Specifically, we employ the Swin Transformer as the backbone and develop Swin-P2PNet, which captures global contextual information through hierarchical window attention, improving the accuracy of pseudo-labels. Additionally, we design the AST that dynamically adjusts the sample loss weight by combining confidence and uncertainty predictions, thereby alleviating the effect of noise in pseudo-labels. Finally, we introduce a contrastive learning strategy requiring no extra parameters. This strategy enhances the model’s ability to learn latent representations from unlabeled data. Extensive experiments have been conducted on three public datasets, namely ShanghaiTech, JHU-Crowd++, and UCF-QNRF. The results demonstrate that our method achieves performance comparable to state-of-the-art methods.

Sequential recommendation aims to capture the temporal dependencies of items in a user's historical interactions and make recommendations based on this. Previous generative methods addressed the issue of data not directly reflecting user preference uncertainty by modeling the distribution of latent item representations. Diffusion model (DM)-based methods have achieved significant success due to their high-quality generation and stable training. However, they lack satisfactory user sequence representations to guide the generation process, impacting recommendation performance. Moreover, these methods overlook the drawback of slow inference speed, severely limiting their practical value. To obtain effective generative guidance signals and accelerate the recommendation process, we propose DAE4Rec. In this approach, a Graph Auto-Encoder (GAE) is used to obtain interpretable item node representations, revealing global transitions of items that previous methods struggled to uncover. Then, we use it to construct a generative guidance signal with lower coupling and variance for the diffusion model. Additionally, by employing a non-Markov chain derived from the forward diffusion process, it is the first to implement a 'skip-step' reverse process in diffusion model-based methods. And a creatively designed compensator is used to bridge the performance gap caused by 'skip-step'. Extensive experiments on three real-world datasets demonstrate that DAE4Rec outperforms other state-of-the-art generative sequential recommenders.

The industrial Internet integrates industrial systems with Internet technologies, significantly enhancing production efficiency and reducing costs through collaboration with intelligent devices. Under the context of the industrial Internet, a production process typically comprises multiple tasks, each task requiring one or more types of production factors, which makes the service composition of production factors more complex and challenging compared to the traditional service composition under Web environments. To optimize the service composition of production factors under industrial Internet, this paper proposes a collaboration-aware service composition optimization model by considering the collaboration relationships among services. Specifically, both historical collaboration and present collaboration are considered, where the historical collaboration relationships among services include the collaboration between two directly-followed tasks and collaboration within one task, and the present collaboration relationships include the collaboration among the selected services in present service composition. To derive the optimal service composition plan, we propose using the advanced teaching-learning-based optimization algorithm (TLBO). Extensive experiments are conducted to compare with other population-based optimization methods under a real-world ship production process to verify the superiority of our approach. Experimental results show that the integration of collaboration relationships improves the overall quality of the optimized service composition plans in terms of time and cost, and the selected TLBO is more effective to derive the optimized service composition plans than other population-based optimization methods.

In today's information-rich society, where users are inundated with vast multi-sourced data, Recommendation system which may focus more on users' lifestyle preferences becomes increasingly significant, especially when providing well-being oriented daily living support. However, it still relies on limited data, overlooking complex relationships between user context, behavior, and item attributes, which often fails to provide truly personalized suggestions, and lifestyle-aware optimization remains underdeveloped. In this study, we propose a Graph Neural Network (GNN)-enhanced Multimodal Fusion with Contrastive Learning model, called GNNMF-CL, which can be applied to pursue high performance in smart systems based on multimodal learning from integrated visual, text, and user's lifestyle information etc. Specifically, a food recommendation system for smart health is designed and built, which extracts visual features using ResNet-50, encodes lifestyle features through contrastive learning based on VLM-generated captions and nutritional information, and captures taste preference features via Graph Attention Network (GAT). In addition, a Heterogeneous Graph Transformer (HGT) is developed to improve the multimodal feature fusion, and the five loss functions are defined to jointly optimize high-order neighborhood relations and ranking of recommended items in model training. Experiments and evaluations on the AllRecipes dataset demonstrate that our proposed model significantly outperforms other existing methods, in achieving high performance recommendations that align with users' health goals and contribute to broader applications in the smart health domain.

Federated learning (FL) enables collaborative model training without exposing raw data, offering a privacy-aware alternative to centralized learning. However, FL remains vulnerable to various privacy attacks that exploit shared model updates, including membership inference, property inference, and gradient inversion. Source inference attacks further threaten FL by identifying which client contributed a specific training sample, posing severe risks to user and institutional privacy. Existing source inference attacks mainly assume passive adversaries and overlook more realistic scenarios where the server actively manipulates the training process. In this paper, we present an enhanced source inference attack that demonstrates how a malicious server can amplify behavioral differences between clients to more accurately infer data origin. Our approach introduces active training manipulation and data augmentation to expose client-specific patterns. Experimental results across five representative FL algorithms and multiple datasets show that our method significantly outperforms prior passive attacks. These findings reveal a deeper level of privacy vulnerability in FL and call for stronger defense mechanisms under active threat models.

The rapid growth of multimedia content has driven the development of recommender systems. Most previous work focuses on uncovering latent relationships among items to learn better representations. However, this approach does not sufficiently account for user affinities, potentially leading to an imbalance in the structure modeling of users and items. Moreover, the sparsity and imbalance of user-item interactions further hinder effective representation learning. To address these challenges, we propose a framework called BLAST, which balances structures and interactions via large language models and optimal transport for multimodal recommendation. Specifically, we utilize large language models to summarize side information and generate user profiles. Based on these profiles, we design an intra- and inter-entity structure balancing module to capture item-item and user-user relationships, integrating these affinities into the final representations. Furthermore, we impose constraints on negative sample selection, augment the training data with false negative items and the optimal transport algorithm, thereby leading to smoother interactions. We evaluate BLAST on three real-world datasets, and the results demonstrate that our method significantly outperforms state-of-the-art baselines, which validates the superiority and effectiveness of BLAST.

Recommender systems play a critical role in many applications by providing personalized recommendations based on user interactions. However, it remains a major challenge to capture complex sequential patterns and address noise in user interaction data. While advanced neural networks have enhanced sequential recommendation by modeling high-order item dependencies, they typically assume that the noisy interaction data as the user's preferred preferences. This assumption can lead to suboptimal recommendation results. We propose a Variational Graph Auto-Encoder driven Graph Enhancement (VGAE-GE) method for robust augmentation in sequential recommendation. Specifically, our method first constructs an item transition graph to capture higher-order interactions and employs a Variational Graph Auto-Encoder (VGAE) to generate latent variable distributions. By utilizing these latent variable distributions for graph reconstruction, we can improve the item representation. Next, we use a Graph Convolutional Network (GCN) to transform these latent variables into embeddings and infer more robust user representations from the updated item embeddings. Finally, we obtain the reconstructed user check-in data, and then use a Mamba-based recommender to make the recommendation process more efficient and the recommendation results more accurate. Extensive experiments on five public datasets demonstrate that our VGAE-GE model improves recommendation performance and robustness.

Sequential recommendation generates embedding representations from historical user-item interactions to recommend the next potential interaction item. Due to the complexity and variability of historical user-item interactions, extracting effective user features is quite challenging. Recent studies have employed sequential networks such as time series networks and Transformers to capture the intricate dependencies and temporal patterns in historical user-item interactions, extracting more effective user features. However, limited by the scarcity and suboptimal quality of data, these methods struggle to capture subtle differences in user sequences, which results in diminished recommendation accuracy. To address the above issue, we propose a contrastive learning framework with LLMs-based view augmentation (CLLMRec), which effectively mines differences in behavioral sequences through sample generation. Specifically, CLLMRec utilizes LLMs (Large Language Models) to augment views and expand user behavior sequence representations, providing high-quality positive and negative samples. Subsequently, CLLMRec employs the augmented views for effective contrastive learning, capturing subtle differences in behavioral sequences to suppress interference from irrelevant noise. Experimental results on three public datasets demonstrate that the proposed method outperforms state-of-the-art baseline models, and significantly enhances recommendation performance.

Test-time adaptation (TTA) offers the potential to enhance model generalizability without relying on training data or retraining processes. However, TTA faces challenges under covariate shift, where discrepancies between the distributions of training and testing phases hinder model performance. This limitation stems from the fact that existing methods usually rely heavily on training data and fail to establish a good connection between the model and the marginal distribution of test data, resulting in reduced generalization ability. To mitigate this issue, we introduce a novel self-supervised framework that integrates latent score matching and pseudo-label refinement into the TTA paradigm to enhance the model's perception of the test data distribution. Our approach, Joint Test-time Adaptation with Refined Pseudo-labels and Latent Score Matching, reinterprets a classifier as a score estimator and trains it using pseudo-label refinement. This enables the model to better align with the test distribution through latent score matching, while simultaneously preserving discriminative performance via pseudo-label refinement. Extensive experiments across diverse architectures and benchmarks demonstrate that TAPS consistently outperforms state-of-the-art methods in terms of generalization performance under various distribution shifts.

The unmanned aerial vehicle (UAV) resupply mode offers a viable alternative to parcel delivery by replenishing ground vehicles at intermediate open-space supply points. Usually, orders are treated as equally important with no guarantees of on-time delivery. In this paper, we propose a priority-based delivery framework to efficiently manage real-time delivery demands with varying priorities. Specifically, this framework integrates a fleet of UAVs for resupply and trucks for final delivery. By considering the inherent priority of orders and their elapsed waiting time, a dynamic priority discipline is introduced and customized to different order contexts. This approach ensures that urgent orders are prioritized to ensure timely fulfillment within their deadlines, while low-prioritized orders can still be completed without excessive waiting. Simulations using a real-world city map of Helsinki, along with mobility features of both trucks and UAVs, show significant performance gains over baseline schemes in terms of on-time delivery rate, average delivery time, and waiting fairness among all orders.

Next Point-of-Interest (POI) recommendation has become a crucial task in Location-Based Social Networks (LBSNs), which provide personalized recommendations by predicting the user's next check-in locations. Commonly used models including Recurrent Neural Networks (RNNs) and Graph Convolutional Networks (GCNs) have been widely explored. However, these models face significant challenges, including the difficulty of capturing the hierarchical and tree-like structure of POIs in Euclidean space and the sparsity problem inherent in POI recommendations. To address these challenges, we propose a Hyperbolic Variational Graph Auto-Encoder (HVGAE) for next POI recommendation. Specifically, we utilize a Hyperbolic Graph Convolutional Network (Hyperbolic GCN) to model hierarchical structures and tree-like relationships by converting node embeddings from euclidean space to hyperbolic space. Then we use Variational Graph Auto-Encoder (VGAE) to convert node embeddings to probabilistic distributions, enhancing the capture of deeper latent features and providing a more robust model structure. Furthermore, we combine the Mamba4Rec recommender and Rotary Position Embedding (RoPE) and propose Rotary Position Mamba (RPMamba) to effectively utilize POI embeddings rich in sequential information, which improves the accuracy of the next POI recommendation. Extensive experiments on three public datasets demonstrate the superior performance of the HVGAE model.

In the trend of continuously advancing urban intelligent transport construction, traditional traffic signal control (TSC) struggles to make effective decisions with complex traffic conditions. Although multi-agent deep reinforcement learning (MARL) shows promise in optimizing traffic flow, most existing studies ignore the complex relationships between signal lights and fail to communicate with neighbors effectively. Moreover, the deterministic strategies generated by Q-learning-based methods struggle to be extended to large-scale urban road networks. Therefore, this paper proposes a multi-agent graph-based soft actor-critic (MAGSAC) approach for TSC, which combines graph neural networks with the Soft Actor-Critic (SAC) algorithm and extends it to multi-agent environments to address the TSC problem. Specifically, we employ graph-based networks and attention mechanism to expand the receptive domain of agents, enable environmental information to be shared among agents, and utilize the attention mechanism to filter out unimportant information. The algorithm adheres to the Centralized Training Decentralized Execution (CTDE) paradigm to minimize the non-stationarity of MARL. Finally, a rigorous experimental evaluation was conducted using the CityFlow simulator on both synthetic traffic grids and real-world urban road networks. Experimental results show that MAGSAC outperforms other TSC methods in performance metrics, including average queue length and waiting time, and achieves excellent performance under complex urban traffic conditions.

In the framework of 6G-driven Symbiotic Internet of Things (Symbiotic IoT), vehicular edge computing systems enhance the inference efficiency of large-scale AI models through collaboration between road side units (RSUs) and in-vehicle terminals. As a core component of symbiotic systems, RSUs face intermittent challenges posed by renewable energy supply when hosting offloaded vehicular deep learning tasks. Existing energy consumption prediction methods rely heavily on massive sampled data and lack a priori evaluation, leading to significant difficulties and risks in acquiring large-scale measured data: when the remaining power of RSUs is insufficient to sustain the sampling period, the prediction process is forced to interrupt, wasting both the operational energy already consumed and the additional scheduling computation energy. This article proposes a two-stage energy consumption prediction method for symbiotic Internet of Vehicles (IoV) edge nodes, addressing the above challenges through a collaborative mechanism of “a priori rough estimation—dynamic precise prediction.” In the first stage, a nonlinear regression model is established based on limited measured data, combined with analysis of model architecture and hardware instruction-level energy consumption characteristics, to enable a priori energy estimation without actual model execution. This supports lightweight decision-making for RSUs during task allocation, reducing computational overhead compared to traditional sampling methods. In the second stage, real-time energy consumption monitoring is used to dynamically update prediction curves, achieving prediction calibration within extremely short operation cycles to balance prediction accuracy with the energy constraints of edge nodes.Experimental results in typical scenarios of IOV applications demonstrate that the proposed a priori evaluation method achieves an average error of 10% under different data volumes and batch sizes, while the real-time prediction method yields short-term and long-term prediction errors below 3% and 6%, respectively—outperforming existing methods in accuracy. This approach effectively assists RSUs in energy management, reducing energy waste caused by the absence of a priori evaluation and prediction errors, and contributes to the efficient utilization of renewable energy in low-latency scenarios, such as connected vehicles.

With the advent of Industrial Internet-of-things (IIoT), flexible manufacturing has gained increasing attention. Continuous changes in market demand, real-time data collection and device interconnectivity have made the production process more dynamic and unpredictable, rendering traditional manufacturing scheduling methods inadequate in addressing these changes. This issue can be framed as a Dynamic Flexible Job Shop Scheduling Problem (DFJSP), which seeks to accommodate ever-evolving production scenarios and requirements by making real-time modifications and optimizing resource allocation. Traditional scheduling algorithms designed for static, single-environment scenarios are increasingly inadequate for handling the growing complexity of production environments. In this context, there is a pressing need for efficient and real-time scheduling algorithms. We propose a Multi-Agent Reinforcement Learning (MARL) algorithm to solve DFJSP, where each device is associated with a corresponding agent, allowing the algorithm to scale flexibly. The complexity and dynamics of scheduling problems introduce additional challenges and complexities in state representation and decision-making. We propose two solutions to alleviate this problem. First, we employ a Heterogeneous Graph Neural Network (HGNN) to capture the relational dependencies between tasks and extract state features. Through multiple feature updates, each agent is enabled to make decisions based on global information. Furthermore, leveraging the inherent Permutation Invariance (PI) and Permutation Equivariance (PE) features of tasks in the waiting queue, we apply hypernetwork techniques to address the issue of dimensionality explosion caused by the excessive state space in scheduling environments. Experiments conducted under various scenario settings demonstrate that our scheduling method can significantly reduce task latency and improve resource utilization.

Encrypted Traffic Classification (ETC) using Deep Learning (DL) faces two bottlenecks: homogeneous network traffic representation and ineffective model updates. Currently, multimodal-based DL combined with the Continual Learning (CL) approaches mitigate the above problems but overlook silent applications, whose traffic is absent due to guideline violations leading developers to cease their operation and maintenance. Specifically, silent applications accelerate the decay of model stability, while new and active applications challenge model plasticity. This paper presents Multi-ARCL, a multimodal adaptive replay-based distributed CL framework for ETC. The framework prioritizes using crypto-semantic information from flows' payload and flows' statistical features to represent. Additionally, the framework proposes an adaptive relay-based continual learning method that effectively eliminates silent neurons and retrains new samples and a limited subset of old ones. Exemplars of silent applications are selectively removed during new task training. To enhance training efficiency, the framework uses distributed learning to quickly address the stability-plasticity dilemma and reduce the cost of storing silent applications. Experiments show that ARCL outperforms state-of-the-art methods, with an accuracy improvement of over 8.64% on the NJUPT2023 dataset.

The rapid development and usage of digital technologies in modern intelligent systems and applications bring critical challenges on data security and privacy. It is essential to allow cross-organizational data sharing to achieve smart service provisioning, while preventing unauthorized access and data leak to ensure end users’ efficient and secure collaborations. Federated Learning (FL) offers a promising pathway to enable innovative collaboration across multiple organizations. However, more stringent security policies are needed to ensure authenticity of participating entities, safeguard data during communication, and prevent malicious activities. In this paper, we propose a Decentralized Federated Graph Learning (FGL) with Lightweight Zero Trust Architecture (ZTA) model, named DFGL-LZTA, to provide context-aware security with dynamic defense policy update, while maintaining computational and communication efficiency in resource-constrained environments, for highly distributed and heterogeneous systems in next-generation networking. Specifically, with a re-designed lightweight ZTA, which leverages adaptive privacy preservation and reputation-based aggregation together to tackle multi-level security threats (e.g., data-level, model-level, and identity-level attacks), a Proximal Policy Optimization (PPO) based Deep Reinforcement Learning (DRL) agent is introduced to enable the real-time and adaptive security policy update and optimization based on contextual features. A hierarchical Graph Attention Network (GAT) mechanism is then improved and applied to facilitate the dynamic subgraph learning in local training with a layer-wise architecture, while a so-called sparse global aggregation scheme is developed to balance the communication efficiency and model robustness in a P2P manner. Experiments and evaluations conducted based on two open-source datasets and one synthetic dataset demonstrate the usefulness of our proposed model in terms of training performance, computational and communication efficiency, and model accuracy, compared with other four state-of-the-art methods for next-generation networking security in modern distributed learning systems.

The traditional image quality assessment (IQA) methods are usually based on convolutional neural networks (CNNs). For these IQA methods using CNNs, limited by the feature size of the fully connected layer, the input image needs be tailored to a pre-defined size, which usually results in destroying the original structure and content of the input image and thus reduces the accuracy of the quality assessment. In this paper, a blind image quality assessment method (named CSPP-IQA), which is based on multi-scale spatial pyramid pooling, is proposed. CSPP-IQA allows inputting the original image when assessing the image quality without any image adjustment. Moreover, by facilitating the convolutional block attention module and image understanding module, CSPP-IQA achieved better accuracy, generalization and efficiency than traditional IQA methods. The result of experiments running on real-scene IQA datasets in this study verified the effectiveness and efficiency of CSPP-IQA.

The advancement of mobile multimedia communications, 5G, and Internet of Things (IoT) has led to the widespread use of edge devices, including sensors, smartphones, and wearables. This has generated in a large amount of distributed data, leading to new prospects for deep learning. However, this data is confined within data silos and contains sensitive information, making it difficult to be processed in a centralized manner, particularly under stringent data privacy regulations. Federated learning (FL) offers a solution by enabling collaborative learning while ensuring privacy. Nonetheless, data and device heterogeneity complicate FL implementation. This research presents a specialized FL algorithm for heterogeneous edge computing. It integrates a lightweight grouping strategy for homogeneous devices, a scheduling algorithm within groups, and a Split Learning (SL) approach. These contributions enhance model accuracy and training speed, alleviate the burden on resource-constrained devices, and strengthen privacy. Experimental results demonstrate that the GSFL outperforms FedAvg and SplitFed by 6.53× and 1.18×. Under experimental conditions with \(\alpha=0.05\) , representing a highly heterogeneous data distribution typical of extreme Non-IID scenarios, GSFL showed better accuracy compared to FedAvg by 10.64%, HACCS by 4.53%, and Cluster-HSFL by 1.16%. GSFL effectively balances privacy protection and computational efficiency for real-world applications in mobile multimedia communications.

As an advanced carrier of on-board sensors, connected autonomous vehicle (CAV) can be viewed as an aggregation of self-adaptive systems with monitor-analyze-plan-execute (MAPE) for vehicle-related services. Meanwhile, machine learning (ML) has been applied to enhance analysis and plan functions of MAPE so that self-adaptive systems have optimal adaption to changing conditions. However, most of ML-based approaches don’t utilize CAVs’ connectivity to collaboratively generate an optimal learner for MAPE, because of sensor data threatened by gradient leakage attack (GLA). In this article, we first design an intelligent architecture for MAPE-based self-adaptive systems on web 3.0-based CAVs, in which a collaborative machine learner supports the capabilities of managing systems. Then, we observe by practical experiments that importance sampling of sparse vector technique (SVT) approaches cannot defend GLA well. Next, we propose a fine-grained SVT approach to secure the learner in MAPE-based self-adaptive systems that uses layer and gradient sampling to select uniform and important gradients. At last, extensive experiments show that our private learner spends a slight utility cost for MAPE (e.g., \(0.77\%\) decrease in accuracy) defending GLA and outperforms the typical SVT approaches in terms of defense (increased by \(10\) – \(14\%\) attack success rate) and utility (decreased by \(1.29\%\) accuracy loss).

Social media platforms such as Weibo and TikTok have become more influential than traditional media. Sentiment in social media comments reflects users’ attitudes and impacts society, making sentiment analysis (SA) crucial. AI driven models, especially deep-learning models, have achieved excellent results in SA tasks. However, most existing models are not interpretable enough. First, deep learning models have numerous parameters, and their transparency is insufficient. People cannot easily understand how the models extract features from input data and make sentiment judgments. Second, most models lack intuitive explanations. They cannot clearly indicate which words or phrases are key for emotion prediction. Moreover, extracting sentiment factors from comments is challenging because a comment often contains multiple sentiment characteristics. To address these issues, we propose a dual-branch combination network (DCN) for SA of social media short comments, achieving both word-level and sentence-level interpretability. The network includes a key word feature extraction network (KWFEN) and a key word order feature extraction network (KWOFEN). KWFEN uses popular emotional words and SHAP for word-level interpretability. KWOFEN employs position embedding and an attention layer to visualize attention weights for sentence-level interpretability. We validated our method on the public dataset weibo2018 and TSATC. The results show that our method effectively extracts positive and negative sentiment factors, establishing a clear mapping between model inputs and outputs, demonstrating good interpretability performance.

With the widespread adoption of multimedia web APIs (API) in web and mobile applications, a substantial proliferation of these APIs is observed. These APIs have streamlined development processes, reducing both time and costs. Nevertheless, identifying the required APIs from the vast array of options has emerged as a significant challenge. Collaborative filtering (CF)-based recommendation technologies have demonstrated their efficiency in presenting developers with potentially useful APIs. However, these methods often suffer from popularity bias, i.e., popular APIs tend to dominate the recommendation lists. This imbalance in recommendation opportunities among APIs hinders the growth of the multimedia API ecosystem. To mitigate the popularity bias produced by CF-based API recommendation methods, this article introduces a novel debiasing strategy that combines a log postprocessing adjustment (LPA) with determinant point process (DPP). Specifically, the LPA is employed during the prediction phase to yield a more balanced set of candidate APIs. Then, DPP is utilized to generate recommendation lists that are not just relevant but also diverse in terms of API popularity. Experimental results reveal that our proposed method surpasses existing state-of-the-art approaches in multimedia API recommendation, excelling in both accuracy and the capability to mitigate popularity bias effectively.

In the context of transportation cyber-physical systems (T-CPS), backdoor attacks leveraging traffic images have emerged as a significant security threat. As T-CPS increasingly relies on visual information, such as real-time images captured by traffic cameras, for tasks like traffic sign recognition and autonomous driving, the risk of image-based backdoor attacks has grown substantially. Although various detection-based defense techniques have shown some success in identifying backdoored models, they often fail to fully eliminate backdoor effects, leaving residual security risks. To address this challenge, we propose a Frequency-Domain Hybrid Distillation (FDHD) method for backdoor defense, which effectively weakens the association between backdoor triggers and target labels by combining distillation mechanisms in both the frequency and pixel domains. Furthermore, we design a loss function that integrates feature reconstruction with adaptive alignment, enhancing the student network’s ability to mimic the teacher network and thereby bolstering the backdoor defense capability. Extensive experiments conducted by FDHD on multiple benchmark datasets against the five latest attacks demonstrate that our proposed defense method effectively reduces backdoor threats while maintaining high accuracy in predicting clean samples. This approach will protect against image-based backdoor attacks in T-CPS and lay the foundation for enhancing future traffic safety.

Recently, For-Hire Vehicles (FHVs) have emerged as major players in Vehicular CrowdSensing (VCS). However, the heterogeneity of tasks issued by Data Requesters (DRs) and the heterogeneity of sensors equipped on FHVs under different Vehicle Platforms (VPs) bring difficulties to task allocation and execution. It can be concluded that it is important to reasonably analyze the relationship among DRs, VPs, and FHVs, as well as to motivate VPs and FHVs to complete sensing tasks. Therefore, taking advantage of the real-time simulation and intelligent decision-making of Digital Twins (DT), this paper proposes a DT-drivenDouble-layerReverseAuctionMethod (DRAM). In the first layer, the reverse auction is established between each DR and VPs, and in the second layer, the reverse auction is established between each VP and FHVs. Meanwhile, we also introduce a sensing fairness index to ensure the sensing balance of different sub-regions and consider it in the DRAM process. Here, the idea of backward induction is used to solve the above problems, with the goal of minimizing the overhead of winning VP and the average overhead of all DRs. Finally, the effectiveness of the DRAM proposed in this paper is verified based on the real data set. Compared with the baseline method, DRAM can reduce the average overhead of DR by about 4%-25%. Meanwhile, in terms of sensing fairness, it can be improved by up to 55%.

In many RFID-enabled applications, objects are classified into different categories, and the information associated with each object's category (called category information) is written into the attached tag, allowing the reader to access it later. The category information sampling in such RFID systems, which is to randomly choose (sample) a few tags from each category and collect their category information, is fundamental for providing real-time monitoring and analysis in RFID. However, to the best of our knowledge, two technical challenges, i.e., how to guarantee a minimized execution time and reduce collection failure caused by missing tags, remain unsolved for this problem. In this paper, we address these two limitations by considering how to use the shortest possible time to sample a different number of random tags from each category and collect their category information sequentially in small batches. In particular, we first obtain a lower bound on the execution time of any protocol that can solve this problem. Subsequently, we present a near-OPTimalCategory information sampling protocol (OPT-C) that solves the problem with an execution time close to the lower bound. Finally, extensive simulation results demonstrate the superiority of OPT-C over existing protocols, while real-world experiments further validate its practicality.

Recently, machine/deep learning techniques are achieving remarkable success in a variety of intelligent control and management systems, promising to change the future of artificial intelligence (AI) scenarios. However, they still suffer from some intractable difficulty or limitations for model training, such as the out-of-distribution (OOD) issue, in modern smart manufacturing or intelligent transportation systems (ITSs). In this study, we newly design and introduce a deep generative model framework, which seamlessly incorporates the information theoretic learning (ITL) and causal representation learning (CRL) in a dual-generative adversarial network (Dual-GAN) architecture, aiming to enhance the robust OOD generalization in modern machine learning (ML) paradigms. In particular, an ITL- and CRL-enhanced Dual-GAN (ITCRL-DGAN) model is presented, which includes an autoencoder with CRL (AE-CRL) structure to aid the dual-adversarial training with causality-inspired feature representations and a Dual-GAN structure to improve the data augmentation in both feature and data levels. Following a newly designed feature separation strategy, a causal graph is built and improved based on the information theory, which can enhance the causally related factors among the separated core features and further enrich the feature representation with the counterfactual features via interventions based on the refined causal relationships. The ITL is incorporated to improve the extraction of low-dimensional feature representations and learn the optimized causal representations based on the idea of "information flow." A dual-adversarial training mechanism is then developed, which not only enables the generator to expand the boundary of feature distribution in accordance with the optimized feature representation from AE-CRL, but also allows the discriminator to further verify and improve the quality of the augmented data for OOD generalization. Experiment and evaluation results based on an open-source dataset demonstrate the outstanding learning efficiency and classification performance of our proposed model for robust OOD generalization in modern smart applications compared with three baseline methods.

With the rapid development of the mobile internet, there is an increasing demand for quick access to effective data. Consequently, more research is focusing on data processing and optimization of recommendation systems in edge computing environments. However, in traditional edge computing environments, recommendation systems typically depend on frequent data query interactions among all edge servers to obtain results, which increases time delays. This issue is further exacerbated by the need to process large amounts of data within edge storage systems. To address this challenge, we propose an efficient recommendation method based on data allocation. Specifically, during the data allocation process, we first extract similar features of users, ensuring that the characteristics of some user data align as closely as possible with the overall user feature distribution. Then, using an improved consistent hashing algorithm, we achieve a uniform data layout, allowing the recommendation system to efficiently and accurately provide recommendations by querying data from the nearest edge server to the user. Finally, extensive experiments on real datasets show that our method significantly reduces time delays and improves the accuracy of recommendation results.

Recently, patch-based transformer methods have demonstrated strong effectiveness in time series forecasting. However, the complexity of self-attention imposes demands on memory and compute resources. In addition, though patches can capture comprehensive temporal information while preserving locality, temporal information within patches remains important for time series prediction. The existing methods mainly focus on modeling long-term dependencies across patches, while paying little attention to the short-term dependencies within patches. In this paper, we propose the Global and Local Frequency-domain Network (GLFNet), a novel architecture that efficiently learns global time dependencies and local time relationships in the frequency domain. Specifically, we design a frequency filtering layer to learn the temporal interactions instead of self-attention. Then we devise a dual filtering block consisting of global filter block and local filter block which learns the global dependencies across patches and local dependencies within patches. Experiments on seven benchmark datasets demonstrate that our approach achieve superior performance with improved efficiency.

SQL injection is a significant and persistent threat to web services. Most existing protections against SQL injections rely on traffic-level anomaly detection, which often results in high false-positive rates and can be easily bypassed by attackers. This paper introduces SQLStateGuard, the world's first middleware-driven statement-level SQL injection defense approach, to address these issues. The SQLStateGuard uses a custom SQL middleware based on the idea of Runtime Application Self-Protection to capture raw SQL statements. These statements are then analyzed by SQLSG-Net, a database-oriented detection network based on gated linear units. If SQLSG-Net detects malicious SQL statements, the SQL middleware will block them. Experiments show that the detection accuracy of SQLStateGuard exceeds 99%, outperforming existing approaches, and it can identify the type of a specific SQL injection. Additionally, SQLStateGuard has no fingerprint and does not respond to SQL syntax errors, making it more challenging for attackers to gather information. This paper also presents a novel dataset generation process for SQLStateGuard and shares two statement-level SQL injection datasets with the research community, including over 145,000 malicious SQL statements categorized by the type of SQL injection.

With the development of service computing technology, the number of publicly available Web APIs online has increased dramatically. Developers tend to use Web APIs to implement their software development requirements. However, due to the large number of Web APIs, how to select the appropriate Web API from the huge resource library for Mashup development has become a challenge. To solve this problem, some researchers have proposed Web API recommendation methods based on collaborative filtering or matrix factorization. However, the Web API recommendation performance is still limited due to the reliance on Mashup- API interaction and the ignorance of Mashup and Web API description documents. Moreover, the Mashup-API interaction matrix is extremely sparse, resulting in low accuracy in matrix factorization and collaborative filtering. To further improve the recommendation performance, we propose a novel joint matrix factorization method for the Mashup-API interaction matrix by incorporating the Mashup-Mashup similarity matrix and the API-API similarity matrix. A set of experiments are conducted on a real-world dataset, and the experimental results show that the proposed method outperforms the baselines.

With the wide use of Massive Open Online Courses (MOOCs) and online education, the large number and variety of educational resources on the online course open platform make it difficult for users to choose, and the requirements for course recommendation algorithms and model performance are getting higher and higher. The existing course recommendation algorithms often ignore the interactive information between n learners and courses and the course descriptions, resulting in poor recommendation performance. To solve the problems above, our work proposes an online course recommendation approach via exploring user interactions and course description documents. Specifically, LightGCN model is used to represent the user-course interaction graph to derive the structural embeddings of users and courses. Moreover, BERT model is used to represent the course description documents to derive the description embeddings of courses. Based on the three embeddings, ANFM (Attention Neural Factorization Machine) model is applied to perform online course recommendation by modeling low-order and high-order feature interactions, and learning the weights of the feature interactions through the attention mechanism. Finally, solid experiments are conducted over a real-world public dataset. Experimental results show that the proposed approach outperforms the baseline methods under NDCG, Recall and Precision.

Aiming to address challenges in SAR image ship detection tasks, such as the dense concentration of inshore ships, small target sizes, and large overlapping areas, this paper proposes an improved algorithm LAOBB-YOLOv8n based on YOLOv8n for detecting rotating ships in SAR images. In this model, we propose the Detect_LSCD detection head, which replaces the BatchNorm in the convolutional layers of the Detect module in YOLOv8n with GroupNorm to improve the positioning and classification capabilities of the detection head; Replace the original Spatial Pyramid Pooling-Fast (SPPF) module with the Attention-based Intrascale Feature Interaction (AIFI) module, which focuses on processing advanced image features through self-attention mechanism, thus improving the flexibility and accuracy of the model in processing complex scenes while reducing unnecessary computational consumption; Based on the original network structure, we add a small target detection layer to improve the detection ability for small targets; Use rotating bounding box instead of traditional horizontal bounding box, which can better accommodate the rotating posture of the target and reduce the overlapping areas between detection boxes, thereby improving the accuracy of inshore target detection. The comparative and ablation experiments are conducted on the RSDD-SAR dataset to verify the model’s effectiveness. The experimental results show that the mAP50 of LAOBB-YOLOv8n reaches 98%, which is 1.4% higher than that of YOLOv8n. Although the number of parameters of LAOBB-YOLOv8n is larger than that of YOLOv8n, it is still very small compared with other baseline models, and the accuracy of LAOBB-YOLOv8n is much higher than other models.

Maize, a significant global food crop, is essential in agriculture and the economy. The price of maize futures is affected by many factors, and its data is a nonlinear, unstable, and long-term correlation, so it is difficult to predict it accurately. Accordingly, this paper presents a combined model based on Ensemble Empirical Mode Decomposition (EEMD), Convolutional Neural Network (CNN), and Improved Gated Recurrent Unit (IGRU). EEMD decomposes the maize price data to produce multiple Intrinsic Mode Function (IMF) components and residual sequences. Subsequently, the noisy IMF components are removed, and the remaining IMF components are reconstructed into low, medium, and high frequencies. The CNN is tasked with the extraction of eigenvalues for these components. The IGRU improves in two key respects on the original Gated Recurrent Unit (GRU) model. First, it enhances the update and reset gates. Second, it incorporates the Self-Attention (SA) module. Thereby, the model's predictive capabilities are improved. This study uses the primary maize futures trading data from the Dalian Commodity Exchange as the experimental data. A comparative analysis of the EEMD-CNN-IGRU model with seven baseline models shows that it outperforms other models in all evaluation indexes.

Open-source software has emerged as a pivotal force in the advancement of information technology. Robust open-source compliance governance is essential for the sustainable and healthy growth of both open-source software and its communities. License incompatibility analysis, in particular, represents a critical challenge hindering the progress of open-source software. Traditional methods of incompatibility analysis often fail to account for diverse usage scenarios or are tailored to a limited subset of scenarios. This limitation obstructing their ability to handle the intricate compatibility arising from varied programming language interactions, leading to a high false positives. Our study embarks from an examination of license exceptions, delving into the incompatibility analysis challenges through extensive empirical research on these exceptions. We discovered that the majority of exceptions are, in fact, detectable. Leveraging this empirical insight, our research further develops the license compatibility analysis model by introducing a new, refined legal terminology representation alongside a novel method for license compatibility reasoning. This approach begins with modeling different scenarios to represent license compatibility variably. Furthermore, based on these modeling outcomes, we have designed and implemented LiScopeLens, a tool capable of discerning dependency behaviors for granular compatibility assessment, starting with binary dependencies. Our experimental findings affirm that LiScopeLens proficiently determines the license compatibility status of open-source software across various usage scenarios, demonstrating its significant practical utility.

Voice is one of the most widely used media for information transmission in human society. While high-quality synthetic voices are extensively utilized in various applications, they pose significant risks to content security and trust building. Numerous studies have concentrated on AI-synthesized voice detection to mitigate these risks, with many claiming to achieve promising performance. However, recent research has demonstrated that fake voice detectors suffer from serious overfitting to speaker-irrelative features (SiFs) and cannot be used in real-world scenarios. In this paper, we analyze the limitations of existing fake voice detectors and propose a new design philosophy, guiding the detection model to prioritize learning human voice features rather than the difference between the human voice and the synthetic voice. Based on this philosophy, we propose a novel AI-synthesized voice detection framework named SiFSafer, which uses pre-trained speech representation models to enhance the learning of feature distribution in human voices and the adapter fine-tuning to optimize the performance. The evaluation shows that the average EERs of existing fake voice detectors in the ASVspoof datasets can exceed 20% if the SiFs like silence segments are removed, while SiFSafer achieves an EER of less than 8%, indicating that SiFSafer is robust to SiFs and strongly resistant to existing attacks.

Federated learning addresses privacy concerns in multimedia recommender systems by enabling collaborative model training without exchanging raw data. However, existing federated recommendation models are mainly based on basic backbones like Matrix Factorization (MF), which are inadequate to capture complex implicit interactions between users and multimedia content. Graph Convolutional Networks (GCNs) offer a promising method by utilizing the information from high-order neighbors, but face challenges in federated settings due to problems such as over-smoothing, data heterogeneity, and elevated communication expenses. To resolve these problems, we propose a Cluster-driven Personalized Federated Recommender System with Interest-aware Graph Convolution Network (CPF-GCN) for multimedia recommendation. CPF-GCN comprises a local interest-aware GCN module that optimizes node representations through subgraph-enhanced adaptive graph convolution operations, mitigating the over-smoothing problem by adaptively extracting information from layers and selectively utilizing high-order connectivity based on user interests. Simultaneously, a cluster-driven aggregation approach at the server significantly reduces communication costs by selectively aggregating models from clusters. The aggregation produces a global model and cluster-level models, combining them with the user's local model allows us to tailor the recommendation model for the user, achieving personalized recommendations. Moreover, we propose an adversarial optimization technique to further augment the robustness of CPF-GCN. Experiments on three datasets demonstrate that CPF-GCN significantly outperforms the state-of-the-art models.

Dataset-Based Defense is a common proactive defense method aimed at countering backdoor attacks by filtering and removing poisoned samples from the training dataset. Recent research has utilized diffusion models to purify contaminated samples. However, during both forward and backward denoising processes, the inherent noise and distribution characteristics of the data can also affect the purification of poisoned samples, making the process unreliable. To address this issue, this paper re-examines the purification process from the perspective of causal graphs, revealing how backdoor triggers and noise act as confounding factors, generating false associations between contaminated images and labels. To eliminate this false association, we propose a Causality-Augmented Diffusion Model-Based Defense (CADMBD) method. CADMBD applies multiple iterations of forward noise addition and backward denoising to the data input in the diffusion model. During the backward denoising process, residual calculation and causal layer adjustment are used to eliminate backdoor pathways, thereby establishing a true causal relationship between benign features and labels. Finally, the most suitable generated image is selected as the final purified image based on pixel and feature distances from each iteration. Experimental results demonstrate that CADMBD effectively addresses various complex attacks, significantly enhancing the purification effect while minimizing the impact on benign sample accuracy, outperforming baseline defense methods.

Cell-free (CF) massive multiple-input multiple-output (mMIMO) is a promising technology for enhancing the spectrum efficiency in future wireless systems. It leverages distributed access points (APs) to jointly serve multiple users over the same time-frequency resource. In this paper, we study the uplink performance of CF mMIMO orthogonal frequency division multiplexing (OFDM) wireless systems under asynchronous signal reception. First, we provide a complete and detailed frequency-domain uplink asynchronous signal reception model for CF mMIMO-OFDM systems, which shows that asynchronous reception introduces phase shift, inter-carrier interference (ICI), and inter-symbol interference (ISI) at the receivers. Then, we design the minimum mean square error channel estimation under asynchronous reception. The results indicate that the phase shift, ICI, and ISI destroy the orthogonality of pilot sequences at APs, leading to significant channel estimation error. Furthermore, the FDM pilot sequences prove to be more effective in mitigating the impact of asynchronous reception on channel estimation compared to the CDM pilot sequences. Given the imperfect channel state information caused by pilot contamination and asynchronous reception, we characterize the uplink achievable spectral efficiency under asynchronous reception. The results indicate that severe uplink rate loss mainly arises from phase shift caused by asynchronous reception. Numerical simulation results corroborate our analyses.

The emerging field of Artificial Intelligence of Things (AIoT) is propelled by the remarkable success of deep learning and hardware evolution, which has a significant impact on our daily lives. However, because of their notorious computing resource intensity, the widespread deployment of AIoT devices requires substantial electricity consumption as support, inevitably escalating energy consumption, and ultimately leads to a significant carbon emissions. Existing research on neural network compression and acceleration struggles to achieve energy-efficient inference on resource-constrained AIoT devices. To address this issue, we propose a software-hardware co-design approach that integrates advanced neural network optimization techniques with hardware power management capabilities to enable energy-efficient inference and ultimately achieve sustainable AIoT. We introduce a lightweight split and refinement block that adaptively reduces redundant computation in both channel and spatial dimensions. Several early exit (EE) branches are added to the backbone, which are controlled by a policy-based EE predictor. With the predicted EE index, a curve-fitting-based frequency scaling algorithm is presented to calculate the optimal frequency that minimizes energy overhead while maintaining latency constraints. Extensive experiments on CIFAR and CINIC classification tasks validate that our proposed method consistently reduces energy consumption for neural network inference while outperforming other competitive methods.

In the rapidly evolving landscape of the Internet of Things (IoT), concerns about privacy and security have become significant as interconnected devices communicate and collaborate. Fingerprints, serving as unique biometric identifiers, play a crucial role in the authentication and identification processes within this interconnected and exchanged network. However, attention is often directed towards the disclosure of visible fingerprints, overlooking latent fingerprints. This is primarily due to the challenges involved in extracting latent fingerprints, especially those remaining on the adhesive side of tape. Traditional methods physically/chemically peel tape to extract these fingerprints, but cause irreversible damage to the tape, hindering accurate fingerprint extraction. In this context, our investigation reveals that Optical Coherence Tomography (OCT) technology allows for the extraction of high-quality OCT fingerprint images from the adhesive side of tape, yielding precise fingerprint recognition and gender classification results. Concretely, we build a novel type of robotic-arm spectral-domain OCT (SD-OCT), which is software-controlled for the movement of the sample arm, making sample scanning more flexible and efficient. Furthermore, we utilize a deep learning network to perform representation learning on OCT fingerprints for the purpose of gender classification. In the first branch, we input OCT fingerprints into an EfficientNet-B3 network to learn their spatial domain features. Simultaneously, in the second branch, we design a network that utilizes Discrete Cosine Transform (DCT) to extract frequency domain features from OCT fingerprints. Ultimately, we integrate the spatial and frequency domain features extracted from OCT fingerprint images to generate comprehensive features. Therefore, in this paper, we introduce a novel Gender Classification approach based on Spatio-Frequency Feature Fusion of OCT Fingerprint Images (named GenClassOCT-SF). The GenClassOCT-SF involves a robotic-arm SD-OCT system for superior-quality fingerprints acquisition and a deep learning network for spatial and frequency domain feature extraction. The fusion of these features enables highly accurate gender classification. Finally, we conduct gender classification experiments on the collected OCT fingerprint dataset to demonstrate the effectiveness of our proposed method.

Cell-free massive multiple-input–multiple-output (CF mMIMO) is a promising technology to enhance the spectrum efficiency in future wireless systems by leveraging distributed access points (APs) to jointly serve the users over the same time-frequency resource. However, for uplink communications under orthogonal frequency division multiplexing (OFDM), the signals received at different APs from different users are generally asynchronous, due to the varying distances between the APs and users. In this article, we study the performance of CF mMIMO-OFDM wireless system under asynchronous signal reception in terms of uplink channel estimation and achievable spectral efficiency. In particular, we devise the minimum mean square error (MMSE) channel estimation under asynchronous reception and reveal how the phase shift, intersymbol interference, and intercarrier interference introduced by asynchronous reception destroy the orthogonality between different pilot sequences at the APs. In addition, given the imperfect channel state information caused by pilot contamination and asynchronous reception, we obtain closed-form expressions for the uplink achievable rate for both centralized and distributed operations. We show that for both operations, the uplink achievable rates under asynchronous reception are much smaller than those under synchronous reception due to the phase shift caused by asynchronous reception. Last, numerical results demonstrate that under asynchronous reception, the maximum sum-rate loss with MMSE and maximum ratio (MR) combining in centralized and distributed operations reach 29.5%, 4%, 45.4%, and 8%, respectively. This indicates that while MMSE combining is much more sensitive to asynchronous reception than MR combining, it still achieves a higher achievable rate.

Due to the high dependence of social development on electricity, the failure of energy system equipment often leads to inestimable losses. The use of Internet of Things (IoT) technology to collect real-time data from energy devices and artificial intelligence (AI) technology to prognostic faults in the devices becomes essential to achieve power system security. Although real-time data contains rich descriptions of relevant equipment faults and solutions, the complex data structure and severe imbalance of condition monitoring (CM) data may lead to poor performance of AI models. In this study, we propose a multitask correlation constrained topology learning model for smart prognostic and health management in IoT. In particular, the proposed model mainly includes a feature extraction module, a multitask topology network (MTTN) module, and a class balance loss (CBL) algorithm module. First, the Bi-LSTM is employed to extract word collocation features and numerical features of the data, focusing on the important semantic features in complex structured data. Second, the MTTN is constructed to efficiently utilize the topological dependency between multiple tasks. Finally, a CBL loss function is applied to enhance the focus on the minority classes. Experiment and evaluation results demonstrate that our model has superior learning efficiency and prediction performance, especially on extremely imbalanced data, which can correctly predict the faulty equipment, the fault cause and the fault severity level, providing a rapid and precise reference for advance repair or replacement of energy system equipment in IoT environments.

With the flourish of digital technologies and rapid development of 5G and beyond networks, Metaverse has become an increasingly hotly discussed topic, which offers users with multiple roles for diversified experience interacting with virtual services. How to capture and model users’ multi-platform or cross-space data/behaviors become essential to enrich people with more realistic and immersed experience in Metaverse-enabled smart applications over 5G and beyond networks. In this study, we propose a Personalized Federated Learning with Model-Contrastive Learning (PFL-MCL) framework, which may efficiently enhance the communication and interaction in human-centric Metaverse environments by making use of the large-scale, heterogeneous, and multi-modal Metaverse data. Differing from the conventional Federated Learning (FL) architecture, a multi-center aggregation structure to learn multiple global models based on the changes of dynamically updated local model weights, is developed in global, while a hierarchical neural network structure which includes a personalized module and a federated module to tackle both issues on data heterogeneity and model heterogeneity, is designed in local, so as to enhance the performance of PFL with unique characteristics of Metaverse data. In particular, a two-stage iterative clustering algorithm with a more precise initialization is developed to facilitate the personalized global aggregation with dynamically updated multiple aggregation centers. A personalized multi-modal fusion network is constructed to greatly reduce the computational cost and feature dimensions from the high-dimensional heterogeneous inputs for more efficient cross-modal fusion, based on a hierarchical shift-window attention mechanism and a newly designed bridge attention mechanism. A MCL scheme is then incorporated to speed up the model convergence with less communication overload between the local federated module and global model, while an embedding layer which effectively enables the delivered global model to better adapt to the local personality in each client is further integrated. Compared with five baseline methods, experiment and evaluation results based on two different real-world datasets demonstrate the excellent performance of our proposed PFL-MCL model in a fine-grain personalized training strategy, toward more efficient communication and networking among human-centric Metaverse enabled smart applications.

Traffic sign recognition is a crucial aspect of autonomous vehicle research, and deep learning techniques have significantly contributed to its progress. Nevertheless, the distribution of traffic sign information in natural complex road conditions is long-tailed, and traffic sign identification in complex road conditions has become a significant barrier to autonomous vehicle applications. The imbalanced distribution of information on the dataset migrates to the feature space during training, resulting in imbalanced classifier prediction. In this paper, we propose the gradient rebalanced traffic sign recognition (GRTR) method to address this problem for the first time. GRTR first evaluates the prediction and classification bias of the classifier using the fitted deviation between the model’s output probability and the ground-truth distributions. Then, GRTR dynamically adjusts the correction and compensation factors following the classifier’s prediction and classification biases. GRTR rebalances the positive and negative sample gradients for each category based on the synergistic effect of the correction and compensation factors to prevent the transfer of distribution imbalance and to significantly enhance the performance of the traffic sign classifier under difficult road conditions. Experimental results demonstrate that our GRTR achieves state-of-the-art performance on long-tailed traffic sign and multilabel datasets.Note to Practitioners—Most traffic sign recognition algorithms are still designed based on the assumption of a balanced distribution of traffic signs in the dataset. Real-world autonomous vehicles require traffic sign recognition on datasets with severely imbalanced distributions. This paper proposes a general approach to solving the long-tailed traffic sign recognition problem.

Deep learning and big data analysis are among the most important research topics in the fields of biomedical applications and digital healthcare. With the fast development of artificial intelligence (AI) and Internets of Things (IoT) technologies, deep learning (DL) for big data analytics—including affective learning, reinforcement learning, and transfer learning—are widely applied to sense, learn, and interact with human health. Examples of biomedical applications include smart biomaterials, biomedical imaging, heartbeat/blood pressure measurement, and eye tracking. These biomedical applications collect healthcare data through remote sensors and transfer the data to a centralized system for analysis. With an enormous amount of historical data, DL and big data analysis technologies are able to identify potential linkage between features and possible risks, raise important decision for medical diagnosis, and provide precious advice for better healthcare treatment and lifestyle. Although significant progress has been made with AI, DL, and big data analytic technologies for medical and healthcare research, there remain gaps between the computer-aided treatment design and real-world healthcare demands. In addition, there are unexplored areas in the fields of healthcare and biomedical applications with cutting-edge AI and DL technologies. Hence, exploring the possibility of DL and big data analytics in the fields of biomedical applications and digital healthcare is in high demand.

Large language models (LLMs) are becoming powerful engines for social productivity in the manufacturing lifecycle. Existing application-level LLMs inference services focus on large datacenter and small edge intelligence (EI) scenarios, adopting iteration-level batch schedulers to solve resource utilization and inference speed problems. However, these services are incompatible with the scene of medium-sized local heterogeneous graphics processing unit (GPU) clusters with specific patterns, whose scale is between the two aforementioned scenarios. This type of scene proposes tradeoff problems for inference resource and speed, as well as user satisfaction problems for the semisparse frequency of queries with streaming responses. We propose suboptimal load balancing (SLoB), a distributed LLMs inference service scheduler in medium-sized local heterogeneous GPU clusters. SLoB leverages a multilevel adapter to accommodate LLMs usage patterns of scenes and balance resource utilization with inference efficiency. For semisparse problems, it adopts a mixed-priority pipeline scheduler with the least-padding principle to improve users’ satisfaction, a metric considering the weights of different tokens in streaming responses. Based on the system prototype, our experiments under simulated workloads demonstrate that SLoB gains a maximum improvement of 29.4$\times$under the satisfaction metric compared with the traditional run-to-completion scheduling solution while improving by up to 3.0$\times$compared with the state-of-the-art (SOTA) solution Orca.

In the social production system, image data are rapidly generated from almost all fields such as factories, hospitals, and transportation, promoting higher requirements for image anomaly detection technologies, including low consumption, higher adaptability, and accuracy. However, existing anomaly detection methods are fragile to heterogeneous image data generated by complex social production systems and tend to require strong computing power and resource support. To address the above problems, a knowledge-driven anomaly detection framework is proposed, in which a local feature enhancement method is designed to strengthen the knowledge representation of the initial features extracted from images. The attention mechanism in deep learning is introduced to adjust the feature attention dynamically according to prior knowledge, which solves the problem of feature loss in the cascade training. To verify the effectiveness of the proposed framework, extensive experiments on social production datasets are conducted. The results demonstrate that our framework outperforms the selected methods on image datasets with different complexity and sample distributions.

Fake news is a prevalent issue in modern society, leading to misinformation, and societal harm. News credibility assessment is a crucial approach for evaluating the accuracy and authenticity of news. It plays a significant role in enhancing public awareness and understanding of news, while also effectively mitigating the dissemination of fake news. However, news credibility assessment meets challenges when processing large-scale and constantly growing data, due to insufficient and unreliable labels and standards, and diversity and semantic ambiguity of news contents. Recently, machine learning models have been well developed to address these issues, but suffer from limited effectiveness. A unified framework is also required for them to represent various entities and relationships involved in news stories. This article proposes an entity ontology-based knowledge graph network (EKNet) to leverage knowledge graphs and entity frameworks for news credibility assessment. The model utilizes the information from knowledge graphs by combining entities and relationships from news and knowledge graphs. Experimental results show that the EKNet has advantages in evaluating news credibility over existing methods. Specifically, compared to several strong baselines, the model demonstrates a significant performance improvement in scores across various tasks. Which indicates that using the EKNet to address the challenges in news credibility assessment is highly effective and can conduct better performance for the problem of fake news in the social media environment.

Recently, a multimodal sentiment analysis of social media has attracted increasing attention, and its core idea is to discovery heuristic fusion strategy to analyze the sentiment orientations over heterogeneous multimodal source from a learned compact multimodal representation. The existing multimodal fusion techniques not only struggle to achieve full heterogeneous data interaction, but also they are unable to dynamically assess the quality of various modal data to determine predictability. In this article, we present a novel deep tensor evidence fusion (DTEF) network for multimodal sentiment classification. First, we propose a common view evaluation network that uses a long short-term memory (LSTM) network and a tensor-based neural network to extract rich intermodal and intramodal information. Then, we propose a unique time cue evaluation network that takes advantage of the temporal granularity associated with numerous pattern sequences. To make reliable decisions, we finally incorporate uncertainty through the trusted fusion layer, which improves the accuracy and robustness of sentimental classification. Our model is validated using the CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) and CMU Multimodal Corpus of Sentiment Intensity (CMU-MOSI) datasets, and the experimental findings demonstrate the superior performance of the proposed network in terms of accuracy compared with the state-of-the-art methods.

The rapid development of the digital age has led to a qualitative leap in social media. To meet the cognitive needs of users, social media platforms have been mining users’ private information and disseminating information through various means. However, these platforms lack effective management of information release and various forms of emotional expressions make public propaganda increasingly diverse and complex. Therefore, accurately identifying the relationships between multimodal data poses a challenge. An effective modal representation must consider both the consistency of multimodal data and the complementarity of single-modal data. However, existing methods focus on fusing different modal features into a unified feature representation, while neglecting to evaluate the reliability of prediction results. In this article, we disentangle the consistency and complementarity in the fused representation problem of multimodal data. We construct the modal private task (unique) by using the Dirichlet distribution and evidence theory to solve the uncertainty of each modal prediction. The model can output the uncertainty of prediction and learn complementary information through the fusion of decision layers. At the same time, we construct the modal common task using a low-rank tensor fusion model to learn consistent features. Finally, we compare the model with the current mainstream methods on three public datasets, and the experimental results show that the performance of our method reaches the level of current advanced algorithms.

As a distributed intelligent paradigm, fuzzy federated learning (FuzzyFL) can reduce the uncertainty and noise of biomedical data and is suited to enhance the accurate detection of adolescent idiopathic scoliosis (AIS). The advanced paradigm requires the hospitals to share the gradient of the fuzzy deep neural network (FDNN) rather than biomedical data. Not only that, the recent research works have been devoted to privacy-preserving FuzzyFL for secure AIS detection that adds differential privacy-based noise to the gradients against membership inference attack, attribute inference attack. However, a novel reconstruction attack called gradient leakage attack (GLA) on inferring biomedical data over the gradient brings the security challenges to FuzzyFL and, thus, has a negative influence on AIS detection. It is natural to ask a fundamental question: Can differentially private FuzzyFL for AIS detection over biomedical data defend GLA? In this article, we construct a privacy-preserving FuzzyFL framework calledPrivateFuzzyFLthat offers a great opportunity to present the systematic evaluation of the private FDNN threatened by the GLAs. In our experiments on a set of chest X-ray images and four FDNNs, we compare more than ten private fuzzy federated optimization algorithms in terms of the defense effect and the utility cost and derive that, first, the existing private FDNNs in FuzzyFL can offer a certain amount of privacy protection for biomedical data against the GLA; and second, the perturbation algorithm with better defense effect usually causes the worse AIS detection of the FDNN.

The provision of privacy-preserving recommendations for geological tourist attractions is an important research area. The historical check-in data collected from location-based social networks (LBSNs) can be utilized to mine their preferences, thereby facilitating the promotion of the geological tourism industry. However, such check-ins often contain sensitive user information that poses privacy leakage risks. To address this issue, some methods have been proposed to develop privacy-preserving point-of-interest (POI) recommendation systems. These methods commonly rely on either perturbation-based or federated learning techniques to protect users’ privacy. However, the former can hinder preference capture, while the latter remains vulnerable to privacy breaches during the parameter-sharing process. To overcome these challenges, we propose a novel privacy-preserving POI recommendation model that incorporates users’ privacy preferences based on a simplified graph convolutional neural network. Specifically, we employ a generative model to create a subset of POIs that reflect users’ preferences but do not reveal their private information, and then we design a simplified graph convolutional network to analyze the high-order connectivity between users and POIs that are privacy-preserving. The resulting model enables efficient POI recommendation under strict privacy protection, which is particularly relevant to geological tourism. Experimental results on two public datasets demonstrate the effectiveness of our proposed approach.

Large-scale infectious diseases pose a tremendous risk to humans, with global outbreaks of COVID-19 causing millions of deaths and trillions of dollars in economic losses. To minimize the damage caused by large-scale infectious diseases, it is necessary to develop infectious disease prediction models to provide assistance for prevention. In this paper, we propose an XGBoost-LSTM mixed framework that predicts the spread of infectious diseases in multiple cities and regions. According to big traffic data, it was found that population flow is closely related to the spread of infectious diseases. Clustering and dividing cities according to population flow can significantly improve prediction accuracy. Meanwhile, an XGBoost is used to predict the transmission trend based on the key features of infection. An LSTM is used to predict the transmission fluctuation based on infection-related multiple time series features. The mixed model combines transmission trends and fluctuations to predict infections accurately. The proposed method is evaluated on a dataset of highly pathogenic infectious disease transmission published by Baidu and compared with other advanced methods. The results show that the model has an excellent predictive effect and practical value for large-scale infectious disease prediction.

Transportation big data generated from various Internet of Things devices have the feature of muti-source and heterogeneous. To efficiently represent and analyze these ubiquitous transportation big data, tensor and tensor-based data analysis methods have been widely adopted in recent years. As a tensor-based machine learning method, high-order dominant Z-eigen decomposition (HODZED) in multivariate multi-order Markov model is suitable for multi-modal transportation prediction. However, massive transportation data are usually generated in a streaming way and the transportation system requires frequent updates. To avoid recalculating the history data and provide immediate prediction, we propose a tensor train (TT) based incremental HODZED (TT-IHODZED) method. Concretely, we first present an incremental HODZED (IHODZED) method to update the dominant Z-eigentensor in multivariate multi-order Markov model. Then, TT-based tensor operations are adopted to IHODZED to speed up calculations, especially the repeated Einstein products. Furthermore, to solve the TT-based high-order linear equations in TT-IHODZED method, we also propose a TT-based biconjugate gradient stabilized (TT-HOBiCGS) algorithm. Experimental results based on real-world and synthetic datasets show that, compared to HODZED method, TT-IHODZED significantly improves computation efficiency up to 10 times while keeping the same or even better prediction accuracy.

Traffic and the movement of people are inextricably associated with the potential spread of COVID-19. In Intelligent Transportation System (ITS), Deep Learning (DL) traffic detection approaches driven by transportation big data have significant application values in monitoring, counting and classifying traffic vehicle information during the COVID-19 epidemic blockade, while DL COVID-19 medical diagnostic technology is also very important. However, due to concerns about data privacy and security, traditional data-centralized DL techniques that require uploading training data from multiple cameras or hospitals are no longer suitable. Federated Learning (FL) as a novel collaborative privacy-preserving DL paradigm could address this issue well. Nevertheless, in FL, most existing works train learning models with full-precision weights and communicate them over multiple iterations, which may incur massive additional communication costs and disclose the privacy implied in the trained local models. To tackle these issues, we first propose a novel multi-modal tensor ring decomposition TR-TSVD that not only achieves efficient data reduction but also keeps the correlations among multi-modes. Afterward, applying TR-TSVD to the training process of a convolutional neural network under the FL framework to achieve the goal of reducing communication overhead while ensuring model performance. Additionally, since the weight parameters are transmitted with the TR-TSVD format, attackers cannot infer the data privacy without knowing the specific restoration method. Besides, the additively homomorphic encryption is leveraged to further preserve model security. Extensive experimental results on MNIST, BIT-Vehicle and COVID-CT datasets show that the proposed approach could achieve a better performance.

The proliferation of advanced embedded and communication technologies has facilitated the possibility of modern Intelligent Transportation System (ITS). The hierarchical nature of such large-scale and distributed systems brings obvious challenges in creating a scalable and sustainable computing environment, and hence the development and application of edge intelligence become critical. Federated learning (FL), as an emerging distributed machine learning paradigm, aims to offer secure knowledge sharing and effective learning across multiple devices. However, conventional FL may fall into trouble when facing large-scale and network-agnostic systems with fast moving devices and changing network attributes. In this study, we propose an Adaptive Segmentation enhanced Asynchronous Federated Learning (AS-AFL) model, aiming to improve the learning efficiency and reliability in sustainable ITS via a decentralized fashion. Specifically, a meta-learning based adaptive segmentation scheme is designed to automatically separate the client nodes (e.g., vehicles) into multiple edge groups according to their homogeneous attributes. An integrated aggregation mechanism is then developed to realize the horizontal FL among a group of similar client nodes via the so-called intra-group synchronous aggregation, while allowing the vertical FL across different groups via the so-called inter-group asynchronous aggregation. Experiment and evaluation results based on an open-source dataset demonstrate the outstanding learning and communication performance of our proposed model, compared with several conventional FL schemes in a distributed ITS application scenario.

Understanding the latent disease patterns embedded in electronic health records (EHRs) is crucial for making precise and proactive healthcare decisions. Federated graph learning-based methods are commonly employed to extract complex disease patterns from the distributed EHRs without sharing the client-side raw data. However, the intrinsic characteristics of the distributed EHRs are typically non-independent and identically distributed (Non-IID), significantly bringing challenges related to data imbalance and leading to a notable decrease in the effectiveness of making healthcare decisions derived from the global model. To address these challenges, we introduce a novel personalized federated learning framework named PEARL, which is designed for disease prediction on Non-IID EHRs. Specifically, PEARL incorporates disease diagnostic code attention and admission record attention to extract patient embeddings from all EHRs. Then, PEARL integrates self-supervised learning into a federated learning framework to train a global model for hierarchical disease prediction. To improve the performance of the client model, we further introduce a fine-tuning scheme to personalize the global model using local EHRs. During the global model updating process, a differential privacy (DP) scheme is implemented, providing a high-level privacy guarantee. Extensive experiments conducted on the real-world MIMIC-III dataset validate the effectiveness of PEARL, demonstrating competitive results when compared with baselines.

The proliferation of Internet-of-Things (IoT) technologies in modern smart society enables massive data exchange for offering intelligent services. It becomes essential to ensure secure communications while exchanging highly sensitive IoT data efficiently, which leads to high demands for lightweight models or algorithms with limited computation capability provided by individual IoT devices. In this study, a graph representation learning model, which seamlessly incorporates graph neural network (GNN) and knowledge distillation (KD) techniques, named reconstructed graph with global-local distillation (RG-GLD), is designed to realize the lightweight anomaly detection across IoT communication networks. In particular, a new graph network reconstruction strategy, which treats data communications as nodes in a directed graph while edges are then connected according to two specifically defined rules, is devised and applied to facilitate the graph representation learning in secure and efficient IoT communications. Both the structural and traffic features are then extracted from the graph data and flow data respectively, based on the graph attention network (GAT) and multilayer perceptron (MLP) techniques. These can benefit the GNN-based KD process in accordance with the more effective feature fusion and representation, considering both structural and data levels across the dynamic IoT networks. Furthermore, a lightweight local subgraph preservation mechanism improved by the graph attention mechanism and downsampling scheme to better utilize the topological information, and a so-called global information alignment defined based on the self-attention mechanism to effectively preserve the global information, are developed and incorporated in a refined graph attention based KD scheme. Compared with four different baseline methods, experiments and evaluations conducted based on two public datasets demonstrate the usefulness and effectiveness of our proposed model in improving the efficiency of knowledge transfer with higher classification accuracy but lower computational load, which can be deployed for lightweight anomaly detection in sustainable IoT computing environments.

Rate-Splitting Multiple Access (RSMA) is an emerging and powerful multiple access scheme that relies on splitting and encoding user messages encoded into common and private streams, so as to partially decode multi-user interference and partially treat it as noise. In this paper, the secrecy rate constraint of each user is taken into consideration and a RSMA-based secure beamforming approach is proposed to maximize the weighted sum-rate (WSR). A generalized receiver model is considered where each user is also a potential eavesdropper wiretapping confidential messages for other users after decoding its own message. To solve the intractable non-convexity caused by security constraints in the formulated problem, a novel joint weighted minimum mean square error and successive convex approximation based alternate optimization algorithm is proposed and extended to maximize the instantaneous WSR with perfect channel state information at the transmitter (CSIT) and the weighted ergodic sum-rate with imperfect CSIT. Numerical results validate the effectiveness of the proposed design, which significantly improve the sum-rate performance and robustness to channel errors while guaranteeing message confidentiality and also unveil a better trade-off between message confidentiality and sum-rate performance thanks to its powerful interference management capability.

Green communication has attracted lots of attention recently, where NOMA (Non-Orthogonal Multiple Access) is one of the most promising technologies to realize energy-efficient communication. Specifically, by making massive wireless devices connect to the same time-frequency resource, NOMA can enhance the spectrum efficiency. In this paper, to investigate the freshness of the sensing data, we analyze the Age of Information (AoI) performance in NOMA-enabled Mobile Crowdsensing (MCS) circumstance, where the stochastic geometry theory is adopted. Firstly, we focus on the data submission process between mobile workers (MWs) and service provides (SPs), which drives to establish a model of the NOMA-enabled MCS. Then, given the transmission schemes of NOMA and OMA (Orthogonal Multiple Access), the mathematical expressions of the AoI metric are derived in the closed form respectively. Furthermore, simulation experiments are conducted to obtain AoI numerical results under various parameter settings (e.g., power strategies, queue models, and transmission protocols). Finally, the evaluation results not only prove the validness of the established models, but also provide some efficient solutions to achieve the optimal AoI value under the considered NOMA-enabled MCS scenario.

As an important financial investment commodity, the price fluctuations of gold significantly impact the global economy and the stability of the financial market. Therefore, it is of great significance to accurately predict the price of gold futures. This paper presents a novel gold futures price prediction model, PCA-AGRU, based on Principal Component Analysis (PCA) and Adaptive Gated Recurrent Unit (AGRU). PCA is utilized for the dimensionality reduction of data while extracting essential features. AGRU bases on Gated Recurrent Unit (GRU), embeds Self-Attention (SA), and adds an adaptive adjustment mechanism, making the model more effective in capturing long-term dependencies within time series data. This paper uses the international gold futures market data as the experimental dataset, and uses Maximal Information Coefficient (MIC) to analyze the correlation of the influencing factors. The PCA-AGRU model is compared with five prediction models of GRU, SA-GRU, AGRU, PCA-GRU, and PCA-SA-GRU. The experimental results show that the PCA-AGRU model performs best in forecasting gold futures prices.

Recently, reputation and trust analysis, especially using e-commerce product reviews which could be accumulated as large scale of text data, has contributed increasing greatly to marketing economics. However, it is not easy to find out the useful comments timely, or make the appropriate evaluation automatically, due to their sheer volume and time-varying updates. Existing studies usually treat them based on usefulness or sentiment analysis as a single task, but it is easy to ignore the latent features among different data domain and the potential causal relationships of various tasks. In this study, we focus on the multi-task deep learning, to cope with the reputation and trust analysis in e-commerce systems using product reviews. A so-called BiLSTM (Bidirectional LSTM) and VAE (Variational Autoencoder) Enhanced Multi-Task Neural Network (BiLV-MTNN) model is constructed to facilitate the evaluation, usefulness, and objectivity analysis simultaneously in a semi-supervised way. Specifically, A VAE based latent feature extraction mechanism and a BiLSTM based hidden vector generation mechanism are developed, which can contribute to the extraction of latent features and discovery of causal relationships among different tasks respectively. A labeling method is also proposed to quantify the objectivity of text composition based on the sentiment analysis, so as to improve the feature representation and further benefit the classification accuracy for multi-task learning. Compared with three baseline methods, experiments and evaluations using the Amazon data demonstrate the effectiveness and applicability of the proposed model in dealing with multi-task learning with alleviated negative transfer for trust-aware recommendation applications in e-commerce systems.

Group behavior modeling is an important research topic in the field of social network analysis . Existing methods regarding this topic can only learn the static group preference, ignoring the multiple characteristics of the group behavior , so they cannot model the group behavior in a complete way. In this paper, we propose a Dynamic Multi-view Group Preference Learning (DMGPL) model for group behavior prediction in social networks. Firstly, an area-aware user dynamic preference extracting module is developed concerning user representation learning , which can integrate the dynamic user behavior preference and the corresponding group structural information into group members’ representations. Then, to simultaneously obtain stability, propensity, potentiality, and heterogeneity of the group behavior, Multi-view Group Preference Aggregation (MGA) is designed for group behavior modeling. In MGA, various aggregation strategies are used to capture different kinds of group behavior properties, which can achieve the effect of highlighting different properties in different scenarios. Moreover, considering the influence of the member’s size on group behavior, we define the group scaling degree and perform group representation scaling to make MGA more flexible. Finally, extensive experiments are performed on two real-world datasets, and the results show that the accuracy of group behavior prediction by DMGPL improved by about 10% compared to the baseline models .

Nowadays, emerging sixth-generation (6G) mobile networks, the Internet of Things (IoT), and mobile-edge computing (MEC) technologies have played significant roles in developing a sustainable computing network. In sustainable computing networks, with the increasing scale of data-driven applications, massive privacy-sensitive data are generated. How to effectively process such data on resource-limited IoT devices is challenging. Although edge intelligence (EI) is designed to maintain an appropriate level of ultradelay reliability, low-latency communication (URLLC), real-time data processing, and security and privacy are concerning. In this article, we propose a novel blockchain-supported hierarchical digital twin IoT (HDTIoT) framework, which combines the digital twin to edge network and adopts blockchain technology to achieve secure and reliable real-time computation. We first propose a data and knowledge dual-driven learning solution to ensure real-time interaction and efficient optimization between the physical and the digital worlds. To improve communication and computation efficiency with data and knowledge dual-driven learning, the optimization goal is to minimize the system delay and energy consumption and ensure system reliability and the learning accuracy of IoT devices. Moreover, we propose a proximal policy optimization (PPO)-based multiagent reinforcement learning (MARL) algorithm to solve the resource allocation (RA) problem. Experimental results show that the proposed RA scheme can improve the efficiency of the HDTIoT system, guarantee learning accuracy, reliability, and security, and make a balance between system delay and energy consumption.

Nowadays, the concept of Internet of Everything (IoE) is becoming a hotly discussed topic, which is playing an increasingly indispensable role in modern intelligent applications. These applications are known for their real-time requirements under limited network and computing resources, thus it becomes a highly demanding task to transform and compute tremendous amount of raw data in a cloud center. The edge–cloud computing infrastructure allows a large amount of data to be processed on nearby edge nodes and then only the extracted and encrypted key features are transmitted to the data center. This offers the potential to achieve an end–edge–cloud-based big data intelligence for IoE in a typical two-stage data processing scheme, while satisfying a data security constraint. In this study, a deep-reinforcement-learning-enhanced two-stage scheduling (DRL-TSS) model is proposed to address the NP-hard problem in terms of operation complexity in end–edge–cloud Internet of Things systems, which is able to allocate computing resources within an edge-enabled infrastructure to ensure computing task to be completed with minimum cost. A presorting scheme based on Johnson’s rule is developed and applied to preprocess the two-stage tasks on multiple executors, and a DRL mechanism is developed to minimize the overall makespan based on a newly designed instant reward that takes into account the maximal utilization of each executor in edge-enabled two-stage scheduling. The performance of our method is evaluated and compared with three existing scheduling techniques, and experimental results demonstrate the ability of our proposed algorithm in achieving better learning efficiency and scheduling performance with a 1.1-approximation to the targeted optimal IoE applications.

The high-speed mobile networks offer great potentials to many future intelligent applications, such as autonomous vehicles in smart transportation systems. Such networks provide the possibility to interconnect mobile devices to achieve fast knowledge sharing for efficient collaborative learning and operations, especially with the help of distributed machine learning, e.g., Federated Learning (FL), and modern digital technologies, e.g., Digital Twin (DT) systems. Typically, FL requires a fixed group of participants that have Independent and Identically Distributed (IID) data for accurate and stable model training, which is highly unlikely in real-world mobile network scenarios. In this paper, in order to facilitate the lightweight model training and real-time processing in high-speed mobile networks, we design and introduce an end-edge-cloud structured three-layer Federated Reinforcement Learning (FRL) framework, incorporated with an edge-cloud structured DT system. A dual-Reinforcement Learning (dual-RL) scheme is devised to support optimizations of client node selection and global aggregation frequency during FL via a cooperative decision-making strategy, which is assisted by a two-layer DT system deployed in the edge-cloud for real-time monitoring of mobile devices and environment changes. A model pruning and federated bidirectional distillation (Bi-distillation) mechanism is then developed locally for the lightweight model training, while a model splitting scheme with a lightweight data augmentation mechanism is developed globally to separately optimize the aggregation weights based on a splitted neural network structure (i.e., the encoder and classifier) in a more targeted manner, which can work together to effectively reduce the overall communication cost and improve the non-IID problem. Experiment and evaluation results compared with three baseline methods using two different real-world datasets demonstrate the usefulness and outstanding performance of our proposed FRL model in communication-efficient model training and non-IID issue alleviation for high-speed mobile network scenarios.

Medical images are an important basis for doctors to diagnose diseases, but some medical images have low resolution due to hardware technology and cost constraints. Super-resolution technology can reconstruct low-resolution medical images into high-resolution images and enhance the quality of low-resolution images, thus assisting doctors in diagnosing diseases. However, traditional super-resolution methods mainly learn the mapping relationships among modal pixels from low resolution to high resolution, lacking the learning of high-level semantic features, resulting in a lack of understanding and utilization of semantic information, such as reconstructed objects, object attributes, and spatial relationships between two objects. In this paper, we propose a medical image super-resolution method based on semantic perception transfer learning. First, we propose a novel semantic perception super-resolution method that empowers super-resolution models to perceive high-level semantics by transferring features of the image description generation network in natural language processing. Second, we construct a semantic feature extraction network and an image description generation network and comprehensively utilized image and text modal data to learn transferable, high-level semantic features. Third, we train an end-to-end, semantic perception super-resolution model by fusing dynamic perceptual convolution, a semantic extraction network, and distillation polarization self-attention. Experiments show that semantic perception transfer learning can effectively improve the quality of super-resolution reconstruction.

A deep transfer learning framework adapting mixed subdomains is proposed for cross-species plant disease diagnosis. Most existing deep transfer learning studies focus on knowledge transfer between highly correlated domains. These methods may fail to deal with domains that are poorly correlated. In this study, mixed domain images were generated from source and target image groups for improving the correlation between the mixed domain (training dataset) and the target domain (testing dataset). A subdomain alignment mechanism is employed to transfer knowledge from the mixed domain to the target domain. The proposed framework captures the fine-grained information more effectively. Extensive experiments were conducted and prove that the proposed method produces a more effective result compared with existing deep transfer learning technologies for poorly related subdomains.

The ever-increasing web application programming interfaces (APIs) in various service-sharing communities (e.g., ProgrammableWeb.com and Mashape.com) have enabled software developers to quickly create their interested mashups conveniently and economically. However, the big volume of candidate web APIs and their differences often make it hard for software developers to discover a set of appropriate web APIs for mashup creation by considering API functions and API quality performances (e.g., popularity, compatibility, and diversity) simultaneously. These decrease the mashup development success rate and the mashup developers’ satisfaction significantly. In view of these challenges, a novel web APIs’ recommendation method named the popularity-aware and diverse method of web API compositions’ recommendation (PD-WACR) is proposed in this article. In concrete, we model web APIs’ functions, popularity, and compatibility with an API correlation graph. Afterward, correlation graph-based web APIs’ recommendation is performed with popularity and compatibility guarantee. Moreover, a top-$k$strategy is adopted in the recommendation process, so as to diversify the final recommended web APIs’ results. Finally, massive experiments are carried out on a real-world web API dataset crawled from ProgrammeableWeb.com. Experimental comparisons with related methods show the advantages and innovations of the proposed PD-WACR method.

The proliferation in embedded and communication technologies made the concept of the Internet of Medical Things (IoMT) a reality. Individuals’ physical and physiological status can be constantly monitored, and numerous data can be collected through wearable and mobile devices. However, the silo of individual data brings limitations to existing machine learning approaches to correctly identify a user’s health status. Distributed machine learning paradigms, such as federated learning, offer a potential solution for privacy-preserving knowledge sharing without sending raw personal data. However, federated learning is vulnerable to harmful participants that can degrade the overall model quality by sharing low-quality data. Therefore, it is critical to select suitable participants to ensure the accuracy and efficiency of federated learning. In this article, a unique clustering-based approach is proposed to use social context data for participant selection. Different edge participant groups will be established, and group-specific federated learning will be performed. The models of various edge groups will be further aggregated to strengthen the robustness of the global model. The experimental results demonstrated that through participant selection, clustering-based hierarchical federated learning can achieve better results with less participants in two different IoMT applications for ECG and human motion monitoring. This shows the efficacy of the proposed method in improving federated learning performance and efficiency in various IoMT applications.

Safety production surveillance is of great significance to industrial operation management. While augmented intelligence of things is demonstrating tremendous potential in industrial applications, the analyzed information offers lots of benefits to the higher level planning in the enterprise management systems, to further improve the operational efficiency. In this article, a video surveillance system with augmented intelligence of things is considered as a promising solution to enhance the operational efficiency of enterprises. However, the challenge is to process the surveillance video streams as soon as possible without ignoring any emergencies. This issue can be formulated as a two-stage scheduling problem, which is an NP-hard problem that can be integrated with higher level enterprise systems for operational efficiency improvement. An improved Deep Q-Network (DQN) model with a newly designed prioritized replay scheme, named Bi-Dueling DQN with Prioritized Replay, is proposed to solve this two-stage scheduling problem in a smart enterprise management system. A dense reward function based on a concrete state representation is designed to tackle the sparse reward challenge and to speed up the convergence in actual large-scale task scheduling process. A prioritized replay scheme is then developed to improve the sampling efficiency, so as to effectively reduce the training time in deep reinforcement learning for the optimal two-stage scheduling. The experiment results demonstrated that the proposed approach is able to provide an efficient scheduling policy to resolve the two-stage scheduling problem, while at the same time offering insight information to improve the performance of higher level smart enterprise management systems.

Recently, artificial intelligence approaches are widely suggested to optimize numerous offloading task-scheduling purposes. However, they confront difficulties in maintaining data privacy regarding the context of the data offloading during the course of offloading in the different stages. To address this problem, in this article we proposeC-fDRL, a framework to provide context-aware federated deep reinforcement learning (fDRL) to maintain the context-aware privacy of the task offloading. We perform this in three stages (CloudAI, EdgeAI, and DeviceAI) of the overall system.C-fDRLchecks whether the privacy of high-context-aware data with the task being offloaded is maintained locally at the DeviceAI, and low-context-aware data distributedly at the EdgeAI. When there is an offloading task request or a user needs to offload the data,C-fDRLuses a context-aware data management approach to decouple the context-aware (privacy) data from the tasks. This separates the context-aware data from the task for local computation and allows a new scheduling technique called “context-aware multilevel scheduler.” This places high-context-aware data on local devices and low-context-aware data at the edge device for computation before the actual task execution. We performed experiments to evaluate the data privacy with the offloading tasks and the federated DRL. The results show that the proposedC-fDRLperforms better than the existing framework.

In Industry 4.0, the development of intelligent supply chains is the key to improving enterprise efficiency and customer satisfaction. At present, the strategy with blockchain to optimize the supply chain ensures the security of data sharing and interaction between different suppliers. However, the quality of service (QoS) loss in time efficiency and supplier cooperation is still a severe problem in these strategies. Therefore, it is essential to conduct reputable supplier selection to reduce QoS loss under the premise of blockchain security. Meanwhile, choosing a reputable supplier as the primary peer in the blockchain consensus process can also effectively reduce transaction latency. To solve this problem, a reputation-aware supplier assessment system, named SAS, is proposed. Specifically, Canopy and K-medoids are utilized to classify suppliers preliminarily. Then backpropagation neural network is deployed to evaluate supplier reputation. The results show that SAS is effective in supplier reputation evaluation through extensive experiments.

With the increasing data scale in the Industrial Internet of Things, edge computing coordinated with machine learning is regarded as an effective way to raise the novel latency-sensitive services. To ensure the data privacy for frequent service access, federated learning (FL), as a privacy-preserving distributed framework, is integrated into edge computing, enabling user data invisible to the training process. However, sophisticated network attacks threaten deep learning (DL) models by data poison and malicious reasoning, making the DL-based system untrustworthy. To this end, a synergic data filtering method, named Safe, is proposed to deal with the poisoning attacks. Specifically, considering that the distributed support vector machine is at risk of being attacked due to its distribution and openness to communication, edge-cloud empowered FL framework is designed. Then, the alternating direction method of multipliers is deployed to detect attacked devices whose training processes will be interrupted. Moreover, due to the untrustworthiness of label data, the poisoned data in the attacked devices are figured out and filtered by clustering the trusted data withK-means clustering algorithm. Eventually, extensive experiment results proved that the Safe outperforms correlation methods in detection accuracy and trustworthiness.

Physics theory integrated machine learning models enhance the interpretability and performance of artificial intelligence (AI) techniques to real-world industrial applications, such as the fault detection and diagnosis (FDD) of air handling units (AHU). Traditional machine learning-based automated FDD model demonstrates a high classification accuracy with sufficient training data samples, however, suffers from physical interpretation of the machine learning models. In this article, a physical model integrated Wasserstain generative adversarial network (WGAN) model is presented for AHU FDD with a scenario of insufficient training data samples. The proposed solution tackles the real-world problem of AHU FDD and enhances the model interpretability significantly. A transformer-WGAN model is designed to further improve the proposed FDD framework. Experimental results show that the proposed method outperforms existing AHU FDD methods with imbalanced real-world training data samples.

Modern manufacturing process is commonly composed of multiple automated devices working together efficiently. Cloud-based manufacturing aims to achieve better efficiency by allowing the collaborative manufacturing across a group of automated robots. Cooperations between multiple robots can accomplish more complicated tasks that is beyond the capability of any individual one. However, it is of critical importance to control robots with different capabilities to work in harmony while ensuring safety and reliability during this process. In this paper, a double constrained containment mechanism is proposed to dispatch heterogeneous robots in a distributed containment control framework for smart manufacturing. Following a three-layer control framework, a cloud decision-making center is designed to realize the cloud-based collaborative manufacturing, which is more cost-effective than other commonly used containment mechanisms that only rely on information exchange among leader-robots. A projection-based containment control scheme is developed, which not only consider nonlinearities induced by position constraints and velocity constraints, but also can tackle dynamically changing communication topologies with uncertain communication delay, to efficiently navigate all follower-robots into a safe working zone formed by leaders. A theoretical stability analysis is conducted to prove the proposed mechanism can ensure all followers enter the target area while their positions and velocities remaining in the corresponding constraint sets. Experiment evaluation results under three application scenarios demonstrate the advantage of our method that can offer a more practical solution to other existing multi-robot containment control for cloud-based smart manufacturing, in considering both position and velocity constraints combined with switching topologies and communication delays.

The impact of Internet of Things (IoT) has become increasingly significant in smart manufacturing, while deep generative model (DGM) is viewed as a promising learning technique to work with large amount of continuously generated industrial Big Data in facilitating modern industrial applications. However, it is still challenging to handle the imbalanced data when using conventional Generative Adversarial Network (GAN) based learning strategies. In this article, we propose a distribution bias aware collaborative GAN (DB-CGAN) model for imbalanced deep learning in industrial IoT, especially to solve limitations caused by distribution bias issue between the generated data and original data, via a more robust data augmentation. An integrated data augmentation framework is constructed by introducing a complementary classifier into the basic GAN model. Specifically, a conditional generator with random labels is designed and trained adversarially with the classifier to effectively enhance augmentation of the number of data samples in minority classes, while a weight sharing scheme is newly designed between two separated feature extractors, enabling the collaborative adversarial training among generator, discriminator, and classifier. An augmentation algorithm is then developed for intelligent anomaly detection in imbalanced learning, which can significantly improve the classification accuracy based on the correction of distribution bias using the rebalanced data. Compared with five baseline methods, experiment evaluations based on two real-world imbalanced datasets demonstrate the outstanding performance of our proposed model in tackling the distribution bias issue for multiclass classification in imbalanced learning for industrial IoT applications.

With the development of the Internet of Things (IoT) and 5G technologies, IoT devices deployed on roads are able to collect a large amount of traffic data at any time. Road networks can be easily constructed into a graph structure with spatial-temporal features, and how to use these spatial-temporal features for dynamic traffic flow forecasting has become a heated issue. Although existing studies bring in the consideration of periodicity to deal with spatial-temporal sequence dependence, the similarity of time-varying relationships among cross-spatial nodes has not been well discussed. In this paper, we propose a Graph Attention Network with Spatial-Temporal Clustering (GAT-STC), which considers the so-called recent-aware features and periodic-aware features, to improve the Graph Neural Network (GNN)-based traffic flow forecasting in Intelligent Transportation System (ITS). Specifically, for the recent-aware feature extraction, a distance-based Graph Attention Network (GAT) is improved and constructed to better utilize the hidden features of neighbor nodes within a reliable distance during the recent time interval, thus can effectively capture the dynamic changes in spatial feature representation. For the periodic-aware feature extraction, a spatial-temporal clustering algorithm, in which both features in terms of nodes’ current traffic states and similar trends in terms of their dynamic changes are taken into account, is developed and applied to achieve better learning efficiency. Experiments using three public traffic datasets demonstrate the higher accuracy and better efficiency of our proposed model for traffic flow forecasting, compared with five baseline methods in ITS.

Intelligent transportation infrastructure is essential to intelligent transportation system (ITS). With the continuous development of Internet of Things (IoT) technology, remote monitoring has become a critical part of ITS. However, due to the limitations of network transmission, production cost, and other factors, some video monitoring can obtain only low-resolution (LR) video. LR video features are seriously lost, thus affecting the performance of ITS. In this paper, based on the research of existing super-resolution algorithms, we focus on improving the reconstruction quality of video frame sequences by aiming at the insufficient utilization of inter-frame information and low reconstruction quality of existing video super-resolution algorithms. This paper proposes a video super-resolution algorithm based on inter-frame information utilization, which can effectively improve the performance of ITS. First, a novel U-shaped feature extractor is designed to fully extract the feature expression of video frame sequences. Second, a deformable inter-frame alignment module based on residual learning is constructed to make the inter-frame alignment more accurate and thus promote the mutual utilization of inter-frame information. Finally, an up and down sampling residual block is proposed to extract features that better match the upsampling reconstruction requirements. The experimental results show that the method has better reconstruction quality for monitoring video and is advanced and applicable compared to mainstream video oversampling methods.

Trajectory data contains rich spatial and temporal information. Turning trajectories into graphs and then analyzing them efficiently in an AI-empowered way is a representative branch of trajectory analysis in IoV and ITS environments, which is of great significance. This research attempts to project trajectories onto road networks to predict traffic conditions. Extracting accurate spatio-temporal dependencies is the key to improving the analysis. However, two problems exist in the current study. The first one is the focus on the network structure while ignoring node features, and the second one is that the structure cannot be fully utilized. In addition, the static spatial structure may not accurately reflect the dynamic real spatial dependency. In response to these problems, a novel Spatio-Temporal Graph Convolutional Networks via View Fusion for Trajectory Data Analytics (STFGCN) model is designed. It contains two independent views: the structural view and feature view. The view fusion layer is further designed. It includes an extended graph convolutional module and a causal dilated module. The extended graph convolutional module fully extracts dynamic spatial dependencies, while the causal dilated module captures time tendencies. Stacked view fusion layers and a view fusion module perform fusion operations based on the advantages of the two views, efficiently integrating information from both. Several experiments are performed on two real-world trajectory datasets. The results show that a better prediction performance is obtained, especially on the long-range time prediction task.

Cyber-Manufacturing combines industrial big data with intelligent analysis to find and understand the intangible problems in decision-making, which requires a systematic method to deal with rich signal data. With the development of spectral detection and photoelectric imaging technology, spectral blind deconvolution has achieved remarkable results. However, spectral processing is limited by one-dimensional signal, and there is no available structural information with few training samples. Moreover, in the majority of practical applications, it is entirely feasible to gather unpaired spectrum dataset for training. This training method of unpaired learning is practical and valuable. Therefore, a two-stage deconvolution scheme combining self supervised learning and feature extraction is proposed in this paper, which generates two complementary paired sets through self supervised learning to extract the final deconvolution network. In addition, a new deconvolution network is designed for feature extraction. The spectrum is pre-trained through spectral feature extraction and noise estimation network to improve the training efficiency and meet the assumed noise characteristics. Experimental results show that this method is effective in dealing with different types of synthetic noise.

With the development of intelligent medicine, lesion detection supported by Internet of Things (IoT), big data, and deep learning has become a hotspot. However, lesion detection technology based on deep learning requires huge amounts of high-quality medical image data, and the data from social IoT has the problems of uneven quality and lack of lesion labeling. Current studies usually ignore the unstable quality of IoT data and the interpretability of diagnostic results, resulting in deeper model layers, larger models, poor real-time performance, and lack of persuasion. To address the problems, this article first proposes a core data extraction method for multi-class lesion detection based on unlabeled medical image from social IoT. Then, we propose an ensemble algorithm based on lightweight models to improve the detection accuracy. Finally, we visualize pathological features to enhance the interpretability of core data and detection results. The experimental results show that our method can effectively extract the core data of multiple lesions from low-quality medical images and improve the accuracy of the lightweight lesion detection model as well as the interpretation of detection results.

The newly emerging federated learning (FL) framework offers a new way to train machine learning models in a privacy-preserving manner. However, traditional FL algorithms are based on an event-triggered aggregation, which suffers from stragglers and communication overhead issues. To address these issues, in this paper, we present a time-triggered FL algorithm (TT-Fed) over wireless networks, which is a generalization of classic synchronous and asynchronous FL. Taking the resource-constrained and unreliable nature of wireless networks into account, we jointly consider the user selection and bandwidth optimization problem to minimize the FL training loss. The optimization problem is decomposed into tractable sub-problems with respect to each global aggregation round, and finally solved by our proposed greedy search algorithm. Simulation results show that compared to asynchronous FL (FedAsync) and FL with asynchronous tiers (FedAT) benchmarks, our proposed TT-Fed algorithm improves the converged test accuracy by up to 12.5% and 5%, respectively, under highly imbalanced and non-IID data, while substantially reducing the communication overhead.

Question answering over knowledge graph (KGQA) aims at answering questions posed over the knowledge graph (KG). Multi-hop KGQA requires multi-hop reasoning on KG to achieve the correct answer. Unfortunately, KGs are usually incomplete with many missing links, which poses additional challenges to KGQA. KG embedding-based KGQA methods have recently been proposed as a way to overcome this limitation. However, existing KG embedding-based KGQA methods fail to take full advantage of semantic correlations between questions and paths. Furthermore, their inference process is not easily explainable. To address these challenges, we propose a novel path-aware multi-hop KGQA model (PA-KGQA), which can fully capture semantic correlations between the paths and the questions in a feature-interactive manner. Specifically, we introduce a case-enhanced path retriever to evaluate the importance of paths between topic entities and candidate answer entities, and then propose an interactive convolutional neural network (ICNN) to model the interactions between paths and questions for mining richer correlation features. Experiments show that PA-KGQA achieves state-of-the-art results on multiple benchmark datasets and is explainable.

A highly accurate demand forecast contributes to higher profits with more sales opportunities and lower waste losses because of excess inventory in the modern retail industry. As forecasting models for chain stores require high generalization performance and operational efficiency, it is still challenging to build an efficient model in the demand side to timely cope with different purchasing characteristics among multiple regions using traditional learning schemes. Hence, this paper proposes a pooling attention and gated recurrent unit (PA-GRU) based learning model, with one convolution neural network (CNN) layer for feature extraction and one GRU for time series forecasting with two improved attention mechanisms, to enhance the demand forecast with higher accuracy. The first attention mechanism captures the trend and periodic patterns of the target variable, while the second attention mechanism refines the core features for further forecasting. Deep learning frameworks enable forecasting that adapts to time-varying patterns in different complex demands, which can also be efficiently applied for cross-learning in multistock keeping unit and multi-store environments, facilitated by an improved transfer learning scheme. Experiments are conducted based on a real-world POS dataset of a food supermarket with 3,065,572 customers in 46 stores in nine prefectures. The evaluation results demonstrate the outstanding performance of our proposed model in dealing with multiple time series data for demand forecasting in the retail industry compared to those of several existing learning methods.

This paper explores the task of video moment retrieval (VMR), which aims to localize the temporal boundary of a specific moment from an untrimmed video by a sentence query. Previous methods either extract pre-defined candidate moment features and select the moment that best matches the query by ranking, or directly align the boundary clips of a target moment with the query and predict matching scores. Despite their effectiveness, these methods mostly focus only on aligning the query and single-level clip or moment features, and ignore the different granularities involved in the video itself, such as clip, moment, or video, resulting in insufficient cross-modal interaction. To this end, we propose a Temporal Localization Network with Hierarchical Contrastive Learning (HCLNet) for the VMR task. Specifically, we introduce a hierarchical contrastive learning method to better align the query and video by maximizing the mutual information (MI) between query and three different granularities of video to learn informative representations. Meanwhile, we introduce a self-supervised cycle-consistency loss to enforce the further semantic alignment between fine-grained video clips and query words. Experiments on three standard benchmarks show the effectiveness of our proposed method.

As investigations on physical layer security evolve from point-to-point systems to multi-user scenarios, multi-user interference (MUI) is introduced and becomes an unavoidable issue. Different from treating MUI totally as noise in conventional secure communications, in this paper, we propose a rate-splitting multiple access (RSMA)-based secure beamforming design, where user messages are split and encoded into common and private streams. Each user not only decodes the common stream and the intended private stream, but also tries to eavesdrop the private streams of other users. We formulate a weighted sum-rate (WSR) maximization problem subject to the secrecy rate requirements of all users. To tackle the non-convexity of the formulated problem, a successive convex approximation (SCA)-based approach is adopted to convert the original non-convex and intractable problem into a low-complexity suboptimal iterative algorithm. Numerical results demonstrate that the proposed secure beamforming scheme outperforms the conventional multi-user linear precoding (MULP) technique in terms of the WSR performance while ensuring user secrecy rate requirements.

Summary Behavior is autonomous, convergent, and uncertain, which brings challenges to the modeling of social network behavior spread. In this article, we propose a behavior spread model based on group cohesion under uncertain environments. First, for behavioral convergence, we define group cohesion to quantify the convergent effects of group. Second, based on the game theory to model the autonomy of behavior, according to the characteristics of the game payoffs changing with time and the depth of spread, and integrating group cohesion, a dynamic game payoffs calculation method is designed. Finally, aiming at the uncertainty of behavior, a group behavior spread model based on random utility theory is established. Experiments on multiple real social network behavior spread datasets demonstrate the effectiveness of the proposed model in modeling and predicting behavior spread processes under uncertain environments.

Summary The spread of rumors has a major negative impact on social stability. Traditional rumor spreading models are mostly based on infectious disease models and do not consider the influence of individual differences and the network structure on rumor spreading. In this paper, we propose a rumor Fick‐spreading model that integrates information decay in social networks. The dissemination of rumors in social networks is random and uncertain and is affected by the dissemination capabilities of individuals and the network environment. The rumor Fick‐transition coefficient and Fick‐transition gradient are defined to determine the influence of the individual transition capacity and the network environment on rumor propagation, respectively. The Fick‐state transition probability is used to describe the probability of change of an individual's state. Moreover, an information decay function is defined to characterize the self‐healing probability of individuals. According to the different roles and reactions of users during rumor dissemination, the user state and the rumor dissemination rules among users are refined, and the influence of the network structure on the rumor dissemination is ascertained. The experimental results demonstrate that the proposed model outperforms other rumor spread models.

Inkjet printing technology can make tiles with very rich and realistic patterns, so it is widely adopted in the ceramic industry. However, the frequent nozzle blockage and inconsistent inkjet volume by inkjet printing devices, usually leads to defects such as stayguy and color blocks in the tile surface. Especially, the stayguy in complex pattern is difficult to identify by naked eyes due to it is easily covered by complex patterns and becomes invisible, this brings great challenge to tile quality inspection. Nowadays, the machine learning is employed to address the issues. The existing machine learning methods based on hand-crafted features are capable of stayguy detection of the tiles with a simple pattern, but not applicable for complex patterns due to the interference of pattern in feature extraction. The emerging deep-learning-based methods have the potential to be applied for stayguy detection with complex patterns, but cannot achieve real-time detection due to high complexity. In this paper, a lightweight hand-crafted feature enhanced convolutional neural network (named HFENet) is proposed for rapid defect detection of tile surface. First, we perform data enhancement on the original image by global histogram equalization and image addition. Second, for the special shape of stayguy which is usually vertical, we embed the extended vertical edge detection operator (Prewitt) as convolution kernel into HFENet to extract the hand-crafted vertical edge features of the test image and eliminate the interference of complex pattern in the feature extraction. Third, the 5 × 1 asymmetric convolution kernel with a dilation rate of 2 is used to improve the utilization of convolution kernel and reduce the complexity of the model. Fourth, to reach the real-time requirements, a memory access cost-aware design is proposed, which can orchestrate the number of shallow convolution layers and deep convolution layers in feature extraction. The experiments were performed on the ceramic tile image data set captured by high-resolution industrial cameras in ceramic tile production line. Experimental results show that the HFENet outperforms the state-of-the-art semantic segmentation networks (i.e., UNet, FCN-8s, SegNet, DeepLabV3+, etc.) and lightweight networks (i.e., ShuffleNet, MobileNet, and SqueezeNet). All the code and data are available at a GitHub repository (https://github.com/RobotvisionLab/HFENet).

The advancement of Internet of Things (IoT) technologies leads to a wide penetration and large-scale deployment of IoT systems across an entire city or even country. While IoT systems are capable of providing intelligent services, the large amount of data collected and processed in IoT systems also raises serious security concerns. Many research efforts have been devoted to design intelligent network intrusion detection system (NIDS) to prevent misuse of IoT data across smart applications. However, existing approaches may suffer from the issue of limited and imbalanced attack data when training the detection model, which make the system vulnerable especially for those unknown type attacks. In this study, a novel hierarchical adversarial attack (HAA) generation method is introduced to realize the level-aware black-box adversarial attack strategy, targeting the graph neural network (GNN)-based intrusion detection in IoT systems with a limited budget. By constructing a shadow GNN model, an intelligent mechanism based on a saliency map technique is designed to generate adversarial examples by effectively identifying and modifying the critical feature elements with minimal perturbations. A hierarchical node selection algorithm based on random walk with restart (RWR) is developed to select a set of more vulnerable nodes with high attack priority, considering their structural features, and overall loss changes within the targeted IoT network. The proposed HAA generation method is evaluated using the open-source data set UNSW-SOSR2019 with three baseline methods. Comparison results demonstrate its ability in degrading the classification precision by more than 30% in the two state-of-the-art GNN models, GCN and JK-Net, respectively, for NIDS in IoT environments.

Modern Internet-of-Things (IoT) applications are heavily data driven and often require reliable data streams to achieve high-quality data mining. The concept of edge computing is introduced to reduce data latency and communication bandwidth between the cloud server and IoT edge devices. However, inefficient routing that may cause transmission failure or unnecessary data (re)transmission is still a key obstacle to obtain good and reliable data mining results. In this article, network coding combined with opportunistic routing is used to improve energy efficiency in wireless IoT infrastructure, considering the existence of link correlation. Studies have shown that packet receptions on wireless links are correlated, which is completely contrary to the assumption of link independence used in existing routing mechanisms. This assumption causes estimation errors in the calculation of expected number of transmissions for forwarders, which further affects the selection of forwarder set, and ultimately affects the performance of the protocol. We propose an intrasession network coding mechanism based on the mining of link correlation. A novel smart routing method is proposed to accurately estimate the number of transmissions required by forwarders, together with an algorithm for selecting a forwarder set with more optimal number of transmissions. Simulation results demonstrate that the proposed mechanism can achieve fewer transmissions and offer more energy-efficient communications for wireless edge IoT applications.

Given the explosive growth of portable devices, mobile crowdsensing (MCS) is becoming an essential approach that fully utilizes pervasive idle resources to accomplish sensing tasks. The traditional MCS relies on the centralized server for task handle is susceptible to a single point of failure. Targeting this security issue, researchers have proposed a series of blockchain-based MCS. However, nodes in the blockchain suffer from high computation cost for data processing. Simultaneously, most blockchain-based MCS systems lack an efficient incentive mechanism for service requesters and workers. In this work, we integrate the smart contract and mobile devices to establish a secure, interactive, and fair blockchain-based MCS system called BSIF. To prevent illegitimate participants, BSIF requests all users to verify their identities using private keys from the registration phase. In the case of worker location privacy leakage, the location-based symmetric key generator is adopted to coordinate a session key for target range worker selection. Besides, we transfer the data evaluation process to the requester side (e.g., a personal computer), reducing computation cost in the blockchain nodes. Due to the homomorphic feature of the Paillier Cryptosystem and common interest, the requester cannot violate the directives from the blockchain. Subsequently, the Stackelberg game is adopted to investigate the participation level of the workers and the fair reward mechanism for the requesters to achieve a dynamic balance. Finally, the security analysis and performance evaluation demonstrate that our BSIF can defend against possible adversaries while significantly cutting overhead and giving participants the utmost incentive.

Since the outbreak of COVID-19 pandemic in 2020, a dramatic loss of human life has occurred and this trend presents an unprecedented challenge to public health, economic systems and social operations. Hence, it is urgent for us to take some countermeasures to restrain and dispel epidemic diffusion to the uttermost. Data freshness plays an inevitable role in timely infestor determination during this process. However, existing works pay little attention to optimizing this indicator in health monitoring. To make up this research gap, in this paper, we propose a mixed game-based Age of Information (AoI) optimization scheme, where the edge-based wireless technologies and AI-empowered diagnostic bots are adopted. Firstly, we establish the system model for Epidemic Prevention and Control Center (EPCC)-based health state monitoring network, where ultimate biosensing data is transmitted from AI bots via edge servers. Then, upon deriving AoI expression with a closed form, the minimization goal between edge servers and bots is specified. Simultaneously, we reformulate the AoI optimization problem from the mixed game viewpoint (i.e., coalition formation game and ordinary potential game), and then propose two algorithms for cooperative order-based bot deployment and stochastic learning-based channel selection. Finally, compared with the typical baselines, the experiment result shows our scheme can reach the lower AoI value for biosensing data transmission under different parameter settings.

The rapid advancement of cloud computing has promoted the development of cloud storage services. One of the biggest concerns of cloud users is whether the completeness and recoverability of data can be guaranteed when cloud servers encounter problems. Only when the integrity of data is fully guaranteed can users consume cloud storage with confidence, especially in a complicated cloud environment with multiple clouds. However, the literature still lacks a thorough survey on cloud data integrity auditing for both single copy and multiple replicas. In this article, we survey and compare existing auditing schemes for single copy and multiple replicas based on a set of criteria. Based on our review and analysis, we discuss open issues, potential applications and future directions in the field of the integrity auditing in the cloud, including the implications of such trendy topics as merging blockchain and edge computing into data integrity auditing.

Security-related data collection is an essential part for attack detection and security measurement in Mobile Ad Hoc Networks (MANETs). A detection node (i.e., collector) should discover available routes to a collection node for data collection and collect security-related data during route discovery for determining reliable routes. However, few studies provide incentives for security-related data collection in MANETs. In this article, we propose B4SDC, a blockchain system for security-related data collection in MANETs. Through controlling the scale of Route REQuest (RREQ) forwarding in route discovery, the collector can constrain its payment and simultaneously make each forwarder of control information (namely RREQs and Route REPlies, in short RREPs) obtain rewards as much as possible to ensure fairness. At the same time, B4SDC avoids collusion attacks with cooperative receipt reporting, and spoofing attacks by adopting a secure digital signature. Based on a novel Proof-of-Stake consensus mechanism by accumulating stakes through message forwarding, B4SDC not only provides incentives for all participating nodes, but also avoids forking and ensures high efficiency and real decentralization. We analyze B4SDC in terms of incentives and security, and evaluate its performance through simulations. The thorough analysis and experimental results show the efficacy and effectiveness of B4SDC.

With the emergence of various convolutional neural network (CNN)-based applications and the rapid growth of CNN model scale, the resource-constricted end devices can hardly deploy CNN-based applications. Current work optimizes the CNN model on edge servers and deploys the optimized model on devices in an edge computing environment. However, most of them only optimize the resource consumption within or across models solely, whereas neglecting the other side. In this article, we propose a novel CNN-based resource optimization approach (CroApp) that not only optimizes the resource consumption within the CNN model but also pays attention to resource optimization across the applications. Specifically, we adopt model compression as the “inner-model” optimization method, as well as computation sharing as the “intermodel” optimization method. First, during “inner-model” optimization, the CroApp prunes unnecessary parameters within the model on edge servers to reduce the scale of the model. Then, during “intermodel” optimization, the CroApp trains a set of shareable models based on the pruned model and sends these shareable models to end devices. Finally, the CroApp adaptively adjusts the shared models to reduce resource consumption. The experimental results show that the CroApp outperforms the state-of-the-art approaches in terms of resource reduction, scalability, and application performance.

Along with the popularity of the Internet of Things (IoT) techniques with several computational paradigms, such as cloud and edge computing, microservice has been viewed as a promising architecture in large-scale application design and deployment. Due to the limited computing ability of edge devices in distributed IoT, only a small scale of data can be used for model training. In addition, most of the machine-learning-based intrusion detection methods are insufficient when dealing with imbalanced dataset under limited computing resources. In this article, we propose an optimized intra/inter-class-structure-based variational few-shot learning (OICS-VFSL) model to overcome a specific out-of-distribution problem in imbalanced learning, and to improve the microservice-oriented intrusion detection in distributed IoT systems. Following a newly designed VFSL framework, an intra/inter-class optimization scheme is developed using reconstructed feature embeddings, in which the intra-class distance is optimized based on the approximation during a variation Bayesian process, while the inter-class distance is optimized based on the maximization of similarities during a feature concatenation process. An intelligent intrusion detection algorithm is, then, introduced to improve the multiclass classification via a nonlinear neural network. Evaluation experiments are conducted using two public datasets to demonstrate the effectiveness of our proposed model, especially in detecting novel attacks with extremely imbalanced data, compared with four baseline methods.

Various cyber attacks often occur in logistics network of the Industry 4.0, which poses a threat to Internet security. Intrusion detection can intelligently detect anomalous activities and secure the Internet with the help of anomaly detection algorithms. Different from static data, intrusion detection data are a dynamic data form and have the following characteristics. First, it is multiaspect. Second, it contains point anomalies and group anomalies. Third, there are correlations between different attributes. Nevertheless, these properties pose a challenge on existing anomaly detection approaches. Thus, a novel anomaly detection approach MDS_AD is proposed in this article to deal with the challenges. It combines locality-sensitive hashing (LSH), isolation forest, and PCA techniques. MDS_AD has the following properties. 1) The introduced LSH can operate on multiaspect data. 2) MDS_AD can effectively catch group anomalies from the experimental results. 3) The PCA is utilized to reduce dimensionality for correlations between different attributes. 4) MDS_AD is a streaming approach, which can perform model update and process data in constant memory and time. To confirm the performance of MDS_AD, multiple experiments are designed and implemented on UNSW-NB15 dataset. Experimental results show that MDS_AD outperforms state-of-the-art baselines.

With the bloom of intelligent devices over the world, a large scale of user’s trajectory data are collected. How to mine valuable rules from these data and provide services for the industrial community has become an urgent problem. In this article, we propose a multimodal prediction system to infer users’ mobility pattern embedded in heterogeneous data from cyber–physical–social space. According to users’ mobility pattern, the framework can provide smart services for the industrial community. The highlight is the retrospective multivariate multistep transition tensor (${\text{M}^2}{\text{T}^2}$) chain model, which decomposes a large scale of${\text{M}^2}{\text{T}^2}$into a series of small-scale transition tensors (subtransition tensors) with the tensor maximum likelihood estimation method. Then, that one can solve the stationary probability distribution with the small-scale subtransition tensors so as to highly reduce the computation and storage cost. At the same time, the tensor maximum likelihood estimation method avoids the overfitting of${\text{M}^2}{\text{T}^2}$, so the proposed model improves the performance of prediction systems. In the end, several experiments are constructed to evaluate the proposed model.

Smart manufacturing aims to support highly customizable production processes. Therefore, the associated machine intelligence needs to be quickly adaptable to new products, processes, and applications with limited training data while preserving data privacy. In this article, a new federated transfer learning framework, federated transfer learning for cross-domain prediction, is proposed to address the challenges of data scarcity and data privacy faced by most machine learning approaches in modern smart manufacturing with cross-domain applications. The framework architecture consists of a central server and several groups of smart devices, where each group handles a different application. The existing applications can share their knowledge through the central server as base models, while new applications can convert a base model to their target-domain models with limited application-specific data using a transfer learning technique. Meanwhile, the federated learning scheme is deployed within a group to further enhance the accuracy of the application-specific model. The integrated framework allows model sharing across the central server and different smart devices without exposing any raw data and, hence, protects the data privacy. Two public datasets, COCO and PETS2009, which represent the source and target applications, are employed for evaluations. The simulation results show that the proposed method outperforms two state-of-the-art machine learning approaches by achieving better learning efficiency and accuracy.

Multimodal image fusion is the process of combing relevant biological information that can be used for automated industrial application. In this article, we present a novel framework combining fractal constraint with group sparsity to achieve the optimal fusion quality. First, we adopt the idea of patch division and componentwise separation to perceive the fractal characteristics across multimodality sources. Then, to preserve the spatial information against the redundancy of component-entanglement, the group sparsity is proposed. A dual variable weighting rule is inherently embedded to mitigate the overfitting across the component penalty. Furthermore, the alternating direction method of multipliers is conducted to the proposed model optimization. The experiments show that our model has a better performance in quantitative visual quality and qualitative evaluation analysis. Finally, a real segmentation application of positron emission tomography/computed tomography image fusion proves the effectiveness of our algorithm.

Recently, along with several technological advancements in cyber-physical systems, the revolution of Industry 4.0 has brought in an emerging concept named digital twin (DT), which shows its potential to break the barrier between the physical and cyber space in smart manufacturing. However, it is still difficult to analyze and estimate the real-time structural and environmental parameters in terms of their dynamic changes in digital twinning, especially when facing detection tasks of multiple small objects from a large-scale scene with complex contexts in modern manufacturing environments. In this article, we focus on a small object detection model for DT, aiming to realize the dynamic synchronization between a physical manufacturing system and its virtual representation. Three significant elements, including equipment, product, and operator, are considered as the basic environmental parameters to represent and estimate the dynamic characteristics and real-time changes in building a generic DT system of smart manufacturing workshop. A hybrid deep neural network model, based on the integration of MobileNetv2, YOLOv4, and Openpose, is constructed to identify the real-time status from physical manufacturing environment to virtual space. A learning algorithm is then developed to realize the efficient multitype small object detection based on the feature integration and fusion from both shallow and deep layers, in order to facilitate the modeling, monitoring, and optimizing of the whole manufacturing process in the DT system. Experiments and evaluations conducted in three different use cases demonstrate the effectiveness and usefulness of our proposed method, which can achieve a higher detection accuracy for DT in smart manufacturing.

Sleep staging is an important step in analyzing sleep quality. Traditional manual analysis by psychologists is time-consuming. In this paper, we propose an automatic sleep staging model with an improved attention module and hidden Markov model (HMM). The model is driven by single-channel electroencephalogram (EEG) data. It automatically extracts features through two convolution kernels with different scales. Subsequently, an improved attention module based on Squeeze-and-Excitation Networks (SENet) will perform feature fusion. The neural network will give a preliminary sleep stage based on the learned features. Finally, an HMM will apply sleep transition rules to refine the classification. The proposed method is tested on the sleep-EDFx dataset and achieves excellent performance. The accuracy on the Fpz-Cz channel is 84.6%, and the kappa coefficient is 0.79. For the Pz-Oz channel, the accuracy is 82.3% and kappa is 0.76. The experimental results show that the attention mechanism plays a positive role in feature fusion. And our improved attention module improves the classification performance. In addition, applying sleep transition rules through HMM helps to improve performance, especially N1, which is difficult to identify.

With the prosperity of Industry 4.0, numerous emerging industries continue to gain popularity and their market scales are expanding ceaselessly. The Internet of Vehicles (IoV), one of the thriving intelligent industries, enjoys bright development prospects. However, at the same time, the reliability and availability of IoV applications are confronted with two major bottlenecks of time delay and energy consumption. To make matters worse, massive heterogeneous and multi-dimensional multimedia data generated on the IoV present a huge obstacle to effective data analysis. Fortunately, the advent of edge computing technology enables tasks to be offloaded to edge servers, which significantly reduces total overhead of IoV systems. Deep reinforcement learning (DRL), equipped with its excellent perception and decision-making capability, is undoubtedly a dominant technology to solve task offloading problems. In this article, we first employ an optimized Fuzzy C-means algorithm to cluster vehicles and other edge devices according to their respective service quality requirements. Then, we employ an election algorithm to assist in maintaining the stability of the IoV. Last, we propose a task-offloading algorithm based on the Deep Q Network (DQN) to acquire an optimal task offloading scheme. Massive simulation experiments demonstrate the superiority of our method in minimizing time delay and energy consumption.

In this paper, we focus on antieavesdropping design in a multicell multiuser interference channel coexisting with a multiantenna eavesdropper, in which multiuser interference arises as a nonneglectable factor in securing communication. Supposing the eavesdropper is equipped with an arbitrary number of antennas, we jointly exploit the role of inherent multiuser interference and artificial noise (AN) to enhance security, and propose a noniterative secure transceiver design under a multiple input multiple output (MIMO) framework. The quantity relationship of system parameters is then analyzed to ensure feasibility. And the achievable secrecy rate is then derived without any knowledge of the eavesdropper. Finally, to balance the power allocated to AN and secrecy data, a power allocation strategy aiming at maximizing the achievable secrecy rate is designed, while guaranteeing legitimate users the required quality of service. With the adopted design, both the multiuser interference and AN are leveraged to facilitate communication security such that the proposed secure transceiver design can adapt to changes in eavesdropping antennas. Extensive numerical results have verified our analysis and demonstrated that the proposed power allocation strategy outperforms the baseline algorithms in terms of the achievable secrecy rate.

The newly emerging federated learning (FL) framework offers a new way to train machine learning models in a privacy-preserving manner. However, traditional FL algorithms are based on an event-triggered aggregation, which suffers from stragglers and communication overhead issues. To address these issues, in this paper, we present a time-triggered FL algorithm (TT-Fed) over wireless networks, which is a generalized form of classic synchronous and asynchronous FL. Taking the constrained resource and unreliable nature of wireless communication into account, we jointly study the user selection and bandwidth optimization problem to minimize the FL training loss. To solve this joint optimization problem, we provide a thorough convergence analysis for TT-Fed. Based on the obtained analytical convergence upper bound, the optimization problem is decomposed into tractable sub-problems with respect to each global aggregation round, and finally solved by our proposed online search algorithm. Simulation results show that compared to asynchronous FL (FedAsync) and FL with asynchronous user tiers (FedAT) benchmarks, our proposed TT-Fed algorithm improves the converged test accuracy by up to 12.5% and 5%, respectively, under highly imbalanced and non-IID data, while substantially reducing the communication overhead.

Along with the rapid development of cloud computing, IoT, and AI technologies, cloud video surveillance (CVS) has become a hotly discussed topic, especially when facing the requirement of real-time analysis in smart applications. Object detection usually plays an important role for environment monitoring and activity tracking in surveillance system. The emerging edge-cloud computing paradigm provides us an opportunity to deal with the continuously generated huge amount of surveillance data in an on-site manner across IoT systems. However, the detection performance is still far away from satisfactions due to the complex surveilling environment. In this study, we focus on the multitarget detection for real-time surveillance in smart IoT systems. A newly designed deep neural network model called A-YONet, which is constructed by combining the advantages of YOLO and MTCNN, is proposed to be deployed in an end-edge-cloud surveillance system, in order to realize the lightweight training and feature learning with limited computing sources. An intelligent detection algorithm is then developed based on a preadjusting scheme of anchor box and a multilevel feature fusion mechanism. Experiments and evaluations using two data sets, including one public data set and one homemade data set obtained in a real surveillance system, demonstrate the effectiveness of our proposed method in enhancing training efficiency and detection precision, especially for multitarget detection in smart IoT application developments.

The rapidly developed Health 2.0 technology has provided people with more opportunities to conduct online medical consultation than ever before. Understanding contexts within different online medical communications and activities becomes a significant issue to facilitate patients' medical decision making process. As a subcategory of machine learning, neural networks have drawn increasing attentions in natural language processing applications. In this article, we focus on modeling and analyzing the patient-physician-generated data based on an integrated CNN-RNN framework, in order to deal with the situation that patients' online inquiries are usually not very long. A so-called DP-CRNN algorithm is developed with a newly designed neural network structure, to extract and highlight the combination of semantic and sequential features in terms of patient's inquiries. An intelligent recommendation method is then proposed to provide patients with automatic clinic guidance and pre-diagnosis suggestions, in which a clustering mechanism is utilized to refine the learning process with more precise diagnosis scope and more representative features. Experiments based on the collected real world data demonstrate the effectiveness of our proposed model and method for intelligent pre-diagnosis service in online medical environments.

The advancement of several significant technologies, such as artificial intelligence, cyber intelligence, and machine learning, has made big data penetrate not only into the industry and academic field but also our daily life along with a variety of cyber-enabled applications. In this article, we focus on a deep correlation mining method in heterogeneous big data environments. A hierarchical hybrid network (HHN) model is constructed to describe multitype relationships among different entities, and a series of measures are defined to quantify the internal correlations within one specific layer or external correlations between different layers. An intelligent router based on deep reinforcement learning framework is designed to generate optimal actions to route across the HHN. An improved random walk with the restart-based algorithm is then developed with the intelligent router, based on the hierarchical influence across network associated with multiple correlations. An intelligent recommendation mechanism is finally designed and applied to support users' collaboration works in scholarly big data environments. Experiments based on DBLP and ResearchGate data show the practicability and usefulness of our model and method.

For industrial applications of the artificial intelligence of things, mechanical control usually affects the overall product output and production schedule. Recently, more and more engineers have applied the deep reinforcement learning method to mechanical control to improve the company's profit. However, the problem of deep reinforcement learning training stage is that overfitting often occurs, which results in accidental control and increases the risk of overcontrol. In order to address this problem, in this article, an expected advantage learning method is proposed for moderating the maximum value of expectation-based deep reinforcement learning for industrial applications. With the tanh softmax policy of the softmax function, we replace the sigmod function with the tanh function as the softmax function activation value. It makes it so that the proposed expectation-based method can successfully decrease the value overfitting in cognitive computing. In the experimental results, the performance of the Deep Q Network algorithm, advantage learning algorithm, and propose expected advantage learning method were evaluated in every episodes with the four criteria: the total score, total step, average score, and highest score. Comparing with the AL algorithm, the total score of the proposed expected advantage learning method is increased by 6% in the same number of trainings. This shows that the action probability distribution of the proposed expected advantage learning method has better performance than the traditional soft-max strategy for the optimal setting control of industrial applications.

With the increasing population of Industry 4.0, industrial big data (IBD) has become a hotly discussed topic in digital and intelligent industry field. The security problem existing in the signal processing on large scale of data stream is still a challenge issue in industrial internet of things, especially when dealing with the high-dimensional anomaly detection for intelligent industrial application. In this article, to mitigate the inconsistency between dimensionality reduction and feature retention in imbalanced IBD, we propose a variational long short-term memory (VLSTM) learning model for intelligent anomaly detection based on reconstructed feature representation. An encoder-decoder neural network associated with a variational reparameterization scheme is designed to learn the low-dimensional feature representation from high-dimensional raw data. Three loss functions are defined and quantified to constrain the reconstructed hidden variable into a more explicit and meaningful form. A lightweight estimation network is then fed with the refined feature representation to identify anomalies in IBD. Experiments using a public IBD dataset named UNSW-NB15 demonstrate that the proposed VLSTM model can efficiently cope with imbalance and high-dimensional issues, and significantly improve the accuracy and reduce the false rate in anomaly detection for IBD according to F1, area under curve (AUC), and false alarm rate (FAR).

With the increasing population of Industry 4.0, both AI and smart techniques have been applied and become hotly discussed topics in industrial cyber-physical systems (CPS). Intelligent anomaly detection for identifying cyber-physical attacks to guarantee the work efficiency and safety is still a challenging issue, especially when dealing with few labeled data for cyber-physical security protection. In this article, we propose a few-shot learning model with Siamese convolutional neural network (FSL-SCNN), to alleviate the over-fitting issue and enhance the accuracy for intelligent anomaly detection in industrial CPS. A Siamese CNN encoding network is constructed to measure distances of input samples based on their optimized feature representations. A robust cost function design including three specific losses is then proposed to enhance the efficiency of training process. An intelligent anomaly detection algorithm is developed finally. Experiment results based on a fully labeled public dataset and a few labeled dataset demonstrate that our proposed FSL-SCNN can significantly improve false alarm rate (FAR) and F1 scores when detecting intrusion signals for industrial CPS security protection.

The Internet of Vehicles (IoV) connects vehicles, roadside units (RSUs) and other intelligent objects, enabling data sharing among them, thereby improving the efficiency of urban traffic and safety. Currently, collections of multimedia content, generated by multimedia surveillance equipment, vehicles, and so on, are transmitted to edge servers for implementation, because edge computing is a formidable paradigm for accommodating multimedia services with low-latency resource provisioning. However, the uneven or discrete distribution of the traffic flow covered by edge servers negatively affects the service performance (e.g., overload and underload) of edge servers in multimedia IoV systems. Therefore, how to accurately schedule and dynamically reserve proper numbers of resources for multimedia services in edge servers is still challenging. To address this challenge, a traffic flow prediction driven resource reservation method, called TripRes, is developed in this article. Specifically, the city map is divided into different regions, and the edge servers in a region are treated as a “big edge server” to simplify the complex distribution of edge servers. Then, future traffic flows are predicted using the deep spatiotemporal residual network (ST-ResNet), and future traffic flows are used to estimate the amount of multimedia services each region needs to offload to the edge servers. With the number of services to be offloaded in each region, their offloading destinations are determined through latency-sensitive transmission path selection. Finally, the performance of TripRes is evaluated using real-world big data with over 100M multimedia surveillance records from RSUs in Nanjing China.

With the high mobility, flexibility and manoeuvrability, unmanned aerial vehicles (UAVs)‐enabled mobile edge computing (MEC) plays an important role in many applications that lack of available terrestrial infrastructures, such as disaster response, emergency relief and military scenarios. However, the high mobility of UAVs may severely degrade the implementation of UAV‐enabled MEC in computation‐intensive or delay‐sensitive scenarios from a physical layer signalling perspective, e.g. frequency selective fading effect caused by the low altitude and low elevation angle routes of UAVs. In this study, a generalised weighted‐type fractional Fourier transform (GWFRFT) based signal energy‐spreading transmission (EST) scheme is proposed for signalling and quality of service (QoS) provisioning for UAV‐enabled MEC. In the proposed scheme, the communication signal energy is redistributed evenly at the time‐frequency plane to enhance the robustness of UAV communications against the severe fading channels. The signalling provisioning mechanism of the GWFRFT‐EST scheme is analysed and simulated under poor air‐to‐ground channels with low Rician K ‐factor. Simulation results show that the proposed method greatly improves the bit error rate performance with a negligible computational complexity increase.

Along with the advancement of several emerging computing paradigms and technologies, such as cloud computing, mobile computing, artificial intelligence, and big data, Internet of Things (IoT) technologies have been applied in a variety of fields. In particular, the Internet of Healthcare Things (IoHT) is becoming increasingly important in human activity recognition (HAR) due to the rapid development of wearable and mobile devices. In this article, we focus on the deep-learning-enhanced HAR in IoHT environments. A semisupervised deep learning framework is designed and built for more accurate HAR, which efficiently uses and analyzes the weakly labeled sensor data to train the classifier learning model. To better solve the problem of the inadequately labeled sample, an intelligent autolabeling scheme based on deep Q-network (DQN) is developed with a newly designed distance-based reward rule which can improve the learning efficiency in IoT environments. A multisensor based data fusion mechanism is then developed to seamlessly integrate the on-body sensor data, context sensor data, and personal profile data together, and a long short-term memory (LSTM)-based classification method is proposed to identify fine-grained patterns according to the high-level features contextually extracted from the sequential motion data. Finally, experiments and evaluations are conducted to demonstrate the usefulness and effectiveness of the proposed method using real-world data.

With the development of GPS technology, a new Mobile Internet of Things (M-IoT) is emerging, which perceives the city's rhythm and pulse day and night to collect a large scale of city data. It is urgent to innovate M-IoT service system for these large-scale and heterogeneous data. To cope with the problem, this article proposes a Mobile-IoT based multi-modal reinforcement learning service framework from data perspective, which has three highlights, i) Developing Action-aware High-order Transition Tensor (AHTT) to fuse the heterogeneous data from M-IoTs in a unified form. ii) Developing Multi-modal Markov Decision Process (MMDP) to model the multi-modal reinforcement learning for M-IoT service framework. iii) Developing Tensor Policy Iteration algorithm (TPIA) to solve the optimal tensor policy. Due to using tensor keeps the multi-modal relations of the context information in the process of solving the optimal policy. The proposed M-IoT service system provides more personalized service for taxi drivers. The experiment results shows that most taxi drivers earn more revenue according to the tensor policy.

In the cognitive radio scenario, the case of extremely low SNR is often encountered, which incurs a deceasing of detection performance. To circumvent this difficulty, we propose a promising spectrum sensing algorithm by exploiting dictionary learning. In the proposed algorithm, the energy of maximum sparse component acquired by dictionary learning is utilized as a test statistic to improve detection performance as much as possible. More importantly, we extract the maximum sparse component of the received signal via a learned dictionary and the forced sparsity 1 to maximize the SNR of maximum sparse component. Some simulations are carried out to verify theoretical results. Simulation results show that the proposed algorithm overwhelmingly outperforms the sparse-representation-denosing-based algorithm and the conventional energy-based algorithm in the extremely low SNR case.

Along with the development of several emerging computing paradigms and information communication technologies, it is said that cyber computing technology is playing an increasingly important role across cyber-related systems and applications. In this article, we focus on cyber-social computing and propose a computational model that integrates large-scale group decision-making (LSGDM) into social recommendations for cyber-enabled online services. As a concrete application example, a graph model is built to describe the LSGDM problem among researchers in scholarly big data environments. Following the basic profiling to describe decision-makers within scholarly networks, measures are defined to evaluate one researcher's academic performance and research outcome and further quantify correlations between them based on their collaboration relationships in a constructed network model. A two-stage large-scale decision-making solution is then proposed for social recommendations: A network partition algorithm is developed based on the identification of experts along with their influence extending to a group of researchers, and a random walk with the restart-based algorithm is improved to calculate the weighted decisions for group decision aggregation and alternative ranking. Experiments using the real-world data demonstrate the usefulness and effectiveness of our proposed model and method, which can provide the target researcher with more reliable recommendations.

Recently, health social media have engaged more and more people to share their personal feelings, opinions, and experience in the context of health informatics, which has drawn increasing attention from both academia and industry. In this paper, we focus on the behavioral influence analysis based on heterogeneous health data generated in social media environments. An integrated deep neural network (DNN)-based learning model is designed to analyze and describe the latent behavioral influence hidden across multiple modalities, in which a convolutional neural network (CNN)-based framework is used to extract the time-series features within a certain social context. The learned features based on cross-modality influence analysis are then trained in a SoftMax classifier, which can result in a restructured representation of high-level features for online physician rating and classification in a data-driven way. Finally, two algorithms within two representative application scenarios are developed to provide patients with personalized recommendations in health social media environments. Experiments using the real world data demonstrate the effectiveness of our proposed model and method.

As the blind zone phenomenon becoming more and more common in the next generation communication system, the blind zone recognition technique has gain much attention. One big challenge in blind zone recognition is the fluctuation of received signal strength (RSS) especially in indoor environment where multipath effect is severe. Previous methods using median filter are under the improper hypothesis which results in unsatisfying results. Moreover, traditional blind zone recognition methods mainly focus on outdoor environment and are with the characteristics of low efficiency, inflexibility, and high cost. In this paper, we propose a new algorithm based on fractional Fourier transform to address the RSS fluctuation and solve the problem of blind zone recognition in indoor environment. The comparison to our method and median filter method shows that our method provides much more accurate results.

While social computing related research has focused mostly on how to provide users with more precise and direct information, or on recommending new search methods to find requested information rapidly, the authors believe that network users themselves could be viewed as an important social resource. This study concentrates on analyzing potential and dynamic user correlations, based on topic-aware similarity and behavioral influence, which may help us to discover communities in social networking sites. The dynamically socialized user networking (DSUN) model is extended and refined to represent implicit and explicit user relationships in terms of topic-aware features and social behaviors. A set of measures is defined to describe and quantify interuser correlations, relating to social behaviors. Three types of ties are proposed to describe and discover communities according to influence-based user relationships. Results of the experiment with Twitter data are used to show the discovery of three types of communities, based on the presented model. Comparison with six different schemes and two existing methods demonstrates that the proposed method is effective in discovering influence-based communities. Finally, the scenario-based simulation of collective decision-making processes demonstrates the practicability of the proposed model and method in social interactive systems.