In multi-agent systems, Deep Q Network (DQN) is a basic algorithm that utilize the representational ability of deep learning and the decision-making capacity of reinforcement learning to tackle complex tasks in control and optimization problems. However, standard DQN algorithm faces challenges in generalization and robustness, especially when dealing with varying noise levels in practical environments. To address these challenges, this paper proposes Mutual Deep Q Networks (Mut-DQN), a multi-agent algorithm that combines mutual learning with Deep Reinforcement Learning (DRL). Mut-DQN employs multiple agents with independently initialized Q networks. These agents learn mutually by minimizing a mixed loss function, which includes both Temporal Difference (TD) error and mutual loss. The mutual loss is specifically designed to reduce the variance among different agents’ actions when they encounter the same environment states. By minimizing mutual loss, the system promotes knowledge exchange between agents, enhancing the diversity and generalization of their behaviors. Our experiments demonstrate that Mut-DQN can resist parameter and environment noise compared to standard DQN significantly, meaning that the Mut-DQN can reach a wide and stable solution space. Mut-DQN’s capacity to enable information sharing among agents enhances both robustness and generalization. Such improvement is essential for multi-agent systems that interact frequently with dynamic and noisy environments.

Accurate analysis of traffic flow (TF) data is crucial for the vehicular applications. Conventional deep learning models require task-specific training and are susceptible to high-frequency disturbances, degrading the feature representation capability. To overcome these limitations, this paper proposes a Token-based SelfSupervised Network (TSSN) that can learn TF features in both tokenization and task-agnostic manners. It provides a properly bootstrapped pre-training model for various downstream tasks. In support of the edge computing and vehicular cloud computing, the pooled computational resources facilitate real-time inferences of downstream models. In TSSN, TF data are segmented into tokens. A pretext task, named as Masked Token Prediction (MTP), is then developed to allow TSSN to understand the underlying correlations of TF by predicting randomly masked tokens. By utilizing MTP, TSSN is able to extract the high-level intrinsic semantics of TF, and provide general-purpose token embeddings, leading to improved overall performance and enhanced ability to adapt to different tasks. By substituting the last fully-connected layers with a group of untrained new layers and fine-tuning using small-scale task-specific data, TSSN can be utilized for a variety of downstream tasks in vehicular applications. Simulation results indicate that the TSSN enhances overall performance in comparison to state-of-the-art models.

With the wide range of Internet of Things (IoT) applications, federated learning (FL) is commonly adopted to protect the privacy of IoT data. FL enables privacy-preserving model training while keeping the data locally available. To alleviate the additional load caused by FL, an improved hierarchical aggregation framework is presented in this article to decentralize the model aggregation tasks based on end-device clusters. However, when applying FL to IoT networks, how to keep high efficiency and reliability remains open challenges due to a large number and vulnerability of IoT end devices. In this article, we propose a blockchain-assisted aggregation scheme for FL in IoT networks, where the aggregation node selection is applied for efficiency improvement as well as blockchain for performance verification. During model aggregation, a selection strategy is obtained by the deep deterministic policy gradient (DDPG) algorithm and aims to select the optimal subset of IoT end devices based on multiple metrics. Furthermore, a new performance verification based on the characteristics of blockchain is applied to achieve mutual verification among a number of untrustworthy nodes with the optimal stopping theory, which provides reliable model performance proofs. Simulation results show that the proposed scheme can maintain FL efficiency and reduce the system latency while protecting data privacy.

With a plethora of remote sensing (RS) images, deep neural network-based semantic segmentation models (SegModels) achieve commendable road extraction performance. However, the occlusions caused by vehicles, roadside objects and shadows cannot be directly identified as road pixels, especially on high-resolution RS images. Therefore, relying only on a single SegModel to guarantee road connectivity and boundary smoothness in road extraction tasks is extremely difficult. To address this issue, this paper puts forward a “Segmentation-with-Reconstruction” framework, which comprises a SegModel to generate the binary road labels from RS images, and a reconstruction model to refine the road labels. Specifically, the former can be compatible with arbitrary existing SegModels, while the latter is built by our proposed model named as all-visible denoising auto-encoder (AV-DAE). The AV-DAE is designed to be an encoder-decoder architecture that takes topology-corruption road labels as inputs and true road labels as outputs. To better train the AV-DAE, we further present three noise-adding strategies to corrupt road labels for diverse patterns, and train the AV-DAE to reconstruct them. Being RS-image-agnostic, the AV-DAE pays more attention to the spatial features rather than the spectral features, which enables it to recover the road topology through improving the connectivity and boundary smoothness. Finally, elaborate simulation results demonstrate that the proposed framework can significantly improve the connectivity and boundary smoothness of the extracted roads, while achieving a competitive road extraction performance and high generalization ability, as compared to the benchmarks.

With the rapid development of communication networks, the number of proposals in professional fields has increased explosively. In the era of big data, topic classification and retrieval of proposals has become an important demand for researchers. However, the automatic topic classification of proposals in the professional field has the characteristics of high semantic similarity, that is, fine-grained classification. It is difficult for text classification models in general fields to achieve high-precision classification effects on proposal datasets in professional fields. At the same time, the problem of class imbalance reduces the effective training of the model for minority classes, making the widespread application of the model difficult. In order to solve the proposal classification in professional fields, this paper proposes a keyphrase-enhanced graph convolutional network (KPE-GCN), which uses keyphrase-based data augmentation to effectively alleviate the problem of class imbalance. Meanwhile, we build a two-level heterogeneous graph that combines both word-level and keyphrase-level information, which can add domain features to the proposal structure. It can effectively distinguish the differences between fine-grained categories, and improve the accuracy of fine-grained classification of proposal. We perform extensive experiments on the proposal dataset, and KPE-GCN model can exhibit higher classification performance, raising the F1 value to over 99% for the first time. Our KPE-GCN model provides a unified domain feature extraction scheme, which can be widely used in various professional fields.

Code clone detection is of great significance for intellectual property protection and software maintenance. Deep learning has been applied in some research and achieved better performance than traditional methods. To adapt to more application scenarios and improve the detection efficiency, this paper proposes a low-complex code clone detection with the graph- based neural network. As the input of the neural network, code features are represented by abstract syntax trees (ASTs), in which the redundant edges are removed. The operation of pruning avoids interference in the message passing of the network and reduces the size of the graph. Then, the graph pairs for the code clone detection are sent into the message passing neural networks (MPNN). In addition, the gated recurrent unit (GRU) is used to learn the information between graph pairs to avoid the operation of Graph mapping. After multiple iterations, the attention mechanism is used to read out the graph vector, and the cosine similarity is calculated on the graph vector to obtain the code similarity. Through the experiments on two datasets, the results show that the proposed clone detection scheme removes about 20 % of the redundant edges and reduces 25 % of model weights, 16% of multiply-accumulate operations (MACs). In the end, the proposed method effectively reduces the training time of graph neural network while presenting a similar performance to the baseline network.

Short-term load forecasting (STLF) is an important foundation for the electrical network automation and intelligentization. Classic time series methods such as autoregressive integrated moving average (ARIMA)-based methods, and fashionable data mining technologies such as deep learning are not suitable for STLF on the small datasets with a number of attributes. Therefore, in order to tackle this problem, a novel ensemble formulation is proposed to model the load series in this paper. Specifically, the ensemble formulation decomposes the load consumption into two parts, i.e., the extrinsic-variational component (E-com) described by the external factors, and the intrinsic-stationary component (I-com) representing the internal structure of the series itself. Thereafter, a boosting regression learner consists of several simple learners, is further developed to model the E-com and I-com. By localizing the global attributes, the extraction of E-com is achieved by mining the relationships between load patterns and the corresponding local attributes, while I-com is naturally learned by the classic time series methods. Experimental results show that the proposed method improves the accuracy of STLF on small datasets, as compared to the existing approaches in the literature.

The integration of the blockchain and Internet of Things (IoT) systems can effectively guarantee data security in IoT applications. To facilitate the use of blockchain on resource-constrained IoT end devices, we propose RAFT+ with a new leader selection scheme in this article, which is based on the distributed consensus algorithm RAFT. The design of RAFT+ aims at mitigating the imparities between different types of IoT end devices and enabling these devices to allow different types of IoT end devices to participate in block consensus, thus maintaining strong consistency of the blockchain network. The leader selection scheme is generated by a deep${Q}$-Network (DQN), which can make the optimal selection of the leader under various conditions by leveraging the limited system resources as well as balancing the load of the consensus mechanism on multiple IoT end devices. Simulation results show that RAFT+ can enhance the system performance while maintaining the security of the system under high load conditions.

With the emergence of Blockchain-based Internet of Things (BIoT) applications, smart contracts have become one of the most appealing aspects because they reduce the cost and complexity of distributed administration. However, the immaturity of smart contracts may result in significant financial losses or the leakage of sensitive information. This article first investigates the taxonomy of security issues associated with smart contracts considering BIoT scenarios. To address these security concerns and overcome the limitations of existing methods, a tree-based machine learning vulnerability detection (TMLVD) method is proposed to perform the vulnerability analysis of smart contracts. TMLVD feeds the intermediate representations of smart contracts derived from abstract syntax trees (AST) into a tree-based training network for building the prediction model. Multidimensional features are captured by this model to identify smart contracts as vulnerable. The detection phase can be implemented quickly with limited computing resources and the accuracy of the detection results is guaranteed. The experimental evaluation demonstrated the effectiveness and efficiency of TMLVD on a data set comprised of Ethereum smart contracts.

Efficiency and security have become critical issues during the development of the long-range (LoRa) system for Internet-of-Things (IoT) applications. The centralized work method in the LoRa system, where all packages are processed and kept in the central cloud, cannot well exploit the resources in LoRa gateways and also makes it vulnerable to security risks, such as data falsification or data loss. On the other hand, the blockchain has the potential to provide a decentralized and secure infrastructure for the LoRa system. However, there are significant challenges in deploying blockchain at LoRa gateways with limited edge computing abilities. This article proposes a design and implementation of the blockchain-enabled LoRa system with edge computing by using the open-source Hyperledger Fabric, which is called as HyperLoRa. According to different features of LoRa data, a blockchain network with multiple ledgers is designed, each of which stores a specific kind of LoRa data. LoRa gateways can participate in the operations of the blockchain and share the ledger that keep the time-critical network data with small size. Then, the edge computing abilities of LoRa gateways are utilized to handle the join procedure and application packages processing. Furthermore, a HyperLoRa prototype is implemented on embedded hardware, which demonstrates the feasibility of deploying the blockchain into LoRa gateways with limited computing and storage resources. Finally, various experiments are conducted to evaluate the performances of the proposed LoRa system.

The Long Range (LoRa) systems usually process all the computing tasks on the LoRa central server remotely, which brings large latency to Internet of Things (IoT) applications. In this paper, we propose a new design of a LoRa system with edge computing at the LoRa gateway. Our design enables that some of the time computing tasks for latency-sensitive applications can be dealt with timely. The implementation details of the LoRa gateway are presented along with functionality of each component. Finally, comprehensive experiments are conducted to evaluate the performance of the proposed system. The results show that the proposed system can decrease the latency of IoT applications and balance the workloads between the LoRa central server and the LoRa gateway.

By leveraging mobile edge computing (MEC), a huge amount of data generated by Internet of Things (IoT) devices can be processed and analyzed at the network edge. However, the MEC system usually only has the limited virtual resources, which are shared and competed by IoT edge applications. Thus, we propose a resource allocation policy for the IoT edge computing system to improve the efficiency of resource utilization. The objective of the proposed policy is to minimize the long-term weighted sum of average completion time of jobs and average number of requested resources. The resource allocation problem in the MEC system is formulated as a Markov decision process (MDP). A deep reinforcement learning approach is applied to solve the problem. We also propose an improved deep Q-network (DQN) algorithm to learn the policy, where multiple replay memories are applied to separately store the experiences with small mutual influence. Simulation results show that the proposed algorithm has a better convergence performance than the original DQN algorithm, and the corresponding policy outperforms the other reference policies by lower completion time with fewer requested resources.

Long Range (LoRa) has become one of the most popular Low Power Wide Area (LPWA) technologies, which provides a desirable trade-off among communication range, battery life, and deployment cost. In LoRa networks, several transmission parameters can be allocated to ensure efficient and reliable communication. For example, the configuration of the spreading factor allows tuning the data rate and the transmission distance. However, how to dynamically adjust the setting that minimizes the collision probability while meeting the required communication performance is an open challenge. This paper proposes a novel Data Rate and Channel Control (DRCC) scheme for LoRa networks so as to improve wireless resource utilization and support a massive number of LoRa nodes. The scheme estimates channel conditions based on the short-term Data Extraction Rate (DER), and opportunistically adjusts the spreading factor to adapt the variation of channel conditions. Furthermore, the channel control is carried out to balance the link load of all available channels with the global information of the channel usage, which is able to lower the access collisions under dense deployments. Our experiments demonstrate that the proposed DRCC performs well on improving the reliability and capacity compared with other spreading factor allocation schemes in dense deployment scenarios.

Content caching has brought huge potential for the provisioning of non-safety related infotainment services in future vehicular networks. Assisted by multiaccess edge computing, roadside units (RSUs) could become cache-capable and offer fast caching services to moving vehicles for content providers. On the other hand, deep learning makes it possible to accurately estimate the behavior of vehicles, which enables effective proactive caching strategies. However, caching services considering both the mobility of vehicles and storage could incur increased latency and considerable cost due to the cache size needed in RSUs. In this paper, we model such a problem using Markov decision processes, and propose a heuristic Q-learning solution together with vehicle movement predictions based on a long short-term memory network. The optimal caching strategy which minimizes the latency of caching services can be derived by our heuristic εn-greedy training processes. Numerical results demonstrate that our proposed strategy can achieve better performance compared with several baselines under different prediction accuracies.