In multi-agent systems, Deep Q Network (DQN) is a basic algorithm that utilize the representational ability of deep learning and the decision-making capacity of reinforcement learning to tackle complex tasks in control and optimization problems. However, standard DQN algorithm faces challenges in generalization and robustness, especially when dealing with varying noise levels in practical environments. To address these challenges, this paper proposes Mutual Deep Q Networks (Mut-DQN), a multi-agent algorithm that combines mutual learning with Deep Reinforcement Learning (DRL). Mut-DQN employs multiple agents with independently initialized Q networks. These agents learn mutually by minimizing a mixed loss function, which includes both Temporal Difference (TD) error and mutual loss. The mutual loss is specifically designed to reduce the variance among different agents’ actions when they encounter the same environment states. By minimizing mutual loss, the system promotes knowledge exchange between agents, enhancing the diversity and generalization of their behaviors. Our experiments demonstrate that Mut-DQN can resist parameter and environment noise compared to standard DQN significantly, meaning that the Mut-DQN can reach a wide and stable solution space. Mut-DQN’s capacity to enable information sharing among agents enhances both robustness and generalization. Such improvement is essential for multi-agent systems that interact frequently with dynamic and noisy environments.

Accurate analysis of traffic flow (TF) data is crucial for the vehicular applications. Conventional deep learning models require task-specific training and are susceptible to high-frequency disturbances, degrading the feature representation capability. To overcome these limitations, this paper proposes a Token-based SelfSupervised Network (TSSN) that can learn TF features in both tokenization and task-agnostic manners. It provides a properly bootstrapped pre-training model for various downstream tasks. In support of the edge computing and vehicular cloud computing, the pooled computational resources facilitate real-time inferences of downstream models. In TSSN, TF data are segmented into tokens. A pretext task, named as Masked Token Prediction (MTP), is then developed to allow TSSN to understand the underlying correlations of TF by predicting randomly masked tokens. By utilizing MTP, TSSN is able to extract the high-level intrinsic semantics of TF, and provide general-purpose token embeddings, leading to improved overall performance and enhanced ability to adapt to different tasks. By substituting the last fully-connected layers with a group of untrained new layers and fine-tuning using small-scale task-specific data, TSSN can be utilized for a variety of downstream tasks in vehicular applications. Simulation results indicate that the TSSN enhances overall performance in comparison to state-of-the-art models.

With a plethora of remote sensing (RS) images, deep neural network-based semantic segmentation models (SegModels) achieve commendable road extraction performance. However, the occlusions caused by vehicles, roadside objects and shadows cannot be directly identified as road pixels, especially on high-resolution RS images. Therefore, relying only on a single SegModel to guarantee road connectivity and boundary smoothness in road extraction tasks is extremely difficult. To address this issue, this paper puts forward a “Segmentation-with-Reconstruction” framework, which comprises a SegModel to generate the binary road labels from RS images, and a reconstruction model to refine the road labels. Specifically, the former can be compatible with arbitrary existing SegModels, while the latter is built by our proposed model named as all-visible denoising auto-encoder (AV-DAE). The AV-DAE is designed to be an encoder-decoder architecture that takes topology-corruption road labels as inputs and true road labels as outputs. To better train the AV-DAE, we further present three noise-adding strategies to corrupt road labels for diverse patterns, and train the AV-DAE to reconstruct them. Being RS-image-agnostic, the AV-DAE pays more attention to the spatial features rather than the spectral features, which enables it to recover the road topology through improving the connectivity and boundary smoothness. Finally, elaborate simulation results demonstrate that the proposed framework can significantly improve the connectivity and boundary smoothness of the extracted roads, while achieving a competitive road extraction performance and high generalization ability, as compared to the benchmarks.

The geological characteristics of old landslides can provide crucial information for the task of landslide protection. However, detecting old landslides from high-resolution remote sensing images (HRSIs) is of great challenges due to their partially or strongly transformed morphology over a long time and thus the limited difference with their surroundings. Additionally, small-sized datasets can restrict in-depth learning. To address these challenges, this paper proposes a new iterative classification and semantic segmentation network (ICSSN), which can significantly improve both object-level and pixel-level classification performance by iteratively upgrading the feature extraction module shared by the object classification and semantic segmentation networks. To improve the detection performance on small-sized datasets, object-level contrastive learning is employed in the object classification network featuring a siamese network to realize global features extraction, and a sub-object-level contrastive learning method is designed in the semantic segmentation network to efficiently extract salient features from boundaries of landslides. An iterative training strategy is also proposed to fuse features in the semantic space, further improving both the object-level and pixel-level classification performances. The proposed ICSSN is evaluated on a real-world landslide dataset, and experimental results show that it greatly improves both the classification and segmentation accuracy of old landslides. For the semantic segmentation task, compared to the baseline, the F1 score increases from 0.5054 to 0.5448, the mIoU improves from 0.6405 to 0.6610, the landslide IoU grows from 0.3381 to 0.3743, the PA is improved from 0.945 to 0.949, and the object-level detection accuracy of old landslides surges from 0.55 to 0.90. For the object classification task, the F1 score increases from 0.8846 to 0.9230, and the accuracy score is up from 0.8375 to 0.8875.

Short-term load forecasting (STLF) is an important foundation for the electrical network automation and intelligentization. Classic time series methods such as autoregressive integrated moving average (ARIMA)-based methods, and fashionable data mining technologies such as deep learning are not suitable for STLF on the small datasets with a number of attributes. Therefore, in order to tackle this problem, a novel ensemble formulation is proposed to model the load series in this paper. Specifically, the ensemble formulation decomposes the load consumption into two parts, i.e., the extrinsic-variational component (E-com) described by the external factors, and the intrinsic-stationary component (I-com) representing the internal structure of the series itself. Thereafter, a boosting regression learner consists of several simple learners, is further developed to model the E-com and I-com. By localizing the global attributes, the extraction of E-com is achieved by mining the relationships between load patterns and the corresponding local attributes, while I-com is naturally learned by the classic time series methods. Experimental results show that the proposed method improves the accuracy of STLF on small datasets, as compared to the existing approaches in the literature.