Multi-agent reinforcement learning (MARL) requires effective coordination among multiple decision-making agents to achieve joint goals. Approaches based on a global value function face the curse of dimensionality, while fully decomposed centralized training with decentralized execution (CTDE) methods often suffer from relative overgeneralization. Coordination graphs mitigate this issue but typically fail to capture dynamic collaboration patterns that evolve over time and across tasks. We propose Dynamic Deep Factor Graphs (DDFG), a value decomposition algorithm that represents the global value via factor graphs and learns graph structures on the fly through a graph-generation policy, adapting to evolving inter-agent relations. We provide a theoretical upper bound on the approximation error of high-order decompositions and reveal how the maximum order $D$D trades off accuracy against computation, offering guidance for balancing performance and cost. Using max-sum for inference, DDFG efficiently derives joint policies. Experiments on higher-order predator-prey and SMAC show consistent gains over strong value-decomposition baselines, demonstrating improved sample efficiency and robustness in complex settings.

Recent advancements in reinforcement learning (RL) have driven progress across various domains; however, RL algorithms often struggle in complex multiagent environments due to challenges such as instability, low sample efficiency, and the curse of dimensionality. Hierarchical RL (HRL) provides a structured framework for decomposing complex tasks into more manageable subtasks, making it a promising approach for multiagent systems. In this article, we introduce a novel hierarchical architecture that autonomously generates effective subgoals without explicit constraints, thereby enhancing both training stability and adaptability. To further improve sample efficiency and adaptability, we propose a dynamic goal-generation strategy that adjusts subgoals in response to environmental changes. Additionally, we address the critical challenge of credit assignment in multiagent settings by integrating our hierarchical architecture with a modified QMIX network, thereby facilitating more effective strategy coordination. Extensive comparative experiments against state-of-the-art RL algorithms demonstrate that our approach achieves superior convergence speed and overall performance in multiagent environments. These results validate the effectiveness and flexibility of our method in handling complex coordination tasks. The implementation is publicly available at https://github.com/SICC-Group/GMAH

In this paper, we introduce a novel cooperative target tracking algorithm, namely the quantum-inspired belief propagation, aimed at rectifying the limitations observed in existing localization algorithms employed in multi-target cooperative tracking scenarios. Leveraging the principles of quantum superposition, our algorithm seeks to alleviate the uncertainty inherent in message fusion within belief propagation frameworks, thereby enhancing the accuracy and stability of multi-target cooperative localization. The utilization of the quantum Monte Carlo method facilitates the simulation of the message distribution process, with quantum particles embodying the superposition of multiple states concurrently. This approach effectively addresses the intractable integrations encountered in message updating on factor graphs, rendering the algorithm agnostic to the number of particles involved. Moreover, quantum unitary transformations and quantum black-box operations are deployed to encode factor graph function nodes for the propagation of quantum messages. This innovation surmounts the challenge posed by traditional factor graph function nodes’ inability to process quantum messages. Experimental findings corroborated the superiority of the proposed algorithm in terms of accuracy and robustness.

The rapid expansion of the Internet of Things (IoT) has brought forth new challenges and opportunities in securely managing and sharing vast amounts of data generated by connected devices. Blockchain technology, with its decentralization, tamper-resistance, and traceability, offers a promising framework for IoT data sharing but struggles to safeguard smart contracts and sensitive data. Integrating trusted execution environments (TEEs) with blockchain addresses these concerns, enabling secure execution and communication via remote attestation. However, existing remote attestation methods face challenges, including incompatibility across heterogeneous TEEs, inefficiency under frequent authentication, and vulnerability to DoS attacks. To tackle these, we propose a blockchain-based unified remote attestation scheme for IoT. Our three-tier blockchain architecture—comprising a certificate authority (CA) channel, an authoritative channel, and a business channel—separates authentication, attestation, and operations while ensuring auditability. An abstraction layer supports heterogeneous TEEs, and an authoritative blockchain stores authentication reports, enabling secure, frequent attestations. Additionally, a distributed CA system enhances resilience to DoS attacks. Experimental results validate our scheme’s efficiency and security, offering a robust solution for IoT data sharing.

In recent years, swarm robot systems have garnered increasing attention, both in the industry and academia. These collaborative systems demand effective solutions for data storage, sharing, and security to unlock their full potential. To address these needs, this paper introduces a comprehensive distributed storage and computation framework based on blockchain and federated learning technology. The framework enables real-time collaborative data storage and computation, ensuring the security and reliability of collective intelligence systems. For data storage, we combine blockchain and dynamic containers to achieve secure and efficient storage of diverse robot data. To facilitate secure data utilization and sharing among robots, we present a federated learning-based collaborative computation approach. It allows robots to exchange model parameters while safeguarding data security, providing a versatile collaborative computation framework for collective systems. To validate the security and resilience of our framework, we present a practical scenario involving multi-agent collaborative localization. We conduct a thorough evaluation of the performance and security of this collaborative localization system, offering valuable insights for researchers in the field of swarm robotics.

Accurate and real-time location tracking is vital for various applications in public safety and the military, particularly in search and rescue missions. Traditional filtering localization algorithms are more effective in linear environments and require precise initial estimates and system noise for optimal results. In complex and unreliable environments, these algorithms often yield poor localization results. To address these issues, this paper proposes a multi-agent collaborative localization algorithm based on reinforcement learning compensation filtering to tackle localization problems in complex environments and improve the robustness and accuracy of the localization algorithm. Specifically, this paper introduces a value decomposition-based reinforcement learning network for filtering compensation to reduce overall localization error and address the credit allocation problem in multi-agent reinforcement learning. This approach reduces the system’s positioning errors and addresses credit allocation issues common in multi-agent reinforcement learning.

In unknown emergency environments, agents use range sensors for localization and mapping, making optimal decisions among numerous uncertain candidate target locations. Exploration graphs can significantly reduce state space dimensionality. By employing deep reinforcement learning models with graph neural networks to predict actions, the agents can autonomously explore unknown environments with varying scales of landmarks. However, the opacity of strategy learning and prediction complicates understanding temporal and spatial correlations of state transitions and state-action relationships. To address this, we propose a multidimensional visualization framework to analyze high-dimensional state-action vector correlations. A post-hoc explanation tool based on feature attribution was designed to create importance sub-graphs, describing causal effects among nodes and explaining the agent’s decision-making. The framework’s effectiveness was validated using correlation analysis, node uncertainty entropy reduction, and navigation performance metrics.

Material data forms the foundation of the Industrial Internet of Things (IIoT). The rapid advancement of big data technology has opened up new opportunities for material research and development, ushering in the era of data-driven paradigms. As the cornerstone for material genetic engineering technology, the material big data platform is expanding its data scale and facing an increasing demand for sharing in light of the continuous progress and widespread application of big data technology. However, this development also poses security challenges, including the risks of data leakage and tampering. To address these challenges, this article focuses on the National Materials Genetic Engineering Discrete Data Exchange Platform (MGED). It leverages blockchain technology to design a secure material big-data storage and sharing architecture, S-MBDA, ensuring the security and reliability of the material’s big data platform. Additionally, a verifiable retrieval scheme based on a two-layer index structure of bitmap and MPT tree is proposed to enhance the efficiency of blockchain-based retrieval. This scheme aims to guarantee the integrity of retrieval data while achieving efficient and accurate searches across heterogeneous data sources. Through integrating blockchain technology and adopting a novel retrieval scheme, the article presents a comprehensive approach to secure material data storage, sharing, and retrieval. The proposed architecture and scheme address the critical security concerns associated with material big data platforms and contribute to the efficient and accurate retrieval of heterogeneous data.

Collaborative target tracking is an essential task in positioning systems, particularly in environments characterized by high dynamics, multi-source heterogeneous data, and interactive multi-agent scenarios. The challenge in such networks lies in the direct utilization of multi-source heterogeneous data as feature input for models. Additionally, the presence of high-dynamic time series data complicates the extraction of dependencies by the models. To address these issues, we introduce a novel approach that integrates a factor graph-based data fusion method with a graph neural network. This combination is designed to uncover potential dependencies between time series data and positional information within dynamic networks. Furthermore, we employ a self-attention mechanism, enabling distance-agnostic autonomous selection of complex network features. This innovation allows the model to achieve enhanced accuracy performance while simultaneously reducing computational costs. We validated our approach through simulation experiments. The results demonstrated the method’s effectiveness in fusing and selecting multi-source heterogeneous information within collaborative networks. It also excelled in identifying potential relationships between feature information and positional data, showcasing the robustness and applicability of our proposed solution in challenging collaborative target tracking environments.

In modern navigation and positioning systems, accurate location information is crucial for ensuring system performance and user experience. Particularly, in scenarios involving the use of multiple agents such as robots and drones for rescue operations in unknown complex environments, accurate localization is fundamental for subsequent actions. However, traditional filtering-based localization algorithms may exhibit suboptimal performance and are sensitive to initial estimates and system noise. To address these issues, this paper proposes a multi-agent collaborative localization algorithm based on reinforcement learning compensation filtering to tackle localization problems in complex environments and improve the robustness and accuracy. Specifically, this paper introduces a value decomposition-based reinforcement learning network for filtering compensation to reduce overall localization error and address the credit allocation problem in multi-agent reinforcement learning. The main contributions of this paper are as follows: Firstly, a local localization estimation method based on reinforcement learning compensation Extended Kalman Filter (EKF) is proposed, which further corrects the results of the EKF algorithm and eliminates initial estimation errors. Secondly, a global collaborative localization estimation algorithm (MARL_CF) based on credit allocation in multi-agent reinforcement learning is proposed, which maximizes the reduction of overall localization error through information sharing and global optimization. Finally, the effectiveness of the proposed algorithms is validated through both numerical simulation and physical experiments. The results demonstrate that the proposed MARL_CF significantly improve the accuracy and robustness of localization in complex environments.

With the advent of the era of data-driven material R&D, more and more countries have begun to build material Big Data sharing platforms to support the design and R&D of new materials. In the application process of material Big Data sharing platforms, storage and retrieval are the basis of resource mining and analysis. However, achieving efficient storage and recovery is not accessible due to the multimodality, isomerization, discrete and other characteristics of material data. At the same time, due to the lack of security mechanisms, how to ensure the integrity and reliability of the original data is also a significant problem faced by researchers. Given these issues, this paper proposes a blockchain-based secure storage and efficient retrieval scheme. Introducing the Improved Merkle Tree (MMT) structure into the block, the transaction data on the chain and the original data in the off-chain cloud are mapped through the material data template. Experimental results show that our proposed MMT structure has no significant impact on the block creation efficiency while improving the retrieval efficiency. At the same time, MMT is superior to state-of-the-art retrieval methods in terms of efficiency, especially regarding range retrieval. The method proposed in this paper is more suitable for the application needs of the material Big Data sharing platform, and the retrieval efficiency has also been significantly improved.

Real-time high-accuracy localization has a wide range of applications in scenarios, such as pedestrian navigation, emergency rescue, and vehicle networks. In these conditions, the measurement models are often nonlinear, and traditional Kalman and particle filters cannot provide long-time high-precision location-based services. To this end, we propose a Gaussian condensation filter (GCF) algorithm that can achieve high-accuracy localization in a harsh environment. However, aiming at the degradation of sampling points in target tracking based on the GCF, this article proposes a GCF algorithm based on particle flow which transfers the sample points satisfying the prior distribution of the target state to the posterior distribution, thereby improving the practical accuracy of the target-tracking algorithm. Further, to enhance the information fusion in the cooperative network, we propose a multitarget cooperative tracking algorithm to accomplish spatially constrained timing filtering of state information for improving the error correction of the target nodes on timing estimation. Numerical simulations are conducted to determine the effectiveness of our proposed algorithms. Compared with the GCF, its positioning accuracy is improved to 44.6%. Compared with the Gaussian condensation algorithm based on particle flow (PF), the practical accuracy of the GCF algorithm based on cooperative constrained PF in multitarget tracking is improved to 58.1%.

Typically, anchor-free localization is considered as a nonlinear programming problem in the existing literatures. However, the nonlinear programming algorithms can only achieve constrained optimization and the localization accuracy of such algorithms depends on the precision of initial coordinators, which are the inputs of the algorithm and usually obtained based on GPS. Due to this defect, the algorithm is invalid in GPS-denied area, such as indoor area, dense urban area and forest. In our research, we combined nonlinear programming algorithm with penalty function to solve this problem. Our simulation results show that the localization accuracy of proposed algorithm is not affected by the precision of the initial coordinators, even when the initial coordinators is set randomly. Performance comparisons are also presented to show the improvement of this algorithm.