Agent behavior modeling in 3D virtual environments is a critical challenge in artificial intelligence and military simulation. While rule-based methods (e.g., finite state machines) are widely used, their limitations in adaptability and development efficiency hinder their application in dynamic combat scenarios. To address this, a memory-driven self-learning agent (MDSLA) architecture is proposed, integrating visual, auditory, and game features to simulate human-like battlefield decision-making. The architecture employs an asynchronous advantage actor-critic (A3C) framework to enhance training efficiency and incorporates a memory module for processing historical perception data. Experimental validation in the Vizdoom environment demonstrates that MDSLA outperforms traditional rule-based methods and mainstream reinforcement learning algorithms in convergence speed and combat effectiveness. Furthermore, a parallel simulation mechanism is implemented via high-speed middleware, enabling seamless deployment of the model on both Vizdoom and a high-precision simulation platform (HPSP). Results from HPSP experiments show a 33% reduction in task execution time and a 24.1% improvement in lethality compared to finite state machine-driven agents. This work provides a scalable framework for developing intelligent combat agents with enhanced adaptability and realism in 3D virtual environments.

Large-scale spatial data can be collected using mobile robots with sensing and navigation capabilities. Due to limited battery lifetime and scarcity of charging stations, it is important to plan informative paths so as to maximize the utility of data given a limited travel budget, which is known as the informative path planning (IPP) problem. IPP is NP-hard, and existing solutions suffer from high complexity or low optimality. In this paper, we present a novel IPP solution based on reinforcement learning (RL). The basic idea is to learn the structural characteristics of informative paths, so informative paths can be predicted. As such, when budgets change, we avoid solving the problem from scratch and thus path planning efficiency can be improved dramatically. Among the 20 path planning experiments in two areas, the proposed RL based solution achieves the best path utility in 15 experiments, compared with state-of-the-art algorithms. More importantly, the inference complexity is linear with respect to the budget (equivalently, the maximum number of steps in RL), which is lower than other solutions. Despite the NP-hardness, the path planning process can be finished within a few seconds in our experiments on two graphs of different sizes.

Mobile sensing is an effective way to collect environmental data such as air quality, humidity and temperature at low costs. However, mobile robots are typically battery powered and have limited travel distances. To accelerate data collection in large geographical areas, it is beneficial to deploy multiple robots to perform tasks in parallel. In this paper, we investigate the Multi-Robot Informative Path Planning (MIPP) problem, namely, to plan the most informative paths in a target area subject to the budget constraints of multiple robots. We develop two deep reinforcement learning (RL) based cooperative strategies: independent learning through credit assignment and sequential rollout based learning for MIPP. Both strategies are highly scalable with the number of robots. Extensive experiments are conducted to evaluate the performance of the proposed and baseline approaches using real-world WiFi Received Signal Strength (RSS) data. In most cases, the RL based solutions achieve superior or similar performance as a baseline genetic algorithm (GA)-based solution but at only a fraction of running time during inference. Furthermore, when the budgets and initial positions of the robots change, the pre-trained policies can be applied directly.

Large-scale spatial data such as air quality, thermal conditions and location signatures play a vital role in a variety of applications. Collecting such data manually can be tedious and labour intensive. With the advancement of robotic technologies, it is feasible to automate such tasks using mobile robots with sensing and navigation capabilities. However, due to limited battery lifetime and scarcity of charging stations, it is important to plan paths for the robots that maximize the utility of data collection, also known as the informative path planning (IPP) problem. In this paper, we propose a novel IPP algorithm using reinforcement learning (RL). A constrained exploration and exploitation strategy is designed to address the unique challenges of IPP, and is shown to have fast convergence and better optimality than a classical reinforcement learning approach. Extensive experiments using real-world measurement data demonstrate that the proposed algorithm outperforms state-of-the-art algorithms in most test cases. Interestingly, unlike existing solutions that have to be re-executed when any input parameter changes, our RL-based solution allows a degree of transferability across different problem instances.