Leveraging large language models (LLMs) for collaborative sequential decision-making is a significant challenge, despite strong semantic understanding and extensive prior knowledge. Conversely, multi-agent reinforcement learning (MARL) can learn environment-aligned policies through interaction, but often suffers from inefficient exploration and heavy reliance on centralized global state information. To achieve complementary advantages, we propose the environment-aligned LLMs (EALLMs). In our framework, an LLM serves as a shared policy for all agents and is updated through online MARL to achieve alignment with the environment. Simultaneously, another LLM, fine-tuned with offline datasets, acts as an information integrator to generate global state for communication purposes. Additionally, we design robust, task-specific prompts tailored to multi-agent systems. Extensive experiments demonstrate that EALLMs outperform classical MARL and LLM-based baselines in both exploration efficiency and overall performance on the SMAC and SMACv2 benchmarks. Ablation studies further confirm EALLMs’ ability to achieve competitive results without relying on explicit global state, while preserving the original capabilities of the LLM during alignment.

Multi-agent systems must learn to communicate and understand interactions between agents to achieve cooperative goals in partially observed tasks. However, existing approaches lack a dynamic directed communication mechanism and rely on global states, thus diminishing the role of communication in centralized training. Thus, we propose the Transformer-based graph coarsening network (TGCNet), a novel multi-agent reinforcement learning (MARL) algorithm. TGCNet learns the topological structure of a dynamic directed graph to represent the communication policy and integrates graph coarsening networks to approximate the representation of global state during training. It also utilizes the Transformer decoder for feature extraction during execution. Experiments on multiple cooperative MARL benchmarks demonstrate state-of-the-art performance compared to popular MARL algorithms. Further ablation studies validate the effectiveness of our dynamic directed graph communication mechanism and graph coarsening networks.