City-scale traffic signal control (TSC) involves thousands of heterogeneous intersections with varying topologies, making cooperative decision-making across intersections particularly challenging. Given the prohibitive computational cost of learning individual policies for each intersection, some researchers explore learning a universal policy to control each intersection in a decentralized manner, where the key challenge is to construct a universal representation method for heterogeneous intersections. However, existing methods are limited to universally representing information of heterogeneous ego intersections, neglecting the essential representation of influence from their heterogeneous neighbors. Universally incorporating neighborhood information is nontrivial due to the intrinsic complexity of traffic flow interactions, as well as the challenge of modeling collective influences from neighbor intersections. To address these challenges, we propose CityLight, which learns a universal policy based on representations obtained with two major modules: a Neighbor Influence Encoder to explicitly model neighbor's influence with specified traffic flow relation and connectivity to the ego intersection; a Neighbor Influence Aggregator to attentively aggregate the influence of neighbors based on their mutual competitive relations. Extensive experiments on five city-scale datasets, ranging from 97 to 13,952 intersections, confirm the efficacy of CityLight, with an average throughput improvement of 11.68% and a lift of 22.59% for generalization. Our codes and datasets are released: https://github.com/tsinghua-fib-lab/CityLight.

Visual indoor exploration requires agents to explore a room in a limited time. Currently, planning-based solutions have a time-consuming inference stage and require many handcrafted parameters in different scenes. Reinforcement Learning (RL) schemes on the other hand solve these problems by automatically updating flexible policies and affording faster inference time. Spurred by the advantages of RL, we introduce Spatial Attention Visual Exploration (SAVE), which is based on Active Neural SLAM (ANS) [1]. Specifically, we propose a novel RL-based global planner named Spatial Global Policy (SGP) that utilizes spatial information to promote efficient exploration through global goal guidance. SGP has two major components: a transformer-based spatial-attention module encoding spatial interrelation between the agent and different regions to perform spatial reasoning, and a hierarchical spatial action selector to infer global goals for faster training. The map representations are aligned through our spatial adjustor. Experiments on the Habitat photo-realistic simulator [2] demonstrate that SAVE outperforms current planning-based methods and RL variants, reducing at least 10% of the processing steps, 15% of the repeat ratio, and affording an x2 to x4 faster execution time than planning-based methods.