Reinforcement learning (RL) has become a promising approach for the Traffic Signal Control (TSC) problem, enabling agents (intersections) to learn control policies with dynamic traffic environments. However, existing RL-based methods typically rely solely on current traffic states and expected reward estimates, without leveraging predicted future traffic conditions for more effective decision-making. Therefore, we propose FutureLight, the first RL TSC framework that leverages future traffic data. Specifically, we design a macroscopic, signal-aware, and lane-level simulation FutureLight-RouteSys that efficiently and accurately estimates future traffic conditions. Then, the predicted results are embedded with FutureLight-Encoder into FutureLight-DQN through state augmentation, reward shaping, and hybrid value estimation, which combines simulated near-future rewards with bootstrapped near-future returns. Finally, we propose several pruning techniques to avoid redundant calculations and further improve overall training efficiency. Experimental results demonstrate that our proposed framework consistently improves traffic signal control performance, and also improves training efficiency by thirty times.