Real-time communication (RTC) applications demand consistent low latency to ensure a smooth and interactive user experience. However, wireless networks, including WiFi and cellular, although they provide satisfactory median latency, often suffer from significant tail latency due to the highly variable network bandwidth. We observe that the control loop for managing the sending rate of RTC applications becomes inflated when congestion occurs at the wireless access point (AP), leading to untimely rate adaptation in response to wireless dynamics. Existing solutions fail to quickly adapt to bandwidth fluctuations due to the inflated control loop. In this paper, we propose Zhuge, a purely wireless AP-based solution that addresses these issues by separating congestion feedback from congested queues. Our approach involves the design of a Fortune Teller, which accurately estimates the wireless latency for each packet upon its arrival at the wireless AP. To ensure scalability, we also develop a Feedback Updater that translates the estimated latency into understandable feedback messages for various end-to-end protocols, delivering them back to the senders immediately for rate adaptation. Our evaluation, based on both trace-driven simulations and real-world scenarios, demonstrates that Zhuge significantly reduces the occurrence of large tail latency and alleviates RTC performance degradation by 22% to 95%.