Conventional Augmented Reality (AR) head-mounted displays (HMDs) integrate all essential components within the wearable unit, imposing limitations on weight, heat dissipation, and battery life. Beaming Displays (BDs) address these constraints by externalizing computation and projection; however, achieving reliable low-latency six-degrees-of-freedom (6-DoF) tracking remains a challenge. We present BeamStellar, a BD system that enables low-latency 6-DoF head tracking by combining a position-sensitive detector (PSD) with a spatio-temporally modulated infrared LED array mounted on passive glasses. The system integrates tracking and projection via a co-aligned optical path and performs real-time signal processing on an FPGA to minimize tracking latency. We describe the system architecture, pose reconstruction pipeline, and evaluate the pose-tracking latency of approximately 430 µs. BeamStellar provides a replicable, lightweight AR tracking platform, establishing a new paradigm for low-latency environment-based AR systems.