In this paper, an adaptive time-varying formation optimal tracking control strategy is presented for nonlinear heterogeneous multi-agent systems (MASs) with performance constraints and unknown dynamics. A prescribed-time performance function is designed to quantify the tracking error constraints, while an error transformation function is introduced to eliminate initial value constraints and address singularity issues. Notably, the settling time and initial conditions of the performance function are independent of both the initial tracking error and system parameters. To facilitate reinforcement learning (RL)-based optimal formation control, a performance index function is formulated that incorporates the transformed tracking error, control input, and an exponentially discounted term. This design effectively avoids infinite integral values and ensures minimization of the index function. Moreover, stability analysis demonstrates that the time-varying formation tracking errors converge to the desired accuracy within a prescribed time, regulated by a sliding-mode control mechanism. Finally, numerical simulations on two practical examples validate the feasibility and effectiveness of the proposed control strategy.