For individuals with Social Anxiety (SA), interacting with others can be a challenging experience, a concern that extends into the virtual world. While technology has made significant strides in creating more realistic virtual human agents (VHA), the interplay of gaze and interpersonal distance when interacting with VHAs is often neglected. This paper investigates the effect of dynamic and static Gaze animations in VHAs on interpersonal distance and their relation to SA. A Bayesian analysis shows that static centered and dynamic centering gaze led participants to stand closer to VHAs than static averted and dynamic averting gaze, respectively. In the static gaze conditions, this pattern was found to be reversed in SA: participants with higher SA kept larger distances for static-centered gaze than for averted gaze VHAs. These findings update theory, elucidate how nuanced interactions with VHAs must be designed, and offer renewed guidelines for pleasant VHA interaction design.