In contrast to the real world, users are not able to perceive bystanders in virtual reality (VR). Bystanders may distract users and influence their cognitive load. This involves users to feel discomfort at the thought of unintentionally touching or even bumping into a physical bystander while interacting with the virtual environment. Not knowing the intentions of a bystander or whether one is present can unsettle the user. We investigate how a bystander affects a user’s cognitive load since it has a decisive impact on applications such as VR training. In a between-subjects lab study (N = 42), three conditions were compared: 1) no bystander, 2) an invisible bystander, and 3) a visible bystander (as an avatar). Over a series of iterations, the participants were asked to memorize four pairs of letters, perform a mental rotation task and then recall the pairs of letters. The results of our study demonstrate that a bystander acting as an avatar in the virtual environment increases the user’s cognitive load more than an invisible bystander. Moreover, the cognitive load of a VR user is significantly increased by a bystander. Therefore, our work suggests that either the examiner must be separated from the participant or the examiner’s influence (as a bystander) must be included in the analysis.