In this contribution, we empirically investigated the effect of small talk on the users' non-verbal behaviors and emotions when users interacted with a crowd of virtual humans (VHs) with positive behavioral dispositions. Users were tasked with collecting items in a virtual marketplace via natural speech-based dialogue with a crowd of virtual pedestrians and vendors. The users were able to engage in natural speech-based conversation in a predefined corpus of small talk content that covered various commonplace small talk topics such as conversations about the weather, general concerns, and entertainment based on similar real-life situations. For instance, the VHs with the small talk ability would ask the users some simple questions to make small talk or remind the users of their belongings. We conducted a between-subjects empirical evaluation to investigate whether the user behaviors and emotions were different between a small talk condition and a non-small talk condition, and examined gender effects of the participants. We collected objective and subjective measures of the users to analyze users' emotions and social interaction behaviors, when in conversation with VHs that either possessed small-talk capability or not, besides task or goal oriented dialogue capabilities. Our result revealed that the VHs with small talk capability could alter the emotions and non-verbal behaviors of the users. Furthermore, the non-verbal behaviors between female and male participants differed greatly in the presence or absence of small talk.