Understanding visual attention in users during stressful situations with emotional crowds is crucial for the improvement of emergency response, safety, and efficiency. To explore this, we conducted a study examining how the organizational structure and reaction behavior of a virtual emotional crowd affected subjects' visual attention. Participants completed a task in a virtual reality simulation featuring a fire threat. The experiment employed two between-subjects factors (Crowd Type and Reaction Type) and one within-subjects factor (Simulation Phase). The simulation consisted of a normal phase, where participants interacted with virtual human (VHs) pedestrians and vendors in a virtual marketplace, followed by a stressful phase featuring a fire threat within the same setting. The Crowd Type varied between individuals walking alone (Individual Condition) and small groups of 3-4 members (Group Condition). The Reaction Type determined whether the virtual crowd ignored the threat or moved faster to escape. We analyzed subjects' fixation gaze across conditions, revealing that visual attention behavior was significantly influenced by Crowd Type, Reaction Type, and Simulation Phase.

Evidence from the behavioral psychology literature suggests that in unexpected stressful situations, crowd behavior can have a direct impact on people’s emotions and behaviors. This paper investigates whether this effect is elicited in immersive virtual reality by a crowd of virtual humans (VHs) in an external stressful situation induced by a fire. We conducted a mixed-design study in an experimental scenario where users were asked to collect 10 items from a virtual market. The between-subjects factors were reaction type and agent type, while the within-subjects factor was the simulation phase, which included the normal and stressful situations. There were two reaction types where the VHs showed either active (agitated) or passive (calm) behavior towards the external threat situation. For the agent type, the VHs exhibited either individualistic or group (3-4 member group) behaviors. In a mixed factorial 2(crowd type)x 2(reaction type) x 2(simulation phase) design, we examined the influences of these factors on participants’ emotional and behavioral responses. The results showed that the participants’ task performance and social interaction were affected by the simulated stressor. Negative emotions were elicited when they encountered active VHs and small groups of VHs. Furthermore, a greater change in their interpersonal closeness was experienced when interacting with small groups of VHs, than when interacting with individual VHs.

This study proposes a voice conversion-based approach to personalized text-to-speech (TTS) synthesis. The conversion functions, trained using a small parallel corpus with source and target speech data, can impose the voice characteristics of a target speaker on an existing synthesizer. Frame alignment between a pair of sentences in the parallel corpus is generally used for training voice conversion functions. However, with incorrect alignment, the resultant conversion functions may generate unacceptable conversion results. Traditional frame alignment using minimal spectral distance between the frame-based feature vectors of the source and the target phone sequences can be imprecise because the voice properties of the source and target phones inherently differ. In the proposed method, feature vectors of the parallel corpus are transformed into codewords in an eigenspace. A more precise frame alignment can be obtained by integrating the codeword occurrence distributions into distance estimation. In addition to the spectral property, a prosodic word/phrase boundary prediction model was constructed using an adaptive conditional random field (CRF) to generate personalized prosodic information. Objective and subjective tests were conducted to evaluate the performance of the proposed approach. The experimental results showed that the proposed voice conversion method, based on distribution-based alignment and prosodic word boundary detection, can improve the speech quality and speaker similarity of the converted speech. Compared to other methods, the evaluation results verified the improved performance of the proposed method.