We present Friendly Face - a virtual agent designed to reduce public speaking anxiety by standing within an audience. The agent senses the speaker's behavior during an oral presentation and provides emotional and instrumental support. The system unobtrusively tracks the motion, speech, and prosody of the presenter and provides an intuitive interface to give supportive feedback whenever the presenter looks at the agent, attentive listening behavior through agent gaze and backchannel listening behavior, and time and topic cueing based on real-time analysis of speech content compared to presentation slide contents. An evaluation of Friendly Face agent with a functionally equivalent control system demonstrated that the agent system led to significant reductions in public speaking anxiety compared to a control condition, assessed both objectively with physiological measures and validated self-report instruments.

Prior studies of public speaking behavior have demonstrated that public speaking anxiety monotonically decreases with the number of co-presenters giving an oral presentation and increases with the size of the audience. However, speaker "hand off" behavior---the verbal and nonverbal cues used to transition from one speaker to another---and its effect on speaker anxiety and presentation quality has not been systematically studied. In this work we report on two empirical studies of speaker hand-off behavior used during human co-presentations. We find that the cues used for hand-offs during prepared and rehearsed presentations differ significantly from the cues observed in face-to-face conversational turn-taking. We describe two systems that leverage automatic recognition of these verbal and nonverbal cues to drive hand-offs during co-presentations with a life-sized virtual agent.

This article reports on the development of capabilities for (on-screen) virtual agents and robots to support isolated older adults in their homes. A real-time architecture was developed to use a virtual agent or a robot interchangeably to interact via dialog and gesture with a human user. Users could interact with either agent on 12 different activities, some of which included on-screen games, and forms to complete. The article reports on a pre-study that guided the choice of interaction activities. A month-long study with 44 adults between the ages of 55 and 91 assessed differences in the use of the robot and virtual agent.

The quality of most professional oral presentations is often poor, owing to a number of factors, including public speaking anxiety. We present DynamicDuo, a system that uses an automated, life-sized, animated agent to help inexperienced speakers deliver their presentations in front of an audience. The design of the system was informed by an analysis of TED talks given by two human presenters to identify the most common dual-presentation formats and transition behaviors used. In a within-subjects study (N=12) comparing co-presenting with DynamicDuo against solo-presenting with conventional presentation software, we demonstrated that our system led to significant improvements in public speaking anxiety and speaking confidence for non-native English speakers. Judges who viewed videotapes of these presentations rated those with DynamicDuo significantly higher on speech quality and overall presentation quality for all presenters.

Loneliness and social isolation are significant problems in older adult populations. We describe a conversational agent-based system designed to provide longitudinal social support to isolated older adults. Results from an exploratory pilot study indicate that when the agent proactively draws elders into interactions, it is more effective at addressing loneliness than when the agent passively relies upon elders to initiate interactions. We discuss future research opportunities for affective computing to address this important societal problem.

In this paper we describe a prototype of a serious game designed to improve hospital patients' confidence in managing their hospital stay. We present findings of an evaluation of our visual novel computer game by comparing it with traditional approaches to increasing patient empowerment and patient self-efficacy. The results indicated a significant effect of the type of intervention on improvement in empowerment, but only for participants with gaming experience or who achieved a high level of engagement in the game. This study provides evidence that games may outperform other types of health intervention under certain conditions.

We describe a series of experiments with an agent designed to model human conversational touch-capable of physically touching users in synchrony with speech and other nonverbal communicative behavior-and its use in expressing empathy to users in distress. The agent is composed of an animated human face that is displayed on a monitor affixed to the top of a human mannequin, with touch conveyed by an air bladder that squeezes a user's hand. We demonstrate that when touch is used alone, hand squeeze pressure and number of squeezes are associated with user perceptions of affect arousal conveyed by an agent, while number of squeezes and squeeze duration are associated with affect valence. We also show that, when affect-relevant cues are present in facial display, speech prosody, and touch used simultaneously by the agent, facial display dominates user perceptions of affect valence, and facial display and prosody are associated with affect arousal, while touch had little effect. Finally, we show that when touch is used in the context of an empathic, comforting interaction (but without the manipulation of affect cues in other modalities), it can lead to better perceptions of relationship with the agent, but only for users who are comfortable being touched by other people.