Although we share the optimistic vision of affective computing presented in Interacting with Computers 14(2), we question the extent to which affective sensing can support the kinds of applications proposed in the literature. These applications depend upon the detection of affective reactions to HCI situations and events, but it has yet to be shown that such reactions can reliably be detected in subtle and natural situations. We also point out that, in human–human interaction, intentional commmunicative affect is both easier to recognise and more important than reactive affect. We suggest exploration of this idea may lead to more fruitful applications of affective computing.

This study is motivated by the increased need to understand human response to video-links, 3G telephony and avatars. We focus on response of participants to audiovisual presentations of talking heads, and examine the effect of noise and temporal misalignment of channels. We show that misalignment of audio and visual channels not only cause strange perception phenomena - the McGurk effect but also cause participants to apply extra mental effort, which is detectable from the physiological data collected. These data allow inferences to be drawn about the impact of the McGurk effect, thus providing indication of stress levels. This illuminates both to the mental and physical aspects of users interacting with multi-modal interfaces