This paper explores how na?ve observers recognize and interpret transitive actions (actions involving manipulation of objects) without accompanying speech, in order to derive guidelines for the design of gesture interpretation systems. Semi-structured interviews with 11 observers, interpreting 106 video clips of transitive actions elicited unstaged from 16 participants, reveal that people are generally able to interpret the transitive action as well as characteristics of the object manipulated despite individual variations in how people naturally gesture. In particular, people focus primarily on hand movement and hand shape to correctly interpret object characteristics, and on manner of movement of arms and/or final location of hands to interpret the goal of the transitive action (e.g., arrange objects vs. clear objects). These findings provide insights on aspects of gestures one can focus on to inform and guide the design of gesture interpretation models for interfaces that allow for individual variations in natural gesture production.

This paper explores how interaction with systems using touchless gestures can be made intuitive and natural. Analysis of 912 video clips of gesture production from a user study of 16 subjects communicating transitive actions (manipulation of objects with or without external tools) indicated that 1) dynamic pantomimic gestures where imagined tool/object is explicitly held are performed more intuitively and easily than gestures where a body part is used to represent the tool/object or compared to static hand poses and 2) gesturing while communicating the transitive action as how the user habitually performs the action (pantomimic action) is perceived to be easier and more natural than gesturing while communicating it as an instruction. These findings provide guidelines for the characteristics of gestures and user mental models one must consciously be concerned with when designing and implementing gesture vocabularies of touchless interaction.

This paper presents a testbed for developing multilingual phonotactic descriptions that employs finite state methods to represent the phonotactics of one or more languages. The motivation for this work is to make an extensive range of phonotactic descriptions of varying granularity available for speech technology applications. We discuss the design of the phonotactic testbed and how various modules may be used to generate finite state phonotactic descriptions. We provide an example multilingual application drawn from a partial sample of onset clusters spanning four language families, demonstrating how the commonalities of a broad spectrum of languages can be expressed using individual and generic phonotactic automata. We then discuss how these representations are extended via a three-tiered model to provide the basis for the feature- and event-based phonotactic automata.

The problems of conventional speech recognition approaches include incomplete linguistic knowledge and inability to deal with underspecification. These issues can be addressed by understanding the constraints of speech to predict speech tendencies. We believe that understanding what constraints exist requires an embodied view of speech and that the traditional disembodied view of speech is the fundamental limitation on the robustness of many speech systems. We argue that viewing speech as a form of embodied cognition, or within context of its production and use, provides important insights in speech structure and speech recognition. In making this claim, this paper briefly outlines a strongly embodied account of cognition and develops from that an embodiment paradigm for speech recognition. The embodiment paradigm proposed leads to both an explanatory and descriptive account of linguistic structure. It simplifies the view of speech structure for automatic speech recognisers, by considering only the most directly relevant motivations or constraints influencing communication and thus speech.