The traditional dry-erase whiteboard is a ubiquitous tool in the workplace, particularly in collaborative meeting spaces. Recent studies show that meeting participants commonly capture whiteboard content using integrated cameras on mobile devices such as smartphones or tablets. Yet, little is known about how people curate or use such whiteboard photographs after meetings, or how their curation practices relate to post-meeting actions. To better understand these post-meeting activities, we conducted a qualitative, interview-based study of 19 frequent whiteboard users to probe their post-meeting practices with whiteboard photos. The study identified a set of unmet design needs for the development of improved mobile-centric whiteboard capture systems. Design implications stemming from the study include the need for mobile devices to quickly capture and effortlessly transfer whiteboard photos to productivity-oriented devices and to shared-access tools, and the need to better support the extraction of whiteboard content directly into other productivity application tools.

A rise in room-sized interaction research and responsive architecture is increasing the need for collaboration between HCI researchers and architects. These two communities use language in very different ways making it difficult to maintain a common understanding of terms such as interactive architecture. As a result, there is a need for the interaction research community to become familiar with the industry-standard language used in the Architecture, Engineering, and Construction industries. To this end, we propose a tutorial to aid in the understanding of this language in the specific area of glass facades and new architecture styles. Increasingly, transparent displays and glass are used in interactions research and it is natural to expect that some of this technology will be integrated into the architecture of future buildings. This not only provides HCI researchers with a foundation for collaboration but it also reveals some of the key customer needs that concern architects and may inspire future research.

True multi-user, multimodal interaction over a digital table lets co-located people simultaneously gesture and speak commands to control an application. We explore this design space through a case study, where we implemented an application that supports the KJ creativity method as used by industrial designers. Four key design issues emerged that have a significant impact on how people would use such a multi-user multimodal system. First, parallel work is affected by the design of multimodal commands. Second, individual mode switches can be confusing to collaborators, especially if speech commands are used. Third, establishing personal and group territories can hinder particular tasks that require artefact neutrality. Finally, timing needs to be considered when designing joint multimodal commands. We also describe our model view controller architecture for true multi-user multimodal interaction.

Modern computer displays tend to be in fixed size, rigid, and rectilinear rendering them insensitive to the visual area demands of an application or the desires of the user. Foldable displays offer the ability to reshape and resize the interactive surface at our convenience and even permit us to carry a very large display surface in a small volume. In this paper, we implement four interactive foldable display designs using image projection with low-cost tracking and explore display behaviors using orientation sensitivity.

Co-located collaborators often work over physical tabletops using combinations of expressive hand gestures and verbal utterances. This paper provides the first observations of how pairs of people communicated and interacted in a multimodal digital table environment built atop existing single user applications. We contribute to the understanding of these environments in two ways. First, we saw that speech and gesture commands served double duty as both commands to the computer, and as implicit communication to others. Second, in spite of limitations imposed by the underlying single-user application, people were able to work together simultaneously, and they performed interleaving acts: the graceful mixing of inter-person speech and gesture actions as commands to the system. This work contributes to the intricate understanding of multi-user multimodal digital table interaction.

The rapid development of large interactive wall displays has been accompanied by research on methods that allow people to interact with the display at a distance. The basic method for target acquisition is by ray casting a cursor from one's pointing finger or hand position; the problem is that selection is slow and error-prone with small targets. A better method is the bubble cursor that resizes the cursor's activation area to effectively enlarge the target size. The catch is that this technique's effectiveness depends on the proximity of surrounding targets: while beneficial in sparse spaces, it is less so when targets are densely packed together. Our method is the speech-filtered bubble ray that uses speech to transform a dense target space into a sparse one. Our strategy builds on what people already do: people pointing to distant objects in a physical workspace typically disambiguate their choice through speech. For example, a person could point to a stack of books and say "the green one". Gesture indicates the approximate location for the search, and speech 'filters' unrelated books from the search. Our technique works the same way; a person specifies a property of the desired object, and only the location of objects matching that property trigger the bubble size. In a controlled evaluation, people were faster and preferred using the speech-filtered bubble ray over the standard bubble ray and ray casting approach.

Co-located collaborators often work over physical tabletops with rich geospatial information. Previous research shows that people use gestures and speech as they interact with artefacts on the table and communicate with one another. With the advent of large multi-touch surfaces, developers are now applying this knowledge to create appropriate technical innovations in digital table design. Yet they are limited by the difficulty of building a truly useful collaborative application from the ground up. In this paper, we circumvent this difficulty by: (a) building a multimodal speech and gesture engine around the Diamond Touch multi-user surface, and (b) wrapping existing, widely-used off-the-shelf single-user interactive spatial applications with a multimodal interface created from this engine. Through case studies of two quite different geospatial systems -- Google Earth and Warcraft III -- we show the new functionalities, feasibility and limitations of leveraging such single-user applications within a multi user, multimodal tabletop. This research informs the design of future multimodal tabletop applications that can exploit single-user software conveniently available in the market. We also contribute (1) a set of technical and behavioural affordances of multimodal interaction on a tabletop, and (2) lessons learnt from the limitations of single user applications.

Most commercial software applications are designed for a single user using a keyboard/mouse over an upright monitor. Our interest is exploiting these systems so they work over a digital table. Mirroring what people do when working over traditional tables, we want to allow multiple people to interact naturally with the tabletop application and with each other via rich speech and hand gestures. In previous papers, we illustrated multi-user gesture and speech interaction on a digital table for geospatial applications -- Google Earth, Warcraft III and The Sims. In this paper, we describe our underlying architecture: GSI Demo. First, GSI Demo creates a run-time wrapper around existing single user applications: it accepts and translates speech and gestures from multiple people into a single stream of keyboard and mouse inputs recognized by the application. Second, it lets people use multimodal demonstration -- instead of programming -- to quickly map their own speech and gestures to these keyboard/mouse inputs. For example, continuous gestures are trained by saying "Computer, when I do [one finger gesture], you do [mouse drag]". Similarly, discrete speech commands can be trained by saying "Computer, when I say [layer bars], you do [keyboard and mouse macro]". The end result is that end users can rapidly transform single user commercial applications into a multi-user, multimodal digital tabletop system.

Single Display Groupware (SDG) lets multiple co-located people, each with their own input device, interact simultaneously over a single communal display. While SDG is beneficial, there is risk of interference: when two people are interacting in close proximity, one person can raise an interface component (such as a menu, dialog box, or movable palette) over another person's working area, thus obscuring and hindering the other's actions. Consequently, researchers have developed special purpose interaction components to mitigate interference techniques. Yet is interference common in practice? If not, then SDG versions of conventional interface components could prove more suitable. We hypothesize that collaborators spatially separate their activities to the extent that they partition their workspace into distinct areas when working on particular tasks, thus reducing the potential for interference. We tested this hypothesis by observing co-located people performing a set of collaborative drawing exercises in an SDG workspace, where we paid particular attention to the locations of their simultaneous interactions. We saw that spatial separation and partitioning occurred consistently and naturally across all participants, rarely requiring any verbal negotiation. Particular divisions of the space varied, influenced by seating position and task semantics. These results suggest that people naturally avoid interfering with one another by spatially separating their actions. This has design implications for SDG interaction techniques, especially in how conventional widgets can be adapted to an SDG setting.