As our landscape of wearable technologies proliferates, we find more devices situated on our heads. However, many challenges hinder them from widespread adoption - from their awkward, bulky form factor (today's AR and VR goggles) to their socially stigmatized designs (Google Glass) and a lack of a well-developed head-based interaction design language. In this paper, we explore a socially acceptable, large, head-worn interactive wearable - a hat. We report results from a gesture elicitation study with 17 participants, extract a taxonomy of gestures, and define a set of design concerns for interactive hats. Through this lens, we detail the design and fabrication of three hat prototypes capable of sensing touch, head movements, and gestures, and including ambient displays of several types. Finally, we report an evaluation of our hat prototype and insights to inform the design of future hat technologies.

ThermoTouch is a new type of thermo-haptic display device. It provides a visual display with a grid of thermal pixels that can provide hot or cold haptic feedback. Unlike previous devices, our proposed design uses liquid cooling and resistive heating to output thermal feedback. We describe the hardware and software design of ThermoTouch. Technical measurements on our prototype indicate that ThermoTouch has thermal output properties comparable to Peltier elements, which have been used extensively as thermal transducers in previous works. Our measurements of ThermoTouch's per-area power consumption and its low hardware cost per thermal pixel indicate that our technology improves scalability to large-scale thermal displays over technologies used in previous systems. As an example application of ThermoTouch, we describe an editing, automatic keyframe generation and playback system for video with an additional thermo-haptic feedback channel. Lastly, we describe technical design considerations for creating large-scale thermal displays using ThermoTouch technology.

We conducted a study that compared basic video conferencing, emergent kinetic video-conferencing techniques, and face-to-face meetings. Remote and co-located participants worked together in groups of three. We show, in agreement with prior literature, the strong adverse impact of being remote on participation levels. We also show that local and remote participants perceived differently their own contributions and others'. Local participants exhibited significantly more overlapping talk with remote participants who used an embodied proxy, than with remote participants in basic-video conferencing (and at a rate similar to overlapping speech for co-located groups). We describe differences in how the technologies were used to follow conversation. Our findings indicate that while the kinetic embodied technology increased local participants' perceived presence of remote teammates, it did not enhance remote participants' own sense of telepresence. We discuss our findings in the context of theories of agency and presence, and discuss how these findings extend our understanding of the promise and limitations of embodied video-conferencing solutions.

New technologies arise in a number of ways. They may come from advances in scientific research, through new combinations of existing technologies, or by simply imagining what might be possible in the future. This video describes the evolution of Tabletop Telepresence, a system for remote collaboration through desktop videoconferencing combined with a digital desk. Tabletop Telepresence began as a collection of camera, projector, videoconferencing and user interaction technologies. Working together; artists and research scientists combined these technologies into a means of sharing paper documents between remote desktops, interacting with those documents, requesting services (such as translation), and communicating through a videoconference.

People often use more than one query when searching for information. They revisit search results to re-find information and build an understanding of their search need through iterative explorations of query formulation. These tasks are not well-supported by search interfaces and web browsers. We designed and built SearchPanel, a Chrome browser extension that supports people in their ongoing information seeking. This extension combines document and process metadata into an interactive representation of the retrieved documents that can be used for sense-making, navigation, and re-finding documents. In a real-world deployment spanning over two months, results show that SearchPanel appears to have been primarily used for complex information needs, in search sessions with long durations and high numbers of queries. When process metadata was present in the UI, searchers in explorative search sessions submitted more and longer queries and interacted more with the SERP. These results indicate that the process metadata features in SearchPanel seem to be of particular importance for exploratory search.

Exploratory search is a complex, iterative information seeking activity that involves running multiple queries and finding and examining many documents. We designed a query preview control that visualizes the distribution of newly-retrieved and re-retrieved documents prior to running the query. When evaluating the preview control with a control condition, we found effects on both people's information seeking behavior and improved retrieval performance. People spent more time formulating a query and were more likely to explore search results more deeply, retrieved a more diverse set of documents, and found more different relevant documents when using the preview.

The Active Reading Application (ARA) brings the familiar experience of writing on paper to the tablet. The application augments paper-based practices with audio, the ability to review annotations, and sharing. It is designed to make it easier to review, annotate, and comment on documents by individuals and groups. ARA incorporates several patented technologies and draws on several years of research and experimentation.

In certain applications such as radiology and imagery analysis, it is important to minimize errors. In this paper we evaluate a structured inspection method that uses eye tracking information as a feedback mechanism to the image inspector. Our two-phase method starts with a free viewing phase during which gaze data is collected. During the next phase, we either segment the image, mask previously seen areas of the image, or combine the two techniques, and repeat the search. We compare the different methods proposed for the second search phase by evaluating the inspection method using true positive and false negative rates, and subjective workload. Results show that gaze-blocked configurations reduced the subjective workload, and that gaze-blocking without segmentation showed the largest increase in true positive identifications and the largest decrease in false negative identifications of previously unseen objects.

Virtual, mobile, and mixed reality systems have diverse uses for data visualization and remote collaboration in industrial settings, especially factories. We report our experiences in designing complex mixed-reality collaboration, control, and display systems for a real-world factory, for delivering real-time factory information to multiple types of users. In collaboration with TCHO, a chocolate maker in San Francisco, our research group is building a virtual "mirror" world of a real-world chocolate factory and its processes. Sensor data is imported into the multi-user 3D environment from hundreds of sensors on the factory floor. The resulting "virtual factory" is designed for simulation, visualization, and collaboration, using a set of interlinked, real-time layers of information about the factory and its processes. We are also looking at appropriate industrial uses for mobile devices such as cell phones and tablet computers, and how they intersect with virtual worlds and mixed realities. For example, an experimental iPhone web app provides mobile laboratory monitoring and control. The mobile system is integrated with the database underlying the virtual factory world. These systems were deployed at the real-world factory and lab in 2009, and are now in beta development. Through this mashup of mobile, social, mixed and virtual technologies, we hope to create industrial systems for enhanced collaboration between physically remote people and places - for example, factories in China with managers in Japan or the US.

Embedded Media Markers, or simply EMMs, are nearly transparent iconic marks printed on paper documents that signify the existence of media associated with that part of the document. EMMs also guide users' camera operations for media retrieval. Users take a picture of an EMM-signified document patch using a cell phone, and the media associated with the EMM-signified document location is displayed on the phone. Unlike bar codes, EMMs are nearly transparent and thus do not interfere with the document appearance. Retrieval of media associated with an EMM is based on image local features of the captured EMM-signified document patch. This paper describes a technique for semi-automatically placing an EMM at a location in a document, in such a way that it encompasses sufficient identification features with minimal disturbance to the original document.

We show several aspects of a complex mixed reality system that we have built and deployed in a real-world factory setting. In our system, virtual worlds, augmented realities, and social and mobile applications are all fed from the same infrastructure. In collaboration with TCHO[1], a chocolate maker in San Francisco, we built a virtual "mirror" world of a real-world chocolate factory and its processes. Sensor data is imported into the multi-user 3D environment from hundreds of sensors on the factory floor. The resulting virtual factory is used for simulation, visualization, and collaboration, using a set of interlinked, real-time layers of information. Another part of our infrastructure is designed to support appropriate industrial uses for mobile devices such as cell phones and tablet computers. We deployed this system at the real-world factory in 2009, and it is now is daily use there. By simultaneously developing mobile, virtual, and web-based display and collaboration environments, we aimed to create an infrastructure that did not skew toward one type of application but that could serve many at once, interchangeably. Through this mixture of mobile, social, mixed and virtual technologies, we hope to create systems for enhanced collaboration in industrial settings between physically remote people and places, such as factories in China with managers in the US.

An Embedded Media Marker (EMM) is a transparent mark printed on a paper document that signifies the availability of additional media associated with that part of the document. Users take a picture of the EMM using a camera phone, and the media associated with that part of the document is displayed on the phone. Unlike bar codes, EMMs are nearly transparent and thus do not interfere with the document appearance. Retrieval of media associated with an EMM is based on image features of the document within the EMM boundary. Unlike other feature-based retrieval methods, the EMM clearly indicates to the user the existence and type of media associated with the document location. A semi-automatic authoring tool is used to place an EMM at a location in a document, in such a way that it encompasses sufficient identification features with minimal disturbance to the original document. We will demonstrate how to create an EMM-enhanced document, and how the EMM enables access to the associated media on a cell phone.

This project investigates practical uses of virtual, mobile, and mixed reality systems in industrial settings, in particular control and collaboration applications for factories. In collaboration with TCHO, a chocolate maker start-up in San Francisco, we have built virtual mirror-world representations of a real-world chocolate factory and are importing its data and modeling its processes. The system integrates mobile devices such as cell phones and tablet computers. The resulting "virtual factory" is a cross-reality environment designed for simulation, visualization, and collaboration, using a set of interlinked, real-time 3D and 2D layers of information about the factory and its processes.

One of the core challenges now facing smart rooms is supporting realistic, everyday activities. While much research has been done to push forward the frontiers of novel interaction techniques, we argue that technology geared toward widespread adoption requires a design approach that emphasizes straightforward configuration and control, as well as flexibility. We examined the work practices of users of a large, multi-purpose conference room, and designed DICE, a system to help them use the room's capabilities. We describe the design process, and report findings about the system's usability and about people's use of a multi-purpose conference room.

We describe a new interaction technique that allows users to control nonlinear video playback by directly manipulating objects seen in the video. This interaction technique is similar to video "scrubbing" where the user adjusts the playback time by moving the mouse along a slider. Our approach is superior to variable-scale scrubbing in that the user can concentrate on interesting objects and does not have to guess how long the objects will stay in view. Our method relies on a video tracking system that tracks objects in fixed cameras, maps them into 3D space, and handles hand-offs between cameras. In addition to dragging objects visible in video windows, users may also drag iconic object representations on a floor plan. In that case, the best video views are selected for the dragged objects.

DOTS (Dynamic Object Tracking System) is an indoor, real-time, multi-camera surveillance system, deployed in a real office setting. DOTS combines video analysis and user interface components to enable security personnel to effectively monitor views of interest and to perform tasks such as tracking a person. The video analysis component performs feature-level foreground segmentation with reliable results even under complex conditions. It incorporates an efficient greedy-search approach for tracking multiple people through occlusion and combines results from individual cameras into multi-camera trajectories. The user interface draws the users. attention to important events that are indexed for easy reference at a later time. Different views within the user interface provide spatial information for easier navigation. Our system, with over twenty video cameras installed in hallways and other public spaces in our office building, has been in constant use for almost a year.

Post-Bit is a small e-paper device modeled after paper Post-Its®1. We explored and designed interfaces to handle multi-media contents with paper-like manipulations using this e-paper device. The functions of each Post-Bit combined the affordance of physical tiny sticky memos and digital handling of information. At this stage of the design, we have prototyped two features of the interface: connecting computer-based workspaces and physical workspaces (using a function called Drop-Beyond-Drag), and tangible and tactile operation of multi-media contents. In this paper, we present the integrated design and functionality of the Post-Bit system's main components as shown in the video scenario.