Gaze is a powerful measure of people's attracted attention and reveals where we are looking within our current FOV. Hence gaze-based interfaces are gaining in importance. However, gaze estimation usually requires extensive hardware and depends on a calibration that has to be renewed regularly. We present EyeMirror, a mobile device for calibration-free gaze approximation on surfaces (e.g., displays). It consists of a head-mounted camera, connected to a wearable mini-computer, capturing the environment reflected on the human cornea. The corneal images are analyzed using natural feature tracking for gaze estimation on surfaces. In two lab studies we compared variations of EyeMirror against established methods for gaze estimation in a display scenario, and investigated the effect of display content (i.e. number of features). EyeMirror achieved 4.03° gaze estimation error, while we found no significant effect of display content.

Mobile gaze-based interaction with multiple displays may occur from arbitrary positions and orientations. However, maintaining high gaze estimation accuracy in such situa-tions remains a significant challenge. In this paper, we present GazeProjector, a system that combines (1) natural feature tracking on displays to determine the mobile eye tracker's position relative to a display with (2) accurate point-of-gaze estimation. GazeProjector allows for seam-less gaze estimation and interaction on multiple displays of arbitrary sizes independently of the user's position and orientation to the display. In a user study with 12 partici-pants we compare GazeProjector to established methods (here: visual on-screen markers and a state-of-the-art video-based motion capture system). We show that our approach is robust to varying head poses, orientations, and distances to the display, while still providing high gaze estimation accuracy across multiple displays without re-calibration for each variation. Our system represents an important step towards the vision of pervasive gaze-based interfaces.

Mobile phones have evolved significantly in recent years from single-purpose communication devices to multi-purpose computing devices. Despite this evolution, the interaction model for how incoming calls are handled has barely changed. Current-generation smartphones still use abrupt full-screen notifications to alert users to incoming calls, demanding a decision to either accept or decline the call. These full-screen notifications forcibly interrupt whatever activity the user was already engaged in. This might be undesirable when the user's primary task was more important than the incoming call. This paper explores the design space for how smartphones can alert users to incoming calls. We consider designs that allow users to postpone calls and also to multiplex by way of a smaller partial-screen notification. These design alternatives were evaluated in both a small-scale controlled lab study as well as a large-scale naturalistic in-the-wild study. Results show that a multiplex design solution works best because it allows people to continue working on their primary task while being made aware that there is a caller on the line. The contribution of this work is an enhanced interaction design for handling phone calls, and an understanding of how people use it for handling incoming calls.

With the recent increase in sales figures of tablet computers, a corresponding boost of mobile media consumption on such devices can be observed. Even though tablets provide a comparably larger screen in contrast to smartphones - which are taking the main share of mobile media consumption - the experience of consuming media content on tablets is still limited. For movie theatres the notion of 4 dimensional (4D) films exist, describing a 3D movie that is enhanced with ambient feedback such as environmental light, air streams or moist. In this paper we present ambiPad, a tablet prototype that enriches mobile digital media with ambient feedback around the tablet's display. Besides a light emitting frame, ambiPad allows for thermal stimuli as well. We report on the results of a qualitative user study, in which the feedback channels of ambiPad were rated as highly attractive and desirable by the participants.

Digital technologies are rapidly finding their way into urban spaces. One prominent example is media façades. Due to their size, visibility and their technical capabilities, they offer great potential for interaction and for becoming the future displays of public spaces. To explore their potential, researchers have recently started to develop interactive applications for various media façades. Existing development tools are mostly tailored to one specific media façade in one specific setting. They usually provide limited means to incorporate interaction by a user, and the applications developed are limited to running on only one particular media façade. In this paper, we present a flexible, generalized media façade toolkit, which is capable of mimicking arbitrary media façade installations. The toolkit is capable of running interactive applications on media façades with different form factors, sizes and technical capabilities. Furthermore, it ensures application portability between different media façades and offers the possibility of providing interactivity by enabling user input with different modalities and different interaction devices.

While mobile phones made a significant evolution in recent years from single-purpose communication devices to multi-purpose devices, the fundamental design of phone call applications did not evolve accordingly. While its implementation leveraged from new hardware and software capabilities, the fundamental decisions people are able to make when they receive a call did not change. Currently, when a call comes in, a modal dialog opens where the callee can either decline or accept the call. A recent study found that the current user interfaces of phone call applications (phone UIs) often lead to an increased overhead when application usage is being interrupted by phone calls [6]. In this paper, we revisit phone call UIs for multipurpose smartphones. We contribute a new design space for mobile phone call UIs, going beyond the simple accept-or-decline dilemma. We present a prototype implementation and discuss open challenges.

With the recent trend of increasing display sizes of mobile devices, one-handed interaction has become increasingly difficult when a user wants to maintain a safe grip around the device at the same time. In this paper we evaluate how a combination of hybrid front- and back-of-device touch input can be used to overcome the difficulties when using a mobile device with one hand. Our evaluation shows that, even though such a technique is slower than conventional front-of-device input, it allows for accurate and safe input.

Media façades are one prominent example of how new technologies currently augment urban spaces. At the same time, they offer new, ubiquitous opportunities for novel applications. To achieve a usable and enjoyable outcome, however, designing interaction with media façades demands a structured design process. In this paper, we present our experiences designing iRiS, a system for remote interaction with media façades. We approached the development following a user-centered design approach and addressing the process at two points with additional means: (1) using a purpose-built prototyping toolkit testing and exploring both, content and hardware before deploying the system on the actual façade and (2) experimental use and adaptation of user experience (UX) evaluation methods to investigate the users actions and emotions more holistically in this context.

The increasing number of media façades is a prominent example of the digital augmentation of urban spaces. Many media façades cover most of the outer shell of a building and come with a 3D form factor. They offer great potential for remote interaction in which the interactive area goes beyond the parts of the façade that are visible from the user's current perspective. Common interaction techniques often focus on a fixed part of the media façade. This restricts exploiting the full capabilities and the potential of such gigantic screens. In this paper we describe how to apply cartographic map projections to create 2D map representations of media façades to address this problem. We describe how a continuous interaction with the media façade is possible, independent of the form factor. We analyze existing media façades and provide a set of guidelines for how to create façade maps for different form factors.

Media façades in urban spaces offer great potential for new forms of collaborative multi-user interaction. Beyond interaction, they also allow for connecting people and for triggering social interaction between the different users. We report on our experiences of how simultaneous interaction with a media façade at-a-distance can engage social offline interaction between users. We built an application that allows for simultaneous painting on a façade, and gathered informal feedback during the ARS Electronica Festival in Linz, Austria.

During sales conversations, gestures and mimics are of high importance to communicate information about a product. One prominent example for such sales gestures is the meat and cheese counter, which is one of the remaining spots in supermarkets where sales persons interact with customers. Interactions at such counters in supermarkets normally follow a simple protocol. The customer points at an item of choice. The employee takes out the item and, in most of the cases the product needs to be cut to fit the amount the customer wants to buy. Often it is ambiguous about what specific product the customer and the employees are talking about. Up to now, there are just a few efforts in HCI research to enrich communication at the point of sale. In this paper we report and analyze one scenario in which an intelligent natural user interface can support communication between customer and employee in a sales conversation. Furthermore, we report on our prototype that is able to track pointing gestures by using a depth camera and to display information about items pointed at.

Smartphone users might be interrupted while interacting with an application, either by intended or unintended circumstances. In this paper, we report on a large-scale observational study that investigated mobile application interruptions in two scenarios: (1) intended back and forth switching between applications and (2) unintended interruptions caused by incoming phone calls. Our findings reveal that these interruptions rarely happen (at most 10% of the daily application usage), but when they do, they may introduce a significant overhead (can delay completion of a task by up to 4 times). We conclude with a discussion of the results, their limitations, and a series of implications for the design of mobile phones.

Multi-touch interaction offers opportunities to interact with complex data. Especially the exploration of geographical data, which until today mostly relies on mice and keyboard input, could benefit from this interaction paradigm. However, the gestures that are required to interact with complex systems like Geographic Information Systems (GIS) increase in difficulty with every additional functionality. This paper describes a novel interaction approach that allows non-expert users to easily explore geographic data using a combination of multi-touch gestures and handpostures. The use of the additional input modality -- handpose -- is supposed to avoid more complex multi-touch gestures. Furthermore the screen of a wearable device serves as another output modality that on one hand avoids occlusion and on the other hand serves as a magic lens.

The increasing number of media facades in urban spaces offers great potential for new forms of interaction especially for collaborative multi-user scenarios. In this paper, we present a way to directly interact with them through live video on mobile devices. We extend the Touch Projector interface to accommodate multiple users by showing individual content on the mobile display that would otherwise clutter the facade's canvas or distract other users. To demonstrate our concept, we built two collaborative multi-user applications: (1) painting on the facade and (2) solving a 15-puzzle. We gathered informal feedback during the ARS Electronica Festival in Linz, Austria and found that our interaction technique is (1) considered easy-to-learn, but (2) may leave users unaware of the actions of others.

The electronic market has rapidly grown in the last few years. However, despite this success, consumers still enjoy visiting a store with real products. Therefore various common technologies have been installed in supermarkets to support the customer's shopping process and experience. In this paper, we introduce the IRL SmartCart - an instrumented shopping cart that acts as a user interface to support the shopping process. We show how RFID technology enables recognizing products that are put in the cart's basket. We are also able to determine the cart's position in an instrumented shopping environment. User input and visual output are possible by means of a touch screen, which is fitted in the IRL SmartCart's handle to support different tasks involved in the shopping process. We present and discuss different location- and context-based services that run on the cart interface system, e.g. a personalized shopping list sorted corresponding to the products in the user's vicinity or a navigation service to different products that the customer is searching for.

In this paper we investigate interaction with a media façade that is out of reach for touch-based interaction. We describe four different applications that utilize mobile devices to enable passers-by to interact with the façade. Each application has been designed constrained by limitations given by formal regulations of an editorial board (e.g. to prevent traffic distractions) and with the aim to catch the attention of passers-by to achieve interaction and keep the users engaged. Besides the description of the design and implementation of the different application, we report on initial feedback of users after a first preliminary user test that informs further development and design.

The workshop on mobile interaction in retail environments (MIRE) brings together researchers and practitioners from academy and industry to explore how mobile phones and mobile interaction can be embedded in retail environments to create new shopping experiences and mobile enhanced services.