On the skin, a small distance between two actuators can result in the perception of a single, centralized stimulus rather than two distinct stimuli – this phenomenon is known as the funneling illusion. In this work, we explore the electrotactile funneling illusion on the forearm in a user study with 16 participants. We placed an electrode strip with 9 electrode pairs along their forearm – from wrist to elbow. The calibration of the same perceived intensity of each electrode pair for each participant shows that the calibrated intensities near the wrist are significantly higher compared to the intensities calibrated near the elbow. A linear regression corresponds to this behavior as well as the qualitative feedback of our participants. Based on this, we created an equation that helps to reduce the calibration time considerably. The results of our study show that the funneling illusion can be reliably evoked at distances of up to 7.2 cm. Further, we provide detailed information on occurrence frequency and precision and explored whether an approach adapted for electrotactile feedback can create an apparent tactile motion.

Operating small touchscreens with the finger occludes a large part of the screen. We propose using the watch case as the input space, without enlarging the smartwatch. Therefore, we created two prototypes, one with a touch surface on the watch case (CASE) and one with touch surfaces on the watch case and the wristband (CASE+BAND). In a comparative study, we analyze their suitability in a 1D list scrolling task and 2D map navigation task. The results show that occlusion is less of a problem for the list scrolling task, as visibility is sufficient. In the map navigation task, participants reached task completion times with CASE+BAND that are comparable to touch input. CASE was significantly slower, but only requires minimal additional hardware. However, the results of a subsequent longitudinal study demonstrates the learnability of CASE, which led to task completion times comparable to touch input, and provides insights in the gradual development of expert performance.

Cycling navigation is a complex and stressful task as the cyclist needs to focus simultaneously on the navigation, the road, and other road users. We propose directional electrotactile feedback at the wrist to reduce the auditory and visual load during navigation-aided cycling. We designed a custom electrotactile grid with 9 electrodes that is clipped under a smartwatch. In a preliminary study we identified suitable calibration settings and gained first insights about a suitable electrode layout. In a subsequent laboratory study we showed that a direction can be encoded with a mean error of 19.28\,° (σ = 42.77°) by combining 2 adjacent electrodes. Additionally, by interpolating with 3 electrodes a direction can be conveyed with a similar mean error of 22.54° (σ = 43.57°). We evaluated our concept of directional electrotactile feedback for cyclists in an outdoor study, in which 98.8% of all junctions were taken correctly by eight study participants. Only one participant deviated substantially from the optimal path, but was successfully navigated back to the original route by our system.

Providing rich feedback on small devices, like smartwatches, can be difficult. We propose colorful electrotactile feedback on the back of a smartwatch. Colorful electrotactile feedback provides private notifications, is energy efficient, and can express various sensations in different qualities. In a first study, 13 participants explored 49 different combinations of frequency and pulse width regarding the perceived “colorfulness” of electrotactile feedback. We investigated what sensations can be expressed with electrotactile feedback and which qualities of these sensations are conveyed. To describe the sensations, participants chose the best fitting terms from a list of 21 terms. The three most frequently selected terms were prickling (177), vibrating (163), and irritating (112). The three least frequently selected ones were twitching (31), tickling (29), and itching (28). In a second study with 17 participants we evaluated a reduced set of 9 sensations that we selected and refined based on the results of study 1. We evaluated these sensations regarding recognition rates and achieved recognition rates of up to 84% without prior learning. Furthermore, we investigated the acceptance of colorful electrotactile feedback and present a method for an easier and faster calibration of electrotactile feedback.

A common problem of touch-based smartwatch interaction is the occlusion of the display. Although some models provide solutions like the Apple "digital crown" or the Samsung rotatable bezel, these are limited to only one degree of freedom (DOF). Performing complex tasks like navigating on a map is still problematic as the additional input option helps to zoom, but touching the screen to pan the map is still required. In this work, we propose using a trackball as an additional input device that adds two DOFs to prevent the occlusion of the screen. We created several prototypes to find a suitable placement and evaluated them in a typical map navigation scenario. Our results show that the participants were significantly faster (15.7%) with one of the trackball setups compared to touch input. The results also show that the idle times are significantly higher with touch input than with all trackball prototypes, presumably because users have to reorient themselves after panning with finger occlusion.

Previous literature has reported that users consider hands-free and eyes-free interaction as one of the prime features of IPAs (Intelligent Personal Assistants). Hands-free and eyes-free interaction enables dual tasking. Although users prefer dual tasking with IPAs, it is unknown to what degree current IPAs are compatible with dual tasking. To determine IPA efficiency while dual tasking, we investigate cognitive load in dual-task scenarios with IPAs. In our experiment, we selected a rhythm game as the primary task and everyday IPA requests as secondary tasks. The secondary tasks belonged to four common categories: information search, multimedia control, smart home control, and turn-taking conversations. The findings show that IPAs need significant improvement to support dual tasking. Out of the four categories, only tasks in the smart home and multimedia categories were appropriate for dual tasking, whereas turn-taking conversation and information search had a high cognitive load. Task completion time was significantly different between tasks, but the penalty on the accuracy of the primary task was small. In interviews we found that, due to information abundance in IPA responses and high time pressure during task completion, users tended to make several mistakes. Based on our findings and observations we derive four design recommendations that facilitate dual-tasking while using IPAs.

Tactile patterns are a means to convey navigation instructions to pedestrians and are especially helpful for people with visual impairments. This article presents a concept to provide precise micro-navigation instructions through a tactile around-the-head display. Our system presents four tactile patterns for fundamental navigation instructions in conjunction with continuous directional guidance. We followed an iterative, user-centric approach to design the patterns for the fundamental navigation instructions, combined them with a continuous directional guidance stimulus, and tested our system with 13 sighted (blindfolded) and 2 blind participants in an obstacle course, including stairs. We optimized the patterns and validated the final prototype with another five blind participants in a follow-up study. The system steered our participants successfully with a 5.7 cm average absolute deviation from the optimal path. Our guidance is only a little less precise than the usual shoulder wobbling during normal walking and an order of magnitude more precise than previous tactile navigation systems. Our system allows various new use cases of micro-navigation for people with visual impairments, e.g., preventing collisions on a sidewalk or as an anti-veering tool. It also has applications in other areas, such as personnel working in low-vision environments (e.g., firefighters).

The vibrotactile funneling illusion is the sensation of a single (non-existing) stimulus somewhere in-between the actual stimulus locations. Its occurrence depends upon body location, distance between the actuators, signal synchronization, and intensity. Related work has shown that the funneling illusion may occur on the forehead. We were able to reproduce these findings and explored five further regions to get a more complete picture of the occurrence of the funneling illusion on the head. The results of our study (24 participants) show that the actuator distance, for which the funneling illusion occurs, strongly depends upon the head region. Moreover, we evaluated the centralizing bias (smaller perceived than actual actuator distances) for different head regions, which also showed widely varying characteristics. We computed a detailed heat map of vibrotactile localization accuracies on the head. The results inform the design of future tactile head-mounted displays that aim to support the funneling illusion.

Public bookcases offer the opportunity to serendipitously discover books and to anonymously share books with others. The set of available books as well as the sharing patterns are highly dynamic, as anybody can freely take or donate books. This makes it difficult for users to see what is available or of interest to them. To support book sharing via public bookcases we developed a mobile AR application that highlights relevant books in the camera viewfinder and that facilitates searching for specific books. The application recognizes books via text and color features on the spine. In a lab study with 15 participants we evaluated our book recognition algorithm and found that it outperforms unaided visual search. We interviewed users of public bookcases and analyzed the bookcases’ setup and rate of change. A subsequent field evaluation of the AR application on nine public bookcases found a recognition accuracy of 80 % for 450 books under different conditions. The proposed approach provides the basis for effectively sharing books via public bookcases.

We propose around-the-head spatial vibrotactile patterns for representing different kinds of notifications. The patterns are defined in terms of stimulus location, intensity profile, rhythm, and roughness modulation. A first study evaluates recall and distinguishability of 30 patterns, as well as agreement on meaning without a predetermined context: Agreement is low, yet the recognition rate is surprisingly high. We identify which kinds of patterns users recognize well and which ones they prefer. Static stimulus location patterns have a higher recognition rate than dynamic patterns, which move across the head as they play. Participants preferred dynamic patterns for comfort. A second study shows that participants are able to distinguish substantially more around-the-head spatial patterns than smartphone-based patterns. Spatial location has the highest positive impact on accuracy among the examined features, so this parameter allows for a large number of levels.

Winter sports like skiing are becoming increasingly popular for both competitive and recreational activities. To minimize the risk of injury, new innovations in skiing equipment have been developed in recent years. However, unexpected slope conditions can still increase risks during skiing. The static categorisation of ski slopes in winter sports resorts does not take into account dynamic changes of difficulty due to high traffic volumes or sudden weather changes. Up to now, efforts have been made to measure the current conditions via satellite imaging or installations on the slope. However, this requires intervention in nature and causes high maintenance costs. To solve these issues we present our preliminary design of a wearable system to let skiers implicitly measure current slope conditions during their skiing experience. Audio and motion data are recorded from a prototype mounted on a ski boot. We show that the data generated by the prototype can be successfully classified with a neural network. We collected data from a skiing activity to demonstrate our concept and discuss the identified challenges in fitting the proposed approach to winter sports equipment.

Conversational interfaces (CIs) have the potential to empower a broader spectrum of users to independently conduct visual analysis. Yet, recent approaches do not fully consider the user's characteristics. In particular, the objective of matching the user's language has been understudied in visual analysis. In order to close this gap, we introduce an answer space motivated by Grice's cooperative principle for framing personalized communication in complex data situations. We conducted both an online survey (N=76) to analyze communication preferences and a qualitative experiment (N=10) to investigate personalized conversations with an existing CI. In order to match the user's language properly, our results suggest to consider additional user characteristics along with their knowledge level. While mismatching communication preferences triggers negative reactions, a preference-aligned communication evokes positive reactions. As our analysis confirms the importance of matching the user's language in visual analysis, we provide design implications for future CIs.

Digital pens emit ink on paper and digitize handwriting. The range of the pen is typically limited to a special writing surface on which the pen's tip is tracked. We present Pentelligence, a pen for handwritten digit recognition that operates on regular paper and does not require a separate tracking device. It senses the pen tip's motions and sound emissions when stroking. Pen motions and writing sounds exhibit complementary properties. Combining both types of sensor data substantially improves the recognition rate. Hilbert envelopes of the writing sounds and mean-filtered motion data are fed to neural networks for majority voting. The results on a dataset of 9408 handwritten digits taken from 26 individuals show that motion+sound outperforms single-sensor approaches at an accuracy of 78.4% for 10 test users. Retraining the networks for a single writer on a dataset of 2120 samples increased the precision to 100% for single handwritten digits at an overall accuracy of 98.3%.

The human body reveals emotional and bodily states through measurable signals, such as body language and electroencephalography. However, such manifestations are difficult to communicate to others remotely. We propose EmotionActuator, a proof-of-concept system to investigate the transmission of emotional states in which the recipient performs emotional gestures to understand and interpret the state of the sender.We call this kind of communication embodied emotional feedback, and present a prototype implementation. To realize our concept we chose four emotional states: amused, sad, angry, and neutral. We designed EmotionActuator through a series of studies to assess emotional classification via EEG, and create an EMS gesture set by comparing composed gestures from the literature to sign-language gestures. Through a final study with the end-to-end prototype interviews revealed that participants like implicit sharing of emotions and find the embodied output to be immersive, but want to have control over shared emotions and with whom. This work contributes a proof of concept system and set of design recommendations for designing embodied emotional feedback systems.

Current virtual and augmented reality head-mounted displays usually include no or only a single vibration motor for haptic feedback and do not use it for guidance. We present HapticHead, a system utilizing multiple vibrotactile actuators distributed in three concentric ellipses around the head for intuitive haptic guidance through moving tactile cues. We conducted three experiments, which indicate that HapticHead vibrotactile feedback is both faster (2.6 s vs. 6.9 s) and more precise (96.4% vs. 54.2% success rate) than spatial audio (generic head-related transfer function) for finding visible virtual objects in 3D space around the user. The baseline of visual feedback is as expected more precise (99.7% success rate) and faster (1.3 s) in comparison, but there are many applications in which visual feedback is not desirable or available due to lighting conditions, visual overload, or visual impairments. Mean final precision with HapticHead feedback on invisible targets is 2.3° compared to 0.8° with visual feedback. We successfully navigated blindfolded users to real household items at different heights using HapticHead vibrotactile feedback independently of a head-mounted display.

Current mobile devices commonly use vibration feedback to signal incoming notifications. However, vibration feedback exhibits strong attention capture, limiting its use to short periods and prominent notifications. Instead, we investigate the use of compression feedback for notifications, which scales from subtle stimuli to strong ones and can provide sustained stimuli over longer periods. Compression feedback utilizes inflatable straps around a user's limbs, a form factor allowing for easy integration into many common wearables. We explore technical aspects of compression feedback and investigate its psychophysical properties with several lab and in situ studies. Furthermore, we show how compression feedback enables reactive feedback. Here, deflation patterns are used to reveal further information on a user's query. We also compare compression and vibrotactile feedback and find that they have similar performance.

Electrical muscle stimulation (EMS) has been used successfully in HCI to generate force feedback and simple movements both in stationary and mobile settings. However, many natural limb movements require the coordinated actuation of multiple muscles. Off-the-shelf EMS devices are typically limited in their ability to generate fine-grained movements, because they only have a low number of channels and do not provide full control over the EMS parameters. More capable medical devices are not designed for mobile use or still have a lower number of channels and less control than is desirable for HCI research. In this paper we present the concept and a prototype of a 20-channel mobile EMS system that offers full control over the EMS parameters. We discuss the requirements of wearable multi-electrode EMS systems and present the design and technical evaluation of our prototype. We further outline several application scenarios and discuss safety and certification issues.

Emoji, a set of pictographic Unicode characters, have seen strong uptake over the last couple of years. All common mobile platforms and many desktop systems now support emoji entry, and users have embraced their use. Yet, we currently know very little about what makes for good emoji entry. While soft keyboards for text entry are well optimized, based on language and touch models, no such information exists to guide the design of emoji keyboards. In this article, we investigate of the problem of emoji entry, starting with a study of the current state of the emoji keyboard implementation in Android. To enable moving forward to novel emoji keyboard designs, we then explore a model for emoji similarity that is able to inform such designs. This semantic model is based on data from 21 million collected tweets containing emoji. We compare this model against a solely description-based model of emoji in a crowdsourced study. Our model shows good perfor mance in capturing detailed relationships between emoji.

Electrical muscle stimulation (EMS) is a promising wearable haptic output technology as it can be miniaturized considerably and delivers a wide range of haptic output. However, prototyping EMS applications is challenging. It requires detailed knowledge and skills about hardware, software, and physiological characteristics. To simplify prototyping with EMS in mobile and wearable situations we present the Let Your Body Move toolkit. It consists of (1) a hardware control module with Bluetooth communication that uses off-the-shelf EMS devices as signal generators, (2) a simple communications protocol to connect mobile devices, and (3) a set of control applications as starting points for EMS prototyping. We describe EMS-specific parameters, electrode placements on the skin, and user calibration. The toolkit was evaluated in a workshop with 10 researchers in haptics. The results show that the toolkit allows to quickly generate non-trivial prototypes. The hardware schematics and software components are available as open source software.

With the increasing popularity of smartwatches over the last years, there has been a substantial interest in novel input methods for such small devices. However, feedback modalities for smartwatches have not seen the same level of interest. This is surprising, as one of the primary function of smartwatches is their use for notifications. It is the interrupting nature of current notifications on smartwatches that has also drawn some of the more critical responses to them. Here, we present a subtle notification mechanism for smartwatches that uses light scattering in a wearer's skin as a feedback modality. This does not disrupt the wearer in the same way as vibration feedback and also connects more naturally with the user's body.

Current soft keyboards for emoji entry all present emoji in the same way: in long lists, spread over several categories. While categories limit the number of emoji in each individual list, the overall number is still so large, that emoji entry is a challenging task. The task takes particularly long if users pick the wrong category when searching for an emoji. Instead, we propose a new zooming keyboard for emoji entry. Here, users can see all emoji at once, aiding in building spatial memory where related emoji are to be found. We compare our zooming emoji keyboard against the Google keyboard and find that our keyboard allows for 18% faster emoji entry, reducing the required time for one emoji from 15.6 s to 12.7 s. A preliminary longitudinal evaluation with three participants showed that emoji entry time over the duration of the study improved at up to 60 % to a final average of 7.5 s.

Pedestrian navigation systems require users to perceive, interpret, and react to navigation information. This can tax cognition as navigation information competes with information from the real world. We propose actuated navigation, a new kind of pedestrian navigation in which the user does not need to attend to the navigation task at all. An actuation signal is directly sent to the human motor system to influence walking direction. To achieve this goal we stimulate the sartorius muscle using electrical muscle stimulation. The rotation occurs during the swing phase of the leg and can easily be counteracted. The user therefore stays in control. We discuss the properties of actuated navigation and present a lab study on identifying basic parameters of the technique as well as an outdoor study in a park. The results show that our approach changes a user's walking direction by about 16°/m on average and that the system can successfully steer users in a park with crowded areas, distractions, obstacles, and uneven ground.

We present a novel way to recognize users by the way they press a button. Our approach allows low-effort and fast interaction without the need for augmenting the user or controlling the environment. It eschews privacy concerns of methods such as fingerprint scanning. Button pressing behavior is sufficiently discriminative to allow distinguishing users within small groups. This approach combines recognition and action in a single step, e.g., getting and tallying a coffee can be done with one button press. We deployed our system for 5 users over a period of 4 weeks and achieved recognition rates of 95% in the last week. We also ran a larger scale but short-term evaluation to investigate effects of group size and found that our method degrades gracefully for larger groups.

For many people their phones have become their main everyday tool. While phones can fulfill many different roles they also require users to (1) make do with affordance not specialized for the specific task, and (2) closely engage with the device itself. We propose utilizing the space and objects around the phone to offer better task affordance and to create an opportunity for casual interactions. Such around-device devices are a class of interactors that do not require users to bring special tangibles, but repurpose items already found in the user's surroundings. In a survey study, we determine which places and objects are available to around-device devices. Furthermore, in an elicitation study, we observe what objects users would use for ten interactions.

The thumb and the fingers have different flexibility, and thus, gestures performed on the back of a held tablet are suggested to be different from ones performed on the touchscreen with the thumb of grasping hands. APIs for back-of-device gesture detection should consider that difference. In a user study, we recorded vectors for the four most common touch gestures. We found that drag, swipe, and press gestures are significantly differently when executed on the back versus on the front side of a held tablet. Corresponding values are provided that may be used to define gesture detection thresholds for back-of-tablet interaction.

There is a well-established culture of early prototyping when designing digital interactive systems, such as paper prototyping and wireframe methods. The culture of designing physical objects is somewhat different: early explorations of form is still prototyped via 2D sketches or renderings, but - mostly because of the construction effort involved - prototyping of actual physical objects is deferred to later stages. A problem occurs when designing mixed physical-digital systems, such as tangible user interfaces (TUIs) on interactive surfaces: the high degree of interactivity means that early prototyping is vital, yet there is no viable process for prototyping both the physical and digital aspects simultaneously on a low-fidelity (low-fi) level. Our solution is Paperbox, a toolkit for exploring design ideas for tangible interaction on interactive surfaces. It supports the early exploration of different form factors and immediately provides digital interactivity for the low-fidelity TUI prototypes built with it. We observed our toolkit in use in various settings: as a brainstorming tool by junior designers; in the development of a consumer electronics product in a large industrial company by senior designers; and in a usability study comparing the effect of different levels of fidelity on the outcome. The lessons learnt will enable others to replicate and extend our approach.

Motivated by the addition of gyroscopes to a large number of new smart phones, we study the effects of combining accelerometer and gyroscope data on the recognition rate of motion gesture recognizers with dimensionality constraints. Using a large data set of motion gestures we analyze results for the following algorithms: Protractor3D, Dynamic Time Warping (DTW) and Regularized Logistic Regression (LR). We chose to study these algorithms because they are relatively easy to implement, thus well suited for rapid prototyping or early deployment during prototyping stages. For use in our analysis, we contribute a method to extend Protractor3D to work with the 6D data obtained by combining accelerometer and gyroscope data. Our results show that combining accelerometer and gyroscope data is beneficial also for algorithms with dimensionality constraints and improves the gesture recognition rate on our data set by up to 4%.

We present a wearable interface that consists of motion sensors. As the interface can be worn on the user's fingers (as a ring) or fixed to it (with nail polish), the device controlled by finger gestures can be any generic object, provided they have an interface for receiving the sensor's signal. We implemented four gestures: tap, release, swipe, and pitch, all of which can be executed with a finger of the hand holding the device. In a user study we tested gesture appropriateness for the index finger at the back of a handheld tablet that offered three different form factors on its rear: flat, convex, and concave (undercut). For all three shapes, the gesture performance was equally good, however pitch performed better on all surfaces than swipe. The proposed interface is an example towards the idea of ubiquitous computing and the vision of seamless interactions with grasped objects. As an initial application scenario we implemented a camera control that allows the brightness to be configured using our tested gestures on a common SLR device.

Rotating 3D objects is a difficult task on mobile devices, because the task requires 3 degrees of freedom and (multi-)touch input only allows for an indirect mapping. We propose a novel style of mobile interaction based on mid-air gestures in proximity of the device to increase the number of DOFs and alleviate the limitations of touch interaction with mobile devices. While one hand holds the device, the other hand performs mid-air gestures in proximity of the device to control 3D objects on the mobile device's screen. A flat hand pose defines a virtual surface which we refer to as the PalmSpace for precise and intuitive 3D rotations. We constructed several hardware prototypes to test our interface and to simulate possible future mobile devices equipped with depth cameras. We conducted a user study to compare 3D rotation tasks using the most promising two designs for the hand location during interaction -- behind and beside the device -- with the virtual trackball, which is the current state-of-art technique for orientation manipulation on touch-screens. Our results show that both variants of PalmSpace have significantly lower task completion times in comparison to the virtual trackball.

When the user is engaged with a real-world task it can be inappropriate or difficult to use a smartphone. To address this concern, we developed ShoeSense, a wearable system consisting in part of a shoe-mounted depth sensor pointing upward at the wearer. ShoeSense recognizes relaxed and discreet as well as large and demonstrative hand gestures. In particular, we designed three gesture sets (Triangle, Radial, and Finger-Count) for this setup, which can be performed without visual attention. The advantages of ShoeSense are illustrated in five scenarios: (1) quickly performing frequent operations without reaching for the phone, (2) discreetly performing operations without disturbing others, (3) enhancing operations on mobile devices, (4) supporting accessibility, and (5) artistic performances. We present a proof-of-concept, wearable implementation based on a depth camera and report on a lab study comparing social acceptability, physical and mental demand, and user preference. A second study demonstrates a 94-99% recognition rate of our recognizers.

Graspable tangibles are now being explored on the current generation of capacitive touch surfaces, such as the iPad and the Android tablet. Because the size and form factor is relatively new, early and low fidelity prototyping of these TUIs is crucial in getting the right design. The problem is that it is difficult for the average interaction designer to develop such physical prototypes. They require a substantial amount time and effort to physically model the tangibles, and expertise in electronics to instrument them. Thus prototyping is sometimes handed off to specialists, or is limited to only a few design iterations and alternative designs. Our solution contributes a low fidelity prototyping approach that is time and cost effective, and that requires no electronics knowledge. First, we supply non-specialists with cardboard forms to create tangibles. Second, we have them draw lines on it via conductive ink, which makes their objects recognizable by the capacitive touch screen. They can then apply routine programming to recognize these tangibles and thus iterate over various designs.

Rohs and Oulasvirta (2008) proposed a two-component Fitts' law model for target acquisition with magic lenses in mobile augmented reality (AR) with 1) a physical pointing phase, in which the target can be directly observed on the background surface, and 2) a virtual pointing phase, in which the target can only be observed through the device display. The model provides a good fit (R2=0.88) with laboratory data, but it is not known if it generalizes to real-world AR tasks. In the present outdoor study, subjects (N=12) did building-selection tasks in an urban area. The differences in task characteristics to the laboratory study are drastic: targets are three-dimensional and they vary in shape, size, z-distance, and visual context. Nevertheless, the model yielded an R2 of 0.80, and when using effective target width an R2 of 0.88 was achieved.

Even with the rise of the World Wide Web, TV has remained the most pervasive entertainment medium and is nowadays often used together with other media, which allow for active participation. The idea of connecting non-collocated TV viewers via telecommunication technologies, referred to as Social TV, has recently received considerable attention. Such systems typically include set-top boxes for supporting collaboration. In this research we investigate if real-time opinion sharing about TV shows through a nonverbal (non-textual) iconic UI on mobile phones is reasonable. For this purpose we developed a mobile app, made it available to a large number of users through the Android Market, and conducted an uncontrolled user study in the wild during the soccer world cup 2010. The results of the study indicate that TV viewers who used the app had more fun and felt more connected to other viewers. We also show that by monitoring this channel it is possible to collect sentiments relevant to the broadcasted content in real-time. The collected data exemplify that the aggregated sentiments correspond to important moments, and hence can be used to generate a summary of the event.

Protractor 3D is a gesture recognizer that extends the 2D touch screen gesture recognizer Protractor to 3D gestures. It inherits many of Protractor's desirable properties, such as high recognition rate, low computational and low memory requirements, ease of implementation, ease of customization, and low number of required training samples. Protractor 3D is based on a closed-form solution to finding the optimal rotation angle between two gesture traces involving quaternions. It uses a nearest neighbor approach to classify input gestures. It is thus well-suited for application in resource-constrained mobile devices. We present the design of the algorithm and a study that evaluated its performance.

In the search for novel and more expressive interaction techniques for mobile devices, bodily aspects such as movement, gesture, and touch based interfaces are prominent. For instance, touch-screen gestures have found widespread application in mobile device interfaces while bodily gestures involving device movement are successfully applied in gaming scenarios. Research systems increasingly explore other modalities, like pressure, free-hand and on body interaction in mobile settings. This has become possible through on-going developments that have made sensing and actuating technologies cheaper and more easily integrated in mobile and handheld devices. The turn towards experiential, embodied, and enacted perspectives on cognition and action has also contributed to a shift in what aspects of interaction to focus upon in interaction design. This has led HCIresearchers to explore not only how the whole human body can be taken into account in design, but also to explore new domains of application for instance in leisure, entertainment and public urban environments.

We conducted a series of user studies to understand and clarify the fundamental characteristics of pressure in user interfaces for mobile devices. We seek to provide insight to clarify a longstanding discussion on mapping functions for pressure input. Previous literature is conflicted about the correct transfer function to optimize user performance. Our study results suggest that the discrepancy can be explained by different signal conditioning circuitry and with improved signal conditioning the user-performed precision relationship is linear. We also explore the effects of hand pose when applying pressure to a mobile device from the front, the back, or simultaneously from both sides in a pinching movement. Our results indicate that grasping type input outperforms single-sided input and is competitive with pressure input against solid surfaces. Finally we provide an initial exploration of non-visual multimodal feedback, motivated by the desire for eyes-free use of mobile devices. The findings suggest that non-visual pressure input can be executed without degradation in selection time but suffers from accuracy problems.

We present the $3 Gesture Recognizer, a simple but robust gesture recognition system for input devices featuring 3D acceleration sensors. The algorithm is designed to be implemented quickly in prototyping environments, is intended to be device-independent and does not require any special toolkits or frameworks. It relies solely on simple trigonometric and geometric calculations. A user evaluation of our system resulted in a correct gesture recognition rate of 80%, when using a set of 10 unique gestures for classification. Our method requires significantly less training data than other gesture recognizers and is thus suited to be deployed and to deliver results rapidly.

We present the $3 Gesture Recognizer, a simple but robust gesture recognition system for input devices featuring 3D acceleration sensors. The algorithm is designed to be implemented quickly in prototyping environments, is intended to be device-independent and does not require any special toolkits or frameworks, but relies solely on simple trigonometric and geometric calculations. Our method requires significantly less training data than other gesture recognizers and is thus suited to be deployed and to deliver results rapidly.

In this paper we present a novel interface for mobile map navigation based on Semi-Automatic Zooming (SAZ). SAZ gives the user the ability to manually control the zoom level of an SDAZ interface, while retaining the automatic zooming characteristics of that interface at times when the user is not explicitly controlling the zoom level. In a user study conducted using a realistic mobile map with a wide scale space, we compare SAZ with existing map interface techniques, multi-touch and Speed-Dependent Automatic Zooming (SDAZ). We extend a dynamic state-space model for Speed-Dependent Automatic Zooming (SDAZ) to accept 2D tilt input for scroll rate and zoom level control and implement a dynamically zoomable map view with access to high-resolution map material for use in our study. The study reveals that SAZ performs significantly better than SDAZ and that SAZ is comparable in performance and usability to a standard multi-touch map interface. Furthermore, the study shows that SAZ could serve as an alternative to multi-touch as input technique for mobile map interfaces.

Gestures can offer an intuitive way to interact with a computer. In this paper, we investigate the question whether gesturing with a mobile phone can help to perform complex tasks involving two devices. We present results from a user study, where we asked participants to spontaneously produce gestures with their phone to trigger a set of different activities. We investigated three conditions (device configurations): phone-to-phone, phone-to-tabletop, and phone to public display. We report on the kinds of gestures we observed as well as on feedback from the participants, and provide an initial assessment of which sensors might facilitate gesture recognition in a phone. The results suggest that phone gestures have the potential to be easily understood by end users and that certain device configurations and activities may be well suited for gesture control.

In this paper we explore the design space of around-device interaction (ADI). This approach seeks to expand the interaction possibilities of mobile and wearable devices beyond the confines of the physical device itself to include the space around it. This enables rich 3D input, comprising coarse movement-based gestures, as well as static position-based gestures. ADI can help to solve occlusion problems and scales down to very small devices. We present a novel around-device interaction interface that allows mobile devices to track coarse hand gestures performed above the device's screen. Our prototype uses infrared proximity sensors to track hand and finger positions in the device's proximity. We present an algorithm for detecting hand gestures and provide a rough overview of the design space of ADI-based interfaces.

By equipping a mobile device with distance sensing capabilities, we aim to expand the interaction possibilities of mobile and wearable devices beyond the confines of the physical device itself to include the space immediately around it. Our prototype, an Apple iPhone equipped with six IR distance sensors, allows for rich 3D input, comprising coarse movement-based hand gestures, as well as static position-based gestures. A demonstration application, HoverFlow, illustrates the use of coarse hand gestures for interaction with mobile applications. This type of interaction, which we call Around-Device Interaction (ADI) has the potential to help to solve occlusion problems on small-screen mobile devices and scales well to small device sizes.

We conducted a user study to investigate the effect of visual context in handheld augmented reality interfaces. A dynamic peephole interface (without visual context beyond the device display) was compared to a magic lens interface (with video see-through augmentation of external visual context). The task was to explore objects on a map and look for a specific attribute shown on the display. We tested different sizes of visual context as well as different numbers of items per area, i.e. different item densities. We found that visual context is most effective for sparse item distributions and the performance benefit decreases with increasing density. User performance in the magic lens case approaches the performance of the dynamic peephole case the more densely spaced the items are. In all conditions, subjective feedback indicates that participants generally prefer visual context over the lack thereof. The insights gained from this study are relevant for designers of mobile AR and dynamic peephole interfaces by suggesting when external visual context is most beneficial.

In many mid- to large-sized cities public maps are ubiquitous. One can also find a great number of maps in parks or near hiking trails. Public maps help to facilitate orientation and provide special information to not only tourists but also to locals who just want to look up an unfamiliar place while on the go. These maps offer many advantages compared to mobile maps from services like Google Maps Mobile or Nokia Maps. They often show local landmarks and sights that are not shown on standard digital maps. Often these 'You are here' (YAH) maps are adapted to a special use case, e.g. a zoo map or a hiking map of a certain area. Being designed for a fashioned purpose these maps are often aesthetically well designed and their usage is therefore more pleasant. In this paper we present a novel technique and application called PhotoMap that uses images of 'You are here' maps taken with a GPS-enhanced mobile camera phone as background maps for on-the-fly navigation tasks. We discuss different implementations of the main challenge, namely helping the user to properly georeference the taken image with sufficient accuracy to support pedestrian navigation tasks. We present a study that discusses the suitability of various public maps for this task and we evaluate if these georeferenced photos can be used for navigation on GPS-enabled devices.

Classic paper-based maps provide high-resolution, large-scale information and are ubiquitous in larger cities and even outdoors. In this paper we present a combination of their advantages with a mobile camera projector unit to create a new mobile and intuitive buddy finder interface. Current friend or buddy finder systems on mobile phones such as Google Latitude suffer from the small screen estate and display size of mobile devices. To overcome this problem we use a lightweight mobile camera projector unit to augment the paper map with a projected overlay of the buddies' positions. By using digital and geo-referenced representations of public maps no extra preperation for the tracking is needed. The idea is presented by enhancing Google Latitude, which allows users to browse the positions of their friends, to project the positions directly on the paper map without the cumbersome panning and zooming of a digital map on a small display.

When camera phones are used as magic lenses in handheld augmented reality applications involving wall maps or posters, pointing can be divided into two phases: (1) an initial coarse physical pointing phase, in which the target can be directly observed on the background surface, and (2) a fine-control virtual pointing phase, in which the target can only be observed through the device display. In two studies, we show that performance cannot be adequately modeled with standard Fitts' law, but can be adequately modeled with a two-component modification. We chart the performance space and analyze users' target acquisition strategies in varying conditions. Moreover, we show that the standard Fitts' law model does hold for dynamic peephole pointing where there is no guiding background surface and hence the physical pointing component of the extended model is not needed. Finally, implications for the design of magic lens interfaces are considered.

Virtual globes have progressed from little-known technology to broadly popular software in a mere few years. We investigated this phenomenon through a survey and discovered that, while virtual globes are en vogue, their use is restricted to a small set of tasks so simple that they do not involve any spatial thinking. Spatial thinking requires that users ask "what is where" and "why"; the most common virtual globe tasks only include the "what". Based on the results of this survey, we have developed a multi-touch virtual globe derived from an adapted virtual globe paradigm designed to widen the potential uses of the technology by helping its users to inquire about both the "what is where" and "why" of spatial distribution. We do not seek to provide users with full GIS (geographic information system) functionality, but rather we aim to facilitate the asking and answering of simple "why" questions about general topics that appeal to a wide virtual globe user base.

The Mobile HCI community is moving beyond the interaction between a single user and her mobile device taking the users environment into account. Mobile interaction with the real world concentrates on using mobile devices as tools to interact with real world objects. This workshop continues the successful mobile interaction with the real world workshops 2006 and 2007. Relevant topics include (but are not limited to) mobile interaction with the real world; mobile devices as user interfaces for terminals; and Frameworks, middleware and APIs for the development of applications that take mobile interactions with the real world into account. The workshop combines technical presentations with the presentation of prototypes and focused discussions to drive interaction between participants.

A user study was conducted to compare the performance of three methods for map navigation with mobile devices. These methods are joystick navigation, the dynamic peephole method without visual context, and the magic lens paradigm using external visual context. The joystick method is the familiar scrolling and panning of a virtual map keeping the device itself static. In the dynamic peephole method the device is moved and the map is fixed with respect to an external frame of reference, but no visual information is present outside the device's display. The magic lens method augments an external content with graphical overlays, hence providing visual context outside the device display. Here too motion of the device serves to steer navigation. We compare these methods in a study measuring user performance, motion patterns, and subjective preference via questionnaires. The study demonstrates the advantage of dynamic peephole and magic lens interaction over joystick interaction in terms of search time and degree of exploration of the search space.

Information navigation on handheld displays is characterized by the small display dimensions and limited input capabilities of today's mobile devices. Special strategies are required to help users navigate to off-screen content and develop awareness of spatial layouts despite the small display. On the other hand, handheld devices offer interaction possibilities that desktop computers do not. Handheld devices can easily be moved in space and used as a movable window into a large virtual workspace. We investigate different information navigation methods for small-scale handheld displays using a range of sensor technologies for spatial tracking. We compare user performance in an map navigation task and discuss the tradeoffs of the different sensor and visualization techniques.

Everyday experience as well as recent studies tell that information contained in ecological sonic feedback may improve human control of, and interaction with, a system. This notion is particularly worthwhile to consider in the context of mobile, tilting-based interfaces as have been proposed, developed and studied extensively. Two interfaces are used for this scope, the Ballancer, based on the metaphor of balancing a rolling ball on a track, and a more concretely application-oriented setup of a mobile phone with tilting-based input. First pilot studies have been conducted.

Information navigation techniques for handheld devices support interacting with large virtual spaces on small displays, for example finding targets on a large-scale map. Since only a small part of the virtual space can be shown on the screen at once, typical interfaces allow for scrolling and panning to reach off-screen content. Spatially aware handheld displays sense their position and orientation in physical space in order to provide a corresponding view in virtual space. We implemented various one-handed navigation techniques for camera-tracked spatially aware displays. The techniques are compared in a series of abstract selection tasks that require the investigation of different levels of detail. The tasks are relevant for interfaces that enable navigating large scale maps and finding contextual information on them. The results show that halo is significantly faster than other techniques. In complex situations zoom and halo show comparable performance. Surprisingly, the combination of halo and zooming is detrimental to user performance.

No abstract available.