Smartwatch interaction is typically limited by the device’s compact design and restricted input capabilities. Using the touch display occludes visual content, and for additional physical controls (e.g., buttons, bezel, crown) space is constrained. This paper enhances smartwatch interaction by expanding the functionality and thus increasing the input bandwidth of the crown. Initially, we investigate how users engage with a crown using two different numbers of fingers. Our informative study shows that crown rotation with one finger is faster compared to using two fingers. However, two-finger usage reduces the overshoot amplitude as well as number of fine-grained correction movements in a selection task. Subsequently, we develop a concept and a prototype that differentiates the number of fingers used in the crown interaction, which we call CapCrown . Finally, we discuss a set of applications that utilize this enhanced crown given the differences in interacting with one vs. two fingers. Our contributions include the study on crown interaction with different fingers, the sensor hardware, and the software using machine learning to detect fingers rotating a crown. Finally, we propose new smartwatch applications and system-wide functions that take advantage of our approach.

Space is an essential part of social communication. The distance individuals choose when communicating with each other conveys non-verbal information and social signals. In remote, audiovisual communication, though, spatial information is hardly considered in the technical set-ups, often arbitrary and unequally perceived by the participants. In this article, we describe an audiovisual communication system that takes distance into account. Based on a display equipped with depth sensors, we divide the space into different zones and use human position in the room to enable interaction between remote participants. This way, the physical availability of a communication partner is imitated and distance between individuals can be used naturally to convey non-verbal information in remote set-ups.

Rotary knobs are widely used input devices for computers, household appliances, cars and other machinery. So far, their input capabilities are limited since the rotation of a knob only controls one single parameter. This paper presents the concept of a rotary knob that is capable of recognizing a user’s grasp for controlling different parameters, called MultiKnob. We contribute the hardware and software of a knob for detecting how many fingers are used when touching it. It uses IR light and FTIR-based multi-touch recognition. Within a target acquisition study, we used this new feature for allowing users to select the speed of a cursor for solving tasks. We found that users are able to leverage this novel device: Our 24 participants could make use of this new knob for being faster when moving a cursor into a target area compared to a fixed slow speed knob, and having less overshoots compared to a fixed fast speed knob. Further, we found that the MultiKnob gave rise to the strategy of changing grasp for switching from a fast and coarse-grained cursor movement to a slow and fine-grained cursor movement. We discuss different applications for using such a knob.

Navigating the environment can be challenging, especially when faced with vision or hearing limitations, such as darkness and visual impairments. To address this issue, we propose using electrical muscle stimulation (EMS) to convey spatial information and assist users in avoiding obstacles. We map direction to EMS channels and distance to current intensity, with force-feedback for closer proximity and tactile EMS feedback for greater proximity. An experimental study (n=12) was conducted to evaluate the concept and gain insights into the qualitative perception of EMS. Participants were able to accurately estimate encoded distance (85.7% accuracy) and direction (99.5% accuracy). Our findings demonstrate the feasibility and effectiveness of using tactile and force feedback EMS to convey spatial information comprehensibly.

The aesthetic appeal of a website has strong effects on users’ reactions, appraisals, and even behaviors. However, evaluating website aesthetics through user ratings is resource intensive, and extant models to predict website aesthetics are limited in performance and ability. We contribute a novel and more precise approach to predict website aesthetics that considers rating distributions. Moreover, we use this approach as a baseline model to illustrate how future research might be conducted using predictions instead of participants. Our approach is based on a deep convolutional neural network model and uses innovations in the field of image aesthetic prediction. It was trained with the dataset from Reinecke and Gajos [2014] and was validated using two independent large datasets. The final model reached an unprecedented cross-validated correlation between the ground truth and predicted rating of LCC = 0.752. We then used the model to successfully replicate prior findings and conduct original research as an illustration for AI-based research.

There are a range of package managers out there which are used in almost every software development project to integrate third party packages to a project. These packages are usually stored in central repositories in the cloud. This central cloud approach brings some downsides especially with regard to fault tolerance which we want to tackle with our approach inspired by thoughts of dew computing. In this paper we present a fully distributed package manager which can be used on cloud as well as on the edge or in a hybrid environment. The approach is based on a peer-to-peer network, implemented with hypercore protocol.

Pokémon GO was a short lived mobile location-based gaming phenomenon. After its launch in July 2016, it quickly reached 500 million installs, but afterwards interest faded. As part of a large scale "in the wild" mobile phone study we have recorded phone usage and location measurements between June and September 2016. We investigate who were the people who installed and played Pokémon GO and what effects it had on their behaviour. We chose as a middle point the start of playing the game, and selected users that had activity for at least two weeks before and two weeks after it. In this work we present our findings on a sample of 2, 861 users. We compare demographic characteristics and Big Five personality traits of these users with 7, 904 non-playing users from the same time period. The general daily phone usage of players increased on average by 27 minutes, which represents 16% per day. In terms of large scale movement patterns, these did not change, with regard to diameter and total path length per day.

The forgetting of objects is an everyday problem. Current research projects develop doorway reminder systems which are supposed to remind the user of certain objects or events before leaving home. At present the focus of research is mainly on the technical design of the recognition of persons and objects. For this reason we are concentrating on the design and evaluation of possible system feedback. In particular, the target group of the digital natives is familiar with the concept of tactile feedback for reminder functions. Therefore known vibration patterns were used, which are to be transferred to the user via the floor. This solution was implemented in a first prototype. A user study with 19 volunteers showed a general acceptance of tactile feedback via floor panel and the cognitive link of the vibration patterns with reminder notifications.

Mobile phones have evolved significantly in recent years from single-purpose communication devices to multi-purpose computing devices. Despite this evolution, the interaction model for how incoming calls are handled has barely changed. Current-generation smartphones still use abrupt full-screen notifications to alert users to incoming calls, demanding a decision to either accept or decline the call. These full-screen notifications forcibly interrupt whatever activity the user was already engaged in. This might be undesirable when the user's primary task was more important than the incoming call. This paper explores the design space for how smartphones can alert users to incoming calls. We consider designs that allow users to postpone calls and also to multiplex by way of a smaller partial-screen notification. These design alternatives were evaluated in both a small-scale controlled lab study as well as a large-scale naturalistic in-the-wild study. Results show that a multiplex design solution works best because it allows people to continue working on their primary task while being made aware that there is a caller on the line. The contribution of this work is an enhanced interaction design for handling phone calls, and an understanding of how people use it for handling incoming calls.

The number of available mobile applications is steadily increasing. People have rapidly adopted application stores as means to customize their devices with various functionalities that go beyond communication. Understanding the principles of mobile application usage is crucial for supporting users within this new ecosystem. In this paper, we investigate how people organize applications they have installed on their devices. We asked more than 130 participants for their habits for icon arrangement and collected more than 1,400 screenshots of their devices' menus to further ground our findings. Based on this data we can distinguish five different concepts for arranging icons on smartphone menus, e.g. based on application usage frequency and applications' functional relatedness. Additionally, we investigated how these concepts emerge in relation to frequency of application installations, removals and icon rearrangements, as well as users' experience levels. Finally we discuss implications for the design of smartphone launchers, and highlight differences to icon arrangement on stationary computers.

Mobile phones have evolved from communication devices to indispensable accessories with access to real-time content. The increasing reliance on dynamic content comes at the cost of increased latency to pull the content from the Internet before the user can start using it. While prior work has explored parts of this problem, they ignore the bandwidth costs of prefetching, incur significant training overhead, need several sensors to be turned on, and do not consider practical systems issues that arise from the limited background processing capability supported by mobile operating systems. In this paper, we make app prefetch practical on mobile phones. Our contributions are two-fold. First, we design an app prediction algorithm, APPM, that requires no prior training, adapts to usage dynamics, predicts not only which app will be used next but also when it will be used, and provides high accuracy without requiring additional sensor context. Second, we perform parallel prefetch on screen unlock, a mechanism that leverages the benefits of prediction while operating within the constraints of mobile operating systems. Our experiments are conducted on long-term traces, live deployments on the Android Play Market, and user studies, and show that we outperform prior approaches to predicting app usage, while also providing practical ways to prefetch application content on mobile phones.

Mobile phones have evolved from communication to multi-purpose devices that assist people with applications in various contexts and tasks. The size of the mobile ecosystem is steadily growing and new applications become available every day. This increasing number of applications makes it difficult for end-users to find good applications. Recommender systems suggesting mobile applications are being built to help people to find valuable applications. Since the nature of mobile applications differs from classical items to be recommended (e.g. books, movies, other goods), not only can new approaches for recommendation be developed, but also new paradigms for evaluating performance of recommender systems are advisable. During the lifecycle of mobile applications, different events can be observed that provide insights into users' engagement with particular applications. This gives rise to new approaches for evaluation of recommender systems. In this paper, we present AppFunnel: a framework that allows for usage-centric evaluation considering different stages of application engagement. We present a case study and discuss capabilities for evaluating recommender engines by applying metrics to the AppFunnel.

While mobile phones made a significant evolution in recent years from single-purpose communication devices to multi-purpose devices, the fundamental design of phone call applications did not evolve accordingly. While its implementation leveraged from new hardware and software capabilities, the fundamental decisions people are able to make when they receive a call did not change. Currently, when a call comes in, a modal dialog opens where the callee can either decline or accept the call. A recent study found that the current user interfaces of phone call applications (phone UIs) often lead to an increased overhead when application usage is being interrupted by phone calls [6]. In this paper, we revisit phone call UIs for multipurpose smartphones. We contribute a new design space for mobile phone call UIs, going beyond the simple accept-or-decline dilemma. We present a prototype implementation and discuss open challenges.

Our personal smartphones are our daily companions, coming with us everywhere, including into enterprise meetings. This paper looks at smartphone use in meetings. Via a survey of 398 enterprise workers, we find that people believe phone use interferes with meeting productivity and collaboration. While individuals tend to think that they make productive use of their own phones in meetings, they perceive others as using their phones for unrelated tasks. To help smartphones create a more collaborative meeting environment, we present an application that identifies and describes meeting attendees. We deploy the application to 114 people at real meetings, and find that users value being able to access information about the other people in the room, particularly when those people are unfamiliar. To prevent users from disengaging from the meeting while using their phones, we employ a gaming approach that asks trivia questions about the other attendees. We observe that gameplay focuses attention within the meeting context and sparks conversations. These findings suggest ways smartphone applications might help users engage with the people around them in enterprise environments, rather than removing them from their immediate social context.

With the increasing adoption of smartphones also a problematic phenomena become apparent: People are changing their habits and become addicted to different services that these devices provide. In this paper we present AppDetox: an app that allows users to purposely create rules that keep them from using certain apps. We describe our deployment of the app on a mobile application store, and present initial findings gained through observation of about 11,700 users of the application. We find that people are rather rigorous when restricting their app use, and that mostly they suppress use of social networking and messaging apps.

Context-aware information is widely available in various ways and is becoming more and more important for enhancing retrieval performance and recommendation results. The current main issue to cope with is not only recommending or retrieving the most relevant items and content, but defining them ad hoc. Other relevant issues include personalizing and adapting the information and the way it is displayed to the user's current situation and interests. Ubiquitous computing further provides new means for capturing user feedback on items and providing information.

The explosive growth of the mobile application (app) market has made it difficult for users to find the most interesting and relevant apps from the hundreds of thousands that exist today. Context is key in the mobile space and so too are proactive services that ease user input and facilitate effective interaction. We believe that to enable truly novel mobile app recommendation and discovery, we need to support real context-aware recommendation that utilizes the diverse range of implicit mobile data available in a fast and scalable manner. In this paper we introduce the Djinn model, a novel context-aware collaborative filtering algorithm for implicit feedback data that is based on tensor factorization. We evaluate our approach using a dataset from an Android mobile app recommendation service called appazaar. Our results show that our approach compares favorably with state-of-the-art collaborative filtering methods.

During sales conversations, gestures and mimics are of high importance to communicate information about a product. One prominent example for such sales gestures is the meat and cheese counter, which is one of the remaining spots in supermarkets where sales persons interact with customers. Interactions at such counters in supermarkets normally follow a simple protocol. The customer points at an item of choice. The employee takes out the item and, in most of the cases the product needs to be cut to fit the amount the customer wants to buy. Often it is ambiguous about what specific product the customer and the employees are talking about. Up to now, there are just a few efforts in HCI research to enrich communication at the point of sale. In this paper we report and analyze one scenario in which an intelligent natural user interface can support communication between customer and employee in a sales conversation. Furthermore, we report on our prototype that is able to track pointing gestures by using a depth camera and to display information about items pointed at.

Context-aware information is widely available in various ways and is becoming more and more important for enhancing retrieval performance and recommendation results. The current main issue to cope with is not only recommending or retrieving the most relevant items and content, but defining them ad hoc. Other relevant issues include personalizing and adapting the information and the way it is displayed to the user's current situation and interests. Ubiquitous computing further provides new means for capturing user feedback on items and providing information.

Smartphone users might be interrupted while interacting with an application, either by intended or unintended circumstances. In this paper, we report on a large-scale observational study that investigated mobile application interruptions in two scenarios: (1) intended back and forth switching between applications and (2) unintended interruptions caused by incoming phone calls. Our findings reveal that these interruptions rarely happen (at most 10% of the daily application usage), but when they do, they may introduce a significant overhead (can delay completion of a task by up to 4 times). We conclude with a discussion of the results, their limitations, and a series of implications for the design of mobile phones.

Context-aware information is widely available in various ways such as interaction patterns, location, devices, annotations, query suggestions and user profiles and is becoming more and more important for enhancing retrieval performance and recommendation results. At the moment, the main issue to cope with is not only recommending or retrieving the most relevant items and content, but defining them ad hoc. Other relevant issues are personalizing and adapting the information and the way it is displayed to the user's current situation (device, location) and interests.

Most context-aware systems rely on physical sensors. To some extent, these systems are able to reason about a user's situation by means of the measured data. However, their overall uncertainty in modeling human behavior leads to ambiguity. A language for the mediation of context information between the user and the system is required to enable the user to adjust the machine's interpretation of his context. We describe how keywords that are attributed to activities by users themselves can act as such a mediator. We present results of a study that investigates the nature of this context attributes. The results demonstrate that different users use similar keywords to describe similar situations and different keywords to describe different situations. Therefore, algorithms developed to evaluate the semantic relatedness of tags and resources within folk-sonomies can be applied to exploit knowledge about the users' contexts from the keywords they have assigned. We discuss a prototype that recommends mobile services by enhancing its model with the described keywords.

In this paper we investigate interaction with a media façade that is out of reach for touch-based interaction. We describe four different applications that utilize mobile devices to enable passers-by to interact with the façade. Each application has been designed constrained by limitations given by formal regulations of an editorial board (e.g. to prevent traffic distractions) and with the aim to catch the attention of passers-by to achieve interaction and keep the users engaged. Besides the description of the design and implementation of the different application, we report on initial feedback of users after a first preliminary user test that informs further development and design.

While applications for mobile devices have become extremely important in the last few years, little public information exists on mobile application usage behavior. We describe a large-scale deployment-based research study that logged detailed application usage information from over 4,100 users of Android-powered mobile devices. We present two types of results from analyzing this data: basic descriptive statistics and contextual descriptive statistics. In the case of the former, we find that the average session with an application lasts less than a minute, even though users spend almost an hour a day using their phones. Our contextual findings include those related to time of day and location. For instance, we show that news applications are most popular in the morning and games are at night, but communication applications dominate through most of the day. We also find that despite the variety of apps available, communication applications are almost always the first used upon a device's waking from sleep. In addition, we discuss the notion of a virtual application sensor, which we used to collect the data.

In this paper we present results from a questionnaire on public displays that aim to increase social cohesion. The displays highlight unexpected social links between passers-by in urban areas, which might lead to the strengthening of existing links and even the creation of new links between people. The results from the questionnaire show that the majority of respondents believes that the proposed displays might increase social cohesion, and about half of them would use them. Some respondents however are afraid for their privacy, security, and being annoyed by strangers. For personal displays, which show information only about single persons, the most popular content are name, interests and contact options. For interpersonal displays, which show information about pairs of people, the most popular information are common friends and common interests. The popularity of the proposed displays however depended strongly on the social context where they would be used. We present a preliminary prototype of a personal display, which is deployed in a university context.

The contextual relevance of a service can only be determined by the human himself. However, a measure for relevance is required for context-aware service delivery. In this paper, we draw attention to icon arrangement on mobile devices as a new source of information for adaptive menus. We conducted contextual inquiries to investigate how people arrange icons on a grid-based menu. Our results show that context has an impact on how users arrange their menus: during different activities they prefer different icons to be placed at specific positions. We discuss layout options for icon menus and argue how the relevance can be approximated by observing the icon arrangement. Our results informed the design of a context-aware client for mobile services, which is presented as a prototype.

The aether soon will be pervaded with a high density of digital services for usage on mobile phones. Personalization plays a crucial role for the success and acceptance of such systems. In this paper, we present work in progress on a new approach for improving the personalization of recommender systems for pervasive services. Mobile utilization is extended from services inherent in devices to pervasive services. We describe a new concept for collaborative and content-based filtering based on the users' service rankings given by arrangement of menu icons and discuss different models for interaction with a varying icon menu.