We present MindPhone, a mindfulness-based intervention to tackle absentminded and excessive smartphone use. At unlock, MindPhone prompts one of two questions: what the user intends to do with the smartphone, or what the user intends to do in the real world after using their smartphone. Users may respond actively by writing, or passively by mentally reflecting. We evaluated the effectiveness of the two questions and two response modes in a mixed-method, 2x2 mixed field study with 28 participants over two weeks. Our results show that the real-world prompt significantly reduces absentminded use and encourages a quicker return to the real world, independent of the response mode. Asking about smartphone use intentions raises awareness of reasons for smartphone use. For everyday use of MindPhone, users wish to set the question and response mode based on context. We discuss including awareness of the physical world in future smartphone use interventions.

Tangible technology provides opportunities to design collaborative interactions which allow children to engage in highly collaborative activities. Unfortunately, there are few guidelines on structuring children’s interdependent collaboration with tangible technologies. In this study, we designed and developed a tangible game named MemorINO to “force” children to collaborate. We conducted a classroom study with 23 children and 3 kindergarten teachers. Our investigation revealed two main findings: (1) We could design interactive constraints with tangible technologies to “force” children to attend collaborative activities naturally and interdependently; (2) Tangible environments could help children have good engagements, especially for similar-age group children. Our findings could provide practical guidance on designing tangible interfaces to help children learn to collaborate.

Gaze-aware interfaces should work on all display sizes. This paper researches whether angular velocity or tangential speed should be kept when scaling a gaze-aware interface based on circular smooth pursuits to another display size. We also address the question of which target speed and which trajectory size feels most comfortable for the users. We present the results of a user study where the participants were asked how they perceived the speed and the radius of a circular moving smooth pursuit target. The data show that the users’ judgment of the optimal speed corresponds with an optimal detection rate. The results also enable us to give an optimal value pair for target speed and trajectory radius. Additionally, we give a functional relation on how to adapt the target speed when scaling the geometry to keep optimal detection rate and user experience.

Tangible learning has received increasing attention. However, in the recent decade, it has no comprehensive overview. This study aimed to fill the gap and reviewed 92 publications from all the TEI conference proceedings (2007–2021). We analysed previous studies’ characteristics (e.g., study purpose and interactive modalities) and elaborated on three common topics: collaborative tangible learning, tangibles’ impacts on learning, and comparisons between tangibles and other interfaces. Three key findings were: (1) Tangibles impacted learning because it could scaffold learning, change learning behaviour, and improve learning emotion; (2) We should see the effectiveness of tangibles with rational and critical minds. (3) Some studies emphasised too much on the interaction of tangibles and ignored their metaphor meanings. For future work, we suggest avoiding an intensive cluster on collaboration and children and consider other valuable areas, e.g., tangibles for teachers, tangibles’ social and emotional impacts on students, tangible interaction’s meaning and metaphor.

Today, adaptive mobile applications use mobile sensing and user tracking, allowing for adaptation to the users' context and needs. This raises several privacy concerns. Privacy dashboards provide transparency and sharing control; however, their impact on the users' behavior is unclear. To shed light on the effects of (a) transparency and (b) control features, we developed a mobile sensing privacy dashboard and evaluated it in the wild (N=227). We found that the pure presentation of raw logging data is rather deterring, and users tend to use the app less, but offering the user control over the data collection can compensate for that. Users used the control features rarely and, as such, did not affect the data collocation. Our work informs the design of future privacy-enhancing interfaces in applications relying on passively collected mobile sensing data. Moreover, our results encourage the adoption of privacy dashboards in the applications and relieve developers from concerns about the negative influences of transparency information on the quality of collected data.

The opaque nature of many intelligent systems violates established usability principles and thus presents a challenge for human-computer interaction. Research in the field therefore highlights the need for transparency, scrutability, intelligibility, interpretability and explainability, among others. While all of these terms carry a vision of supporting users in understanding intelligent systems, the underlying notions and assumptions about users and their interaction with the system often remain unclear. We review the literature in HCI through the lens of implied user questions to synthesise a conceptual framework integrating user mindsets, user involvement, and knowledge outcomes to reveal, differentiate and classify current notions in prior work. This framework aims to resolve conceptual ambiguity in the field and enables researchers to clarify their assumptions and become aware of those made in prior work. We further discuss related aspects such as stakeholders and trust, and also provide material to apply our framework in practice (e.g., ideation/design sessions). We thus hope to advance and structure the dialogue on supporting users in understanding intelligent systems.

Creativity is an important part of children’s education. Tangible User Interfaces (TUIs) provide new possibilities for creative learning. In this review, we gave an overview of recent studies that supported children’s creative learning using TUIs. Results showed that TUIs had many advantages, such as they (1) were novice-friendly, (2) supported children’s cognitive process and development, (3) promoted their initiatives, (4) enabled them to think outside the box, and (5) encouraged communication and collaboration in an authentic context. Meanwhile, we summarized previous work’s three main limitations: First, most of the studies did not have a long-term experimental verification with sufficient sample size and objective evaluation; Second, some TUI designs lacked a balance of abstractness, openness, richness, and complexity; Finally, the use of TUIs had little consideration of the teacher’s role. Therefore, further research should focus more on the trans-disciplinary nature of TUIs for creative learning and leverage collaboration between human-computer interaction researchers and school teachers.

We present a dialogue elicitation study to assess how users envision conversations with a perfect voice assistant (VA). In an online survey, N=205 participants were prompted with everyday scenarios, and wrote the lines of both user and VA in dialogues that they imagined as perfect. We analysed the dialogues with text analytics and qualitative analysis, including number of words and turns, social aspects of conversation, implied VA capabilities, and the influence of user personality. The majority envisioned dialogues with a VA that is interactive and not purely functional; it is smart, proactive, and has knowledge about the user. Attitudes diverged regarding the assistant’s role as well as it expressing humour and opinions. An exploratory analysis suggested a relationship with personality for these aspects, but correlations were low overall. We discuss implications for research and design of future VAs, underlining the vision of enabling conversational UIs, rather than single command “Q&As”.

The opaque nature of many intelligent systems violates established usability principles and thus presents a challenge for human-computer interaction. Research in the field therefore highlights the need for transparency, scrutability, intelligibility, interpretability and explainability, among others. While all of these terms carry a vision of supporting users in understanding intelligent systems, the underlying notions and assumptions about users and their interaction with the system often remain unclear.

This paper describes the impact of different camera distances in cinematic virtual reality. For our findings, we took a closer look at proxemics, the study on how humans behave regarding space and distances. We explored the four proxemics distances (intimate, personal, social, public) and adapted them to camera distances in cinematic virtual reality. For the user study, a stereoscopic movie with a speaking person was produced for four different distances. The results show that different distances cause different feelings in viewers similar to shot sizes in traditional movies. In our scenario - a person in a museum is speaking about an exhibit - the personal distance was preferred by the participants. As an outcome of this work, the proxemics distances were put in relation to well-known shot sizes used in traditional filmmaking.

Fully autonomous driving leaves drivers with little opportunity to intervene in the driving decision. Giving drivers more control can enhance their driving experience. We develop two collaborative interface concepts to increase the user experience of drivers in autonomous vehicles. Our aim is to increase the joy of driving and to give drivers competence and autonomy even when driving autonomously. In a driving simulator study (N = 24) we investigate how vehicles and drivers can collaborate to decide on driving actions together. We compare autonomous driving (AD), the option to take back driving control (TBC) and two collaborative driving interface concepts by evaluating usability, user experience, workload, psychological needs, performance criteria and interview statements. The collaborative interfaces significantly increase autonomy and competence compared to AD. Joy is highly represented in the qualitative data during TBC and collaboration. Collaboration proves to be good for situations in which quick decisions are called for.

Autonomous Vehicles (AVs) are facilitating the development of a diverse set of applications, from human-less delivering to alternative mobility services. There are a variety of challenges for AVs that might be assessed and solved by remote operators, such as sensor data ambiguity, temporary changes to infrastructure, or unexpected interventions by other road users. With this paper, we propose a design space to support the development of appropriate user interfaces for remote situational awareness and teleoperation. The design space is envisioned to discover and evaluate design alternatives before the implementation and to provide a systematic approach towards the development of novel interfaces.

The pervasive availability of media in foreign languages is a rich resource for language learning. However, learners are forced to interrupt media consumption whenever comprehension problems occur. We present BrainCoDe, a method to implicitly detect vocabulary gaps through the evaluation of event-related potentials (ERPs). In a user study (N=16), we evaluate BrainCoDe by investigating differences in ERP amplitudes during listening and reading of known words compared to unknown words. We found significant deviations in N400 amplitudes during reading and in N100 amplitudes during listening when encountering unknown words. To evaluate the feasibility of ERPs for real-time applications, we trained a classifier that detects vocabulary gaps with an accuracy of 87.13% for reading and 82.64% for listening, identifying eight out of ten words correctly as known or unknown. We show the potential of BrainCoDe to support media learning through instant translations or by generating personalized learning content.

Psychological targeting tries to influence and manipulate users' behaviour. We investigated whether users can protect themselves from being profiled by a chatbot, which automatically assesses users' personality. Participants interacted twice with the chatbot: (1) They chatted for 45 minutes in customer service scenarios and received their actual profile (baseline). (2) They then were asked to repeat the interaction and to disguise their personality by strategically tricking the chatbot into calculating a falsified profile. In interviews, participants mentioned 41 different strategies but could only apply a subset of them in the interaction. They were able to manipulate all Big Five personality dimensions by nearly 10%. Participants regarded personality as very sensitive data. As they found tricking the AI too exhaustive for everyday use, we reflect on opportunities for privacy protective designs in the context of personality-aware systems.

We present the first systematic analysis of personality dimensions developed specifically to describe the personality of speech-based conversational agents. Following the psycholexical approach from psychology, we first report on a new multi-method approach to collect potentially descriptive adjectives from 1) a free description task in an online survey (228 unique descriptors), 2) an interaction task in the lab (176 unique descriptors), and 3) a text analysis of 30,000 online reviews of conversational agents (Alexa, Google Assistant, Cortana) (383 unique descriptors). We aggregate the results into a set of 349 adjectives, which are then rated by 744 people in an online survey. A factor analysis reveals that the commonly used Big Five model for human personality does not adequately describe agent personality. As an initial step to developing a personality model, we propose alternative dimensions and discuss implications for the design of agent personalities, personality-aware personalisation, and future research.

With head mounted displays, consumers are able to transition from the real world to virtual realities. However, this requires frequent transitions between the two realities to maintain their physical integrity and awareness of the real world while in the virtual space. We completed two consecutive studies to investigate the dimensions of a system that supports seamless transition between realities without requiring the user to remove the headset. Our results are twofold: First, based on the the analysis of structured interviews (n=20), we present a conceptualization of existing solutions (n=37) and novel ideas (n=9) in the form of a design space. Second, we present the results of a user study (n=36) in which we tested two exemplary prototypes that evolved from the design space, called “Sky Portal” and “Virtual Phone.” Our exploration shows that our “Virtual Phone” metaphor has the potential to support HMD users in completing bidirectional transitions along Milgram's reality-virtuality continuum. Users are also enabled to complete micro-interactions across the realities, even without performance loss.

The purpose of image restoration is to recover the original state of damaged images. To overcome the disadvantages of the traditional, manual image restoration process, like the high time consumption and required domain knowledge, automatic inpainting methods have been developed. These methods, however, can have limitations for complex images and may require a lot of input data. To mitigate those, we present "interactive Deep Image Prior", a combination of manual and automated, Deep-Image-Prior-based restoration in the form of an interactive process with the human in the loop. In this process a human can iteratively embed knowledge to provide guidance and control for the automated inpainting process. For this purpose, we extended Deep Image Prior with a user interface which we subsequently analyzed in a user study. Our key question is whether the interactivity increases the restoration quality subjectively and objectively. Secondarily, we were also interested in how such a collaborative system is perceived by users.

Autonomous vehicles are complex systems that may behave in unexpected ways. From the drivers’ perspective, this can cause stress and lower trust and acceptance of autonomous driving. Prior work has shown that explanation of system behavior can mitigate these negative effects. Nevertheless, it remains unclear in which situations drivers actually need an explanation and what kind of interaction is relevant to them. Using thematic analysis of real-world experience reports, we first identified 17 situations in which a vehicle behaved unexpectedly. We then conducted a think-aloud study (N = 26) in a driving simulator to validate these situations and enrich them with qualitative insights about drivers’ need for explanation. We identified six categories to describe the main concerns and topics during unexpected driving behavior (emotion and evaluation, interpretation and reason, vehicle capability, interaction, future driving prediction and explanation request times). Based on these categories, we suggest design implications for autonomous vehicles, in particular related to collaboration insights, user mental models and explanation requests.

The complex nature of intelligent systems motivates work on supporting users during interaction, for example, through explanations. However, as of yet, there is little empirical evidence in regard to specific problems users face when applying such systems in everyday situations. This article contributes a novel method and analysis to investigate such problems as reported by users: We analysed 45,448 reviews of four apps on the Google Play Store (Facebook, Netflix, Google Maps, and Google Assistant) with sentiment analysis and topic modelling to reveal problems during interaction that can be attributed to the apps’ algorithmic decision-making. We enriched this data with users’ coping and support strategies through a follow-up online survey (N = 286). In particular, we found problems and strategies related to content, algorithm, user choice, and feedback. We discuss corresponding implications for designing user support, highlighting the importance of user control and explanations of output rather than processes.

Maintaining awareness of real world boundaries whilst being immersed in virtual reality (VR) with head mounted displays (HMDs), is a necessity for the physical integrity of the user. This paper explores whether individual human senses can be allocated to the real and the virtual world and what effect this has on workload, presence, performance and perceived safety. We present the results of a lab study ( N=33) where the auditory and haptic sense of participants was trained to be an indicator for real world boundaries, while their visual sense was bound to a VR experience with an HMD. Our results suggests that allocating senses increases workload. However, while performance is comparable to purely visual indications of boundaries, sense allocation seems to improve presence. Participants prefer the signals to be separate or combined subsequently, depending on the priority and proximity to the boundary. This exploratory study is valuable for developers and researchers who want to start including audio and haptic signals as indicators for real world boundaries.

Advances in vehicle automation and the resulting change of the interior of cars lead to new challenges for user interface concepts. Augmented reality (AR) is a promising solution for the emerging design needs due to its diverse opportunities for user interaction and presenting information. This paper supports the development of novel AR applications. We describe a corresponding use case set consisting of 98 examples from a literature review and two focus groups. Based on these samples we present a design space for in-car AR applications. To demonstrate the benefit thereof, we show a fictional design process including our proposed design space to derive a custom AR system. This work supports designers and engineers by providing a systematic approach for integrating 3D AR interfaces in a vehicle, excluding windshields and windows.

Mindfulness is a term seen with increasing frequency in HCI literature, and yet the term itself is used almost as variously as the number of papers in which it appears. This diversity makes comparing or evaluating HCI approaches around mindfulness or understanding the design space itself a challenging task. We conducted a structured ACM literature search based on the term mindfulness. Our selection process yielded 38 relevant papers, which we analyzed for their definition, motivation, practice, evaluation and technology use around mindfulness. We identify similarities, divergences and areas of interest for each aspect, resulting in a framework composed of four perspectives and seven lines of research. We highlight challenges and opportunities for future HCI research and design.

The complex nature of intelligent systems motivates work on supporting users during interaction, for example through explanations. However, there is yet little empirical evidence on specific problems users face in such systems in everyday use. This paper investigates such problems as reported by users: We analysed 35,448 reviews of three apps on the Google Play Store (Facebook, Netflix and Google Maps) with sentiment analysis and topic modelling to reveal problems during interaction that can be attributed to the apps' algorithmic decision-making. We enriched this data with users' coping and support strategies through a follow-up online survey (N=286). In particular, we found problems and strategies related to content, algorithm, user choice, and feedback. We discuss corresponding implications for designing user support, highlighting the importance of user control and explanations of output, not processes. Our work thus contributes empirical evidence to facilitate understanding of users' everyday problems with intelligent systems.

Emojis are commonly used as non-verbal cues in texting, yet may also lead to misunderstandings due to their often ambiguous meaning. User personality has been linked to understanding of emojis isolated from context, or via indirect personality assessment through text analysis. This paper presents the first study on the influence of personality (measured with BFI-2) on understanding of emojis, which are presented in concrete mobile messaging contexts: four recipients (parents, friend, colleague, partner) and four situations (information, arrangement, salutory, romantic). In particular, we presented short text chat scenarios in an online survey (N=646) and asked participants to add appropriate emojis. Our results show that personality factors influence the choice of emojis. In another open task participants compared emojis found as semantically similar by related work. Here, participants provided rich and varying emoji interpretations, even in defined contexts. We discuss implications for research and design of mobile texting interfaces.

Immersive Virtual Reality (IVR) is a growing 3D environment, where social and commercial applications will require user authentication. Similarly, smart homes in the real world (RW), offer an opportunity to authenticate in the third dimension. For both environments, there is a gap in understanding which elements of the third dimension can be leveraged to improve usability and security of authentication. In particular, investigating transferability of findings between these environments would help towards understanding how rapid prototyping of authentication concepts can be achieved in this context. We identify key elements from prior research that are promising for authentication in the third dimension. Based on these, we propose a concept in which users' authenticate by selecting a series of 3D objects in a room using a pointer. We created a virtual 3D replica of a real world room, which we leverage to evaluate and compare the factors that impact the usability and security of authentication in IVR and RW. In particular, we investigate the influence of randomized user and object positions, in a series of user studies (N=48). We also evaluate shoulder surfing by real world bystanders for IVR (N=75). Our results show that 3D passwords within our concept are resistant against shoulder surfing attacks. Interactions are faster in RW compared to IVR, yet workload is comparable.

For watching omnidirectional movies via HMD, turning the head is the most relevant and natural input technique to choose the visible part of the movie. However, there is a need for additional interactivity in cinematic virtual reality (CVR), e.g. for navigating the movie, for nonlinear story lines or for communication with other viewers watching a movie together. The input device should not disturb the viewing experience and the viewer should not be primarily aware of it. We present a design space based on numerous methods in literature and our own experiences. As a result of the design space we describe interaction techniques which meet the challenges of cinematic virtual reality. For doing this, various dimensions of the design space will be combined. The most promising method, eye-based head gestures, is described in more detail and was implemented for CVR. The conducted user study will be analyzed in future work.

With the adoption of mobile head mounted displays (HMDs) amongst non-experts outside of lab settings, it becomes increasingly important to understand what factors influence a holistic mobile virtual reality (MVR) user experience. We present the results of a field study (N=34), in which we used three methods - a drawing task, a storytelling exercise, and the technology acceptance questionnaire (TAM) - to explore factors, beyond technical capability, that influence the user experience of HMDs. Our analysis (1) highlights factors that designers and researchers can adopt to create and evaluate socially acceptable MVR systems for non-expert users outside a lab context, and (2) puts these factors in context with existing research from industry and academia.

This paper describes aspects which are important for camera positioning in cinematic virtual reality. For our findings, we took a closer look at proxemics, the study on how humans behave in regard to space and distances. We explored well-known shot sizes used in traditional filmmaking and put them in relation to proxemics distances. The results were adapted to camera distances in cinematic virtual reality.

Cinematic Virtual Reality has been increasing in popularity over the last years. Watching 360° movies with head mounted displays, viewers can freely choose the direction of view, and thus the visible section of the movie. In order to explore the viewers' behavior, methods are needed for collecting and analyzing data. In our experiments we compare the viewing behavior for movies with spatial and non-spatial sound and tracked the head movements of the participants. This work-in-progress describes two approaches of spatial statistics - analysis of Space Time Cubes and Getis Ord Gi* statistic - for analyzing head tracking data.

Intelligent systems, which are on their way to becoming mainstream in everyday products, make recommendations and decisions for users based on complex computations. Researchers and policy makers increasingly raise concerns regarding the lack of transparency and comprehensibility of these computations from the user perspective. Our aim is to advance existing UI guidelines for more transparency in complex real-world design scenarios involving multiple stakeholders. To this end, we contribute a stage-based participatory process for designing transparent interfaces incorporating perspectives of users, designers, and providers, which we developed and validated with a commercial intelligent fitness coach. With our work, we hope to provide guidance to practitioners and to pave the way for a pragmatic approach to transparency in intelligent systems.

Cinematic Virtual Reality has been increasing in popularity over the last years. Watching omnidirectional movies with head mounted displays, viewers can freely choose the direction of view, and thus the visible section of the movie. In order to explore the users' viewing behavior, methods are needed for collecting and analyzing data. We developed an analyzing tool, the CVR-Analyzer, which can be used for inspecting head pose and eye tracking data of viewers experiencing CVR movies. The visualized data are displayed on a flattened projection of the movie as flexibly controlled augmenting annotations, such as tracks or heatmaps, synchronously with the time code of the movie and allow inspecting and comparing the users' viewing behavior in different use cases.

We investigate instructor representations (IRs) in the context of virtual trainings with head mounted displays (HMD). Despite the recently increased industry and research focus on virtual training in immersive virtual reality (IVR), the effect of IRs on the performer (VR user) has received little attention. We present the results of a study (N=33), evaluating the effect of three IRs - webcam, avatar and sound-only - on social presence (SP) and performance (PE) of the VR user during task completion. Our results show that instructor representation has an effect on SP and that, contrary to our assumption based on prior work, it affects performance negatively.

Watching a 360° movie with Head Mounted Displays (HMDs) the viewer feels to be inside the movie and can experience it in an immersive way. The head of the viewer is exactly in the same place as the camera was when the scene was recorded. Viewing a movie by HMDs from the perspective of the camera can raise some challenges, e.g. heights of well-known objects can irritate the viewer in the case the camera height does not correspond to the physical eye height. The aim of this work is to study how the position of the camera influences presence, sickness and the user experience of the viewer. For that we considered several watching postures as well as various camera heights. The results of our experiments suggest that differences between camera and eye heights are more accepted, if the camera position is lower than the viewer's own eye height. Additionally, sitting postures are preferred and can be adapted easier than standing postures. These results can be applied to improve guidelines for 360° filmmakers.

Research has brought forth a variety of authentication systems to mitigate observation attacks. However, there is little work about shoulder surfing situations in the real world. We present the results of a user survey (N=174) in which we investigate actual stories about shoulder surfing on mobile devices from both users and observers. Our analysis indicates that shoulder surfing mainly occurs in an opportunistic, non-malicious way. It usually does not have serious consequences, but evokes negative feelings for both parties, resulting in a variety of coping strategies. Observed data was personal in most cases and ranged from information about interests and hobbies to login data and intimate details about third persons and relationships. Thus, our work contributes evidence for shoulder surfing in the real world and informs implications for the design of privacy protection mechanisms.

Cinematic Virtual Reality has been increasing in popularity in the last years. Watching 360° movies with a Head Mounted Display, the viewer can freely choose the direction of view, and thus the visible section of the movie. We explored three cinematic methods of guiding the viewers' attention: lights, sounds, and movements. For that, we developed a measurement technique to obtain heat maps of viewing directions and applied statistical analysis methods for spatial data. The results of our work show that the attention of the viewer can be directed by sound and movements. New sound induces the viewer to search for the source of the sound, not all participants paid attention to the direction of the sound. In our experiments, lights without movements did not draw more attention than other objects. However, a moving light cone changed the viewing direction considerably.

Central goal of this research project was to transfer the empirical findings about children's creative-aesthetic activities [10] to the implementation and evaluation of software. As a consequence the medial-aesthetic interests and needs of the children should be increased and their perspective empowered. In cooperation with primary school children and university students a cross-platform HTML 5 and JavaScript application named "ArtEater" has been built. This app enables open interaction processes with changing creative-aesthetic activities (like painting, drawing, integrating, exploring, staging, arranging, playing, collecting and writing) and is based on art educational requirements in a children's drawing software. Before, during and after the implementation of the web application the software has been evaluated by children after the participatory design principles [16].

We address novel two-handed interaction techniques in dual display interactive workspaces combining direct and indirect touch input. In particular, we introduce the notion of a horizontal tool space with task-dependent graphical input areas. These input areas are designed as single purpose control elements for specific functions and allow users to manipulate objects displayed on a vertical screen using simple one- and two-finger touch gestures and both hands. For demonstrating this concept, we use 3D modeling tasks as a specific application area. Initial feedback of six expert users indicates that our techniques are easy to use and stimulate exploration rather than precise modeling. Further, we gathered qualitative feedback during a multi-session observational study with five novices who learned to use our tool and were interviewed several times. Preliminary results indicate that working with our setup is easy to learn and remember. Participants liked the partitioning character of the dual-surface setup and agreed on the benefiting quality of touch input, giving them a 'hands-on feeling'.

We present experimental work which explores the effect of touch indirectness on spatial memory and navigation performance in a 2D panning task. In this regard and based on the theory of embodied cognition, prior work has observed performance increases for direct touch input over indirect mouse input. As indirect touch systems gain in importance, we designed an experiment to systematically investigate the effect of spatial indirectness while maintaining the proprioceptive and kinesthetic cues provided by touch input. In an abstract search task, participants of our study navigated a 2D space and were asked to reproduce spatial item configurations in a recall task. Our results indicate that spatial memory performance is not decreased by a spatial separation of touch input gestures and visual display. Further, our results suggest that decreasing the size of the input surface in the indirect condition increases the navigation efficiency.

We present a study that investigates the influence of different types of visualizations on collaboration. The visualizations present the group's performance either in a more cooperative or more competitive way. Decades of research suggest that cooperation leads to greater productivity than competition. However, most of the existing group mirror visualizations achieve an increase in productivity and better self-regulation by enabling a direct comparison of performance within the group. We conducted a repeated measures study with 12 groups that were supported by visualizations that displayed the number of ideas of a brainstorming session (1) per person (competitive condition) (2) per group (cooperative condition), (3) per person and per group (mixed condition) and (4) without visualization (baseline). Results indicate that groups that see a combination of individual and group performance (mixed condition) are more productive, more satisfied with their results and participate in a more balanced way.

We present an approach to protect photos on smartphones from unwanted observations by distorting them in a way that makes it hard or impossible to recognize their content for an onlooker who does not know the photographs. On the other hand, due to the chosen way of distortion, the device owners who know the original images have no problems recognizing photos. We report the results of a user study (n=18) that showed very high usability properties for all tested graphical filters (only 11 out of 216 distorted photos were not correctly identified by their owners). At the same time, two of the filters significantly reduced the observability of the image contents.

Supporting collaborative argumentation in higher education is a challenging task, and raising the awareness of students on the relational space of their collaboration may be the key to simplifying it. In this paper, we explore the role of team awareness for enhancing the quality of collaborative argumentation. Ten groups of master students from Media Informatics (n= 28) participated in a multiple-case study for arguing on ill-structured problems where they received different team awareness prompts embedded in collaboration scripts , argued for solving the problems and built argument maps. The initial analysis of the argument maps indicates that the social awareness script has a moderately higher impact on the quality of argumentation compared to the behavioural awareness script.

We present a similarity metric for Android unlock patterns to quantify the effective password space of user-defined gestures. Our metric is the first of its kind to reflect that users choose patterns based on human intuition and interest in geometric properties of the resulting shapes. Applying our metric to a dataset of 506 user-defined patterns reveals very similar shapes that only differ by simple geometric transformations such as rotation. This shrinks the effective password space by 66% and allows informed guessing attacks. Consequently, we present an approach to subtly nudge users to create more diverse patterns by showing background images and animations during pattern creation. Results from a user study (n = 496) show that applying such countermeasures can significantly increase pattern diversity. We conclude with implications for pattern choices and the design of enrollment processes.

In this paper, we present the design and evaluation of dynamic security questions for fallback authentication. In case users lose access to their device, the system asks questions about their usage behavior (e.g. calls, text messages or app usage). We performed two consecutive user studies with real users and real adversaries to identify questions that work well in the sense that they are easy to answer for the genuine user, but hard to guess for an adversary. The results show that app installations and communication are the most promising categories of questions. Using three questions from the evaluated categories was sufficient to get an accuracy of 95.5% - 100%.

We present the results of an MTurk survey (n=383) on the reasons for using and not using biometric authentication systems on smartphones. We focused on Apple's Touch ID as well as Android's Face Unlock as they are the most prevalent systems on the market. For both systems, we categorized the participants as a) current users, b) former users that deactivated it at some point and c) nonusers. The results show that usability is one of the main factors that influences the decision on whether or not to use biometric verification on the smartphone. To our surprise and as opposed to previous research on biometric authentication, privacy and trust issues were not among the most important decision factors.

In this paper, we present SwiPIN, a novel authentication system that allows input of traditional PINs using simple touch gestures like up or down and makes it secure against human observers. We present two user studies which evaluated different designs of SwiPIN and compared it against traditional PIN. The results show that SwiPIN performs adequately fast (3.7 s) to serve as an alternative input method for risky situations. Furthermore, SwiPIN is easy to use, significantly more secure against shoulder surfing attacks and switching between PIN and SwiPIN feels natural.

We performed a systematic evaluation of the shoulder surfing susceptibility of the Android pattern (un)lock. The results of an online study (n=298) enabled us to quantify the influence of pattern length, line visibility, number of knight moves, number of overlaps and number of intersections on observation resistance. The results show that all parameters have a highly significant influence, with line visibility and pattern length being most important. We discuss implications for real-world patterns and present a linear regression model that can predict the observability of a given pattern. The model can be used to provide proactive security measurements for (un)lock patterns, in analogy to password meters.

We describe three scenarios in which fallback authentication on smartphones can occur and evaluate their real-life occurrences in an online survey (n=244) and complementing interviews (n=12). The results provide first insights into frequencies, reasons, countermeasures taken and problems of lockout experiences. Overall, study participants were satisfied with current fallback schemes, but at the same time, fallback authentication was aggravated when special circumstances applied and thus, leave room for improvements. Based on this, we propose an alternative concept for fallback authentication that quizzes users about installed and not installed apps on their device. Authentication succeeds, when users identify a certain number of apps correctly. Our evaluation showed that the concept yields an overall accuracy of 95%.

In this paper, we present XSide, an authentication mechanism that uses the front and the back of smartphones to enter stroke-based passwords. Users can switch sides during input to minimize the risk of shoulder surfing. We performed a user study (n = 32) to explore how switching sides during authentication affects usability and security of the system. The results indicate that switching the sides increases security while authentication speed stays relatively fast (≤ 4 seconds). The paper furthermore provides insights on accuracy of eyes-free input (as used in XSide) and shows how 3D printed prototype cases can improve the back-of-device interaction experience.

In this paper, we present a concept using fake cursors to disguise on-screen password entry. We performed two user studies with different amounts of dummy cursors and differently colored cursors. The results show that dummy cursors significantly improve security. At the same time, decrease in performance is kept within an acceptable range. Depending on the required degree of security, the studies favor 8 or 16 differently colored cursors as the best trade-off between security and usability.

Most of today's smartphones and tablet computers feature touchscreens as the main way of interaction. By using these touchscreens, oily residues of the users' fingers, smudge, remain on the device's display. As this smudge can be used to deduce formerly entered data, authentication tokens are jeopardized. Most notably, grid-based authentication methods, like the Android pattern scheme are prone to such attacks.

In this paper, we present the results of a four-week real world study on app launches on smartphones. The results show that smartphone users are confident in the way they navigate on their devices, but that there are many opportunities for refinements. Users in our study tended to sort apps based on frequency of use, putting the most frequently used apps in places that they considered fastest to reach. Interestingly, users start most apps from within other apps, followed by the use of the homescreen.

Password patterns, as used on current Android phones, and other shape-based authentication schemes are highly usable and memorable. In terms of security, they are rather weak since the shapes are easy to steal and reproduce. In this work, we introduce an implicit authentication approach that enhances password patterns with an additional security layer, transparent to the user. In short, users are not only authenticated by the shape they input but also by the way they perform the input. We conducted two consecutive studies, a lab and a long-term study, using Android applications to collect and log data from user input on a touch screen of standard commercial smartphones. Analyses using dynamic time warping (DTW) provided first proof that it is actually possible to distinguish different users and use this information to increase security of the input while keeping the convenience for the user high.

Internet users are targets for ever-advancing phishing- and other attacks. The risks are, for example, to disclose credit card information or passwords to unauthorized instances. One approach to help users with insecure situations is provided by MoodyBoard, which uses ambient information to highlight potential risks. In this paper, we present findings from an evaluation of this system. Two user studies were conducted in order to find out whether an ambient security tool can protect users during sensitive tasks. We designed a pilot study to find out whether users understand the warnings and a security study to see if it helps to protect users from phishing attacks. Results show that MoodyBoard users behaved significantly more secure.

Automated teller machine (ATM) frauds are increasing drastically these days. When analyzing the most common attacks and the reasons for successful frauds, it becomes apparent that the main problem lies in the PIN based authentication which in itself does not provide any security features (besides the use of asterisks). That is, security is solely based on a user's behavior. Indirect input is one way to solve this problem. This mostly comes at the costs of adding overhead to the input process. We present ColorPIN, an authentication mechanism that uses indirect input to provide security enhanced PIN entry. At the same time, ColorPIN remains a one-to-one relationship between the length of the PIN and the required number of clicks. A user study showed that ColorPIN is significantly more secure than standard PIN entry while enabling good authentication speed in comparison with related systems.

The development of mobile applications has now extended from mobile network providers into the hands of ordinary people as organizations and companies encourage people to come up with their own software masterpieces by opening up APIs and tools. However, as of the moment, these APIs and tools are only usable by people with programming skills. There is a scarcity of tools that enable users without programming experience to easily build customized mobile applications. We present in this paper a tool and its underlying framework that would enable non-technical people to create their own domain-specific mobile applications. As a proof of concept, we focus on the creation of applications in the domain of mobile health monitoring. In the future, we would like to extend our work to cover other domains as well.

With the increase of automated teller machine (ATM) frauds, new authentication mechanisms are developed to overcome security problems of personal identification numbers (PIN). Those mechanisms are usually judged on speed, security, and memorability in comparison with traditional PIN entry systems. It remains unclear, however, what appropriate values for PIN-based ATM authentication actually are. We conducted a field study and two smaller follow-up studies on real-world ATM use, in order to provide both a better understanding of PIN-based ATM authentication, and on how alternative authentication methods can be compared and evaluated. Our results show that there is a big influence of contextual factors on security and performance in PIN-based ATM use. Such factors include distractions, physical hindrance, trust relationships, and memorability. From these findings, we draw several implications for the design of alternative ATM authentication systems, such as resilience to distraction and social compatibility.

Authentication in public spaces is a risky task. Frauds on cash machines (ATMs) are not uncommon nowadays. The biggest group of attacks is observation attacks, which focus on recording the input done by the users. In this work, we present VibraPass, a system created to be resilient against observation attacks using tactile feedback provided by the users' own mobile devices. In this way, secret information is shared between the terminal and the users to add an over-head of 'lies' to the input which makes it hard for attackers to steal the real PIN or password. We present an evaluation, which shows that VibraPass has the potential to replace current authentication systems due to increased security combined with reasonable input speed and error rates.

Model Driven Development (MDD) is an important para-digm in Software Engineering. In MDD, applications are specified systematically using abstract, platform independent models. The models are then transformed into executable code for different platforms and target devices. Model-driven techniques become ever more prominent in any kind of application, such as multimedia and Web, ubiquitous and automotive applications.

Usability analysis is an important step in software development in order to improve certain aspects of the system. However, it is often a challenge especially when it comes to evaluating applications running on mobile devices because of the restrictions posed by the device and the lack of supporting tools and software available to collect the necessary usability data. This paper proposes a methodology and framework to aid developers in preparing the mobile system for usability analysis. The focus is on the simplification of the developer's task in preparing the system for evaluation and the processing of the collected usability data by automating some of the tasks involved in the process.

Authentication systems for public terminals and thus public spaces have to be fast, easy and secure. Security is of utmost importance since the public setting allows manifold attacks from simple shoulder surfing to advanced manipulations of the terminals. In this work, we present EyePassShapes, an eye tracking authentication method that has been designed to meet these requirements. Instead of using standard eye tracking input methods that require precise and expensive eye trackers, EyePassShapes uses eye gestures. This input method works well with data about the relative eye movement, which is much easier to detect than the precise position of the user's gaze and works with cheaper hardware. Different evaluations on technical aspects, usability, security and memorability show that EyePassShapes can significantly increase security while being easy to use and fast at the same time.

The initial vision associated with multimedia is to achieve user interfaces (UI) which are highly interactive, intelligent, usable, and entertaining. Although early multimedia applications did not always fulfill these expectations, today's applications come much closer towards this vision. Examples are Rich Internet Applications like Google Maps, Edutainment Systems, or new UIs in the context of Ubiquitous Computing. The key difference is the increased dynamics of the UI (media objects are created/modified depending to the current context) and the tighter integration of UI elements with the application logic. Although this dynamism allows new forms of applications, it is problematic when engineering such applications: Existing multimedia modeling approaches focus mainly on the spatio-temporal integration of media objects. However, they hardly allow modeling the integration with the application logic or generic media objects. Hence, many of these concepts cannot be handled. To address these challenges, this paper first illustrates the problem with examples from common applications. Subsequently, we propose a solution how to integrate these concepts into a consistent modeling language. The discussed issues can be generalized from the presented language and integrated into other approaches, e.g., modeling languages for Rich Internet Applications.

This research aims to simplify the creation of applications for mobile platforms by developing a high-level and platform independent model of an application, and automatically transforming this high-level model to platform specific code. The research method is a combination of the model-driven development (MDD) approach in software development and application of techniques in the field of human-computer interaction (HCI) particularly on user- centered system design. This research involves developing a graphical modeling language which is specific to mobile applications and coming up with a generic algorithm for the conversion of this graphical model into code. The main focus of the research however, is on the design of the graphical model, and the interaction techniques which will allow non-expert people to create specialized mobile applications with case.

The design of applications using mobile devices needs a different quality assessment than those known for desktop applications. Of the many aspects that have to be taken into account, one important criterion is the average time users need to complete a task. For interactions with the mouse, keyboard or touch screens, there exist models that predict interaction times like Fitts' law or the Keystroke-Level Model (KLM). This paper shows parallels to these models for advanced interactions with mobile phones targeted at pervasive services, including near field communication as well as built-in cameras and sensors. Applications can be evaluated with respect to user performance time without having a prototype running on the phone. To accomplish that, we extend the known KLM by identifying basic interaction elements for mobile phones and give estimates for expert user performance derived from several user tests.

We report on the experience we gained in different software engineering project courses at our department. A student who is specialized in software engineering has to complete, besides lectures on software engineering and related fields, two project courses: an "early" basic project course, and a "late" complex project course. For both of them we developed leveled teaching approaches considering the different knowledge and skills of beginners and "advanced" students, respectively. The paper describes, discusses and evaluates these approaches.

We report on experience from teaching OO technology to undergraduate students. Before they can successfully tackle the projects they have to successfully shift to the OO paradigm, pick up a working knowledge of some OO language, learn and practice OOA and OOD, and get used to advanced ideas like patterns and frameworks. In order to relieve this heavy burden somewhat, we provided an object-oriented application framework as a common base for the projects. That way, the students are given an architecture which they have to adapt to their specific task instead of doing all the design by themselves. We also believe that this policy closely resembles the way beginners are integrated into on-going projects in practice. We describe the Java framework we used, the preparations for and the organization of the project course, educators' and students' experience and some ideas for developing this approach further.