A new generation of GenAI tools fueled by vibe coding practices promises to democratize software development, explicitly targeting users without programming backgrounds. Yet, we lack understanding of how technical and non-technical users actually engage with these tools across the product development lifecycle. We conducted a mixed-methods study combining an online survey (N = 85) with interviews of hackathon participants (N = 31) and practitioners (N = 8), examining how different user groups employ chatbots, local development assistants, and cloud development environments from ideation through deployment. Our findings reveal that cloud development environments accelerate prototyping, enabling non-technical users to generate high-fidelity "throw-away" prototypes valuable for experiential exploration. However, deployment and long-term maintainability remain dependent on technical expertise, with non-technical users consistently encountering barriers when transitioning beyond prototyping. We contribute a comparative analysis of how technical and non-technical users appropriate GenAI tools across the full product development cycle in contexts approximating real-world product building, highlighting implications for tool design and educational practices.

In what ways do generative AI (GenAI) tools embedded in mainstream design software affect UI design practices and outcomes? In this study, we examine the use of FigmaAI, a GenAI feature within the industry-standard platform Figma. We conducted a within-subject study with 16 professional UX designers, each of whom completed two high-fidelity UI design tasks in a think-aloud session: one conventionally, without AI, and one with FigmaAI. We analyse the design process itself alongside experiences and reflections on using and not using AI, gathered from adjacent semi-structured interviews and post-task questionnaires. Our findings suggest that GenAI reshapes UI workflows, shifting them from additive to subtractive processes: designers refine AI drafts rather than building interfaces from scratch. While AI reduces workload and accelerates initial setup, it also constrains exploration, limits perceived ownership, and produces designs that are more visually and structurally similar.

Knowing what collaborators attend to is essential. Previous studies demonstrated that shared gaze enhances coordination and social connectedness in remote settings. In collocated settings, gaze can be both naturally observable and technologically augmented. AR enables gaze cues to be rendered explicitly in the environment. To investigate if and how such cues are beneficial in collocated AR collaboration, we examined both qualitative and quantitative effects across three task types (puzzle, negotiation, search) and two spatial setups (plane, room), focusing on task completion time and the collaborative experience. In our user study with 24 dyads (n=48), we varied gaze visibility and measured task performance, user preference, social connectedness, and shared attention. Our results show that sharing gaze in collocated collaborative AR can increase shared attention, is perceived as helpful, and improves the user experience, similar to remote collaboration, but has a limited impact on the actual task completion time across the chosen tasks.

Conventional navigation systems, fixated on metrics such as time and distance, neglect the driver’s emotional well-being, despite driving routes being inherent emotional triggers. This raises a critical question for the Intelligent User Interface community: How can intelligent systems successfully route information based on emotion? To address this gap, we introduce HappyRouting, an empathic car interface designed as an initial attempt to guide drivers through real-world traffic while actively optimizing for positive emotional states. Our core technical contribution is a machine learning-based emotion map layer that predicts the affective valence along various routes using both static and dynamic contextual data. HappyRouting enables the generation of “happy routes” integrated into a functional vehicular interface prototype. We explored the efficacy of this approach in a preliminary, small-scale driving study (N = 13). Our initial findings provide provocative evidence: Emotion-optimized routes successfully increased the subjectively perceived valence by 11% (p =.007) compared to standard routes. Furthermore, despite taking 1.25 times longer on average, participants consistently perceived the travel duration as shorter. This result suggests that integrating emotional optimization could fundamentally challenge the speed-first paradigm. However, recognizing the constraints of our initial, limited sample, we conclude by discussing ethical and computational challenges that must be resolved before emotion-based routing can be safely and scalably integrated into next-generation intelligent navigation apps.

We explore the metaphorical "daily memory pill"concept - a brief pictorial lifelog recap aimed at reviving and preserving memories. Leveraging psychological strategies, we explore the potential of such summaries to boost autobiographical memory. We developed an automated lifelogging memory prosthesis and a research protocol (Automated Memory Validation "AMV") for conducting privacy-aware, in-situ evaluations. We conducted a real-world lifelogging experiment for a month (n=11). We also designed a browser "Pixel Memories"for browsing one-week worth of lifelogs. The results suggest that daily timelapse summaries, while not yielding significant memory augmentation effects, also do not lead to memory degradation. Participants' confidence in recalled content remains unaltered, but the study highlights the challenge of users' overestimation of memory accuracy. Our core contributions, the AMV protocol and "Pixel Memories"browser, advance our understanding of memory augmentations and offer a privacy-preserving method for evaluating future ubicomp systems.

We evaluated the viability of using Large Language Models (LLMs) to trigger and personalize content in Just-in-Time Adaptive Interventions (JITAIs) in digital health. As an interaction pattern representative of context-aware computing, JITAIs are being explored for their potential to support sustainable behavior change, adapting interventions to an individual's current context and needs. Challenging traditional JITAI implementation models, which face severe scalability and flexibility limitations, we tested GPT-4 for suggesting JITAIs in the use case of heart-healthy activity in cardiac rehabilitation. Using three personas representing patients affected by CVD with varying severeness and five context sets per persona, we generated 450 JITAI decisions and messages. These were systematically evaluated against those created by 10 laypersons (LayPs) and 10 healthcare professionals (HCPs). GPT-4-generated JITAIs surpassed human-generated intervention suggestions, outperforming both LayPs and HCPs across all metrics (i.e., appropriateness, engagement, effectiveness, and professionalism). These results highlight the potential of LLMs to enhance JITAI implementations in personalized health interventions, demonstrating how generative AI could revolutionize context-aware computing.

There seems to be a solid object inside.""What other toppings do you usually like on your pizza?" Figure 1: We imbued a robot with curious behaviors.The figure shows two examples.Left: The robot shakes a container to check whether there is an object inside.Right: The robot asks for the person's preferences.

Figure 1: The two-phase process for eliciting and verifying gestures.

Transferring knowledge across generations is fundamental to human civilization, yet the challenge of passing on complex practical skills persists. Methods without a physically present instructor, such as videos, often fail to explain complex manual tasks, where spatial and social factors are critical. Technologies such as eXtended Reality and Artificial Intelligence hold the potential to retain expert knowledge and facilitate the creation of tailored, contextualized, and asynchronous explanations regardless of time and place. In contrast to videos, the learner's perspective can be different from the recorded perspective in XR. This paper investigates the impact of asynchronous first- and third-person perspectives and gaze visualizations on efficiency, feeling of embodiment, and connectedness during manual tasks. The empirical results of our study ($\mathrm{N}=36$) show that the first-person perspective is better in quantitative measures and preferred by users. We identify best practices for presenting preserved knowledge and provide guidelines for designing future systems.

As autonomous agents, from self-driving cars to virtual assistants, become increasingly present in everyday life, safe and effective collaboration depends on human understanding of agents’ intentions. Current intent communication approaches are often rigid, agent-specific, and narrowly scoped, limiting their adaptability across tasks, environments, and user preferences. A key gap remains: existing models of what to communicate are rarely linked to systematic choices of how and when to communicate, preventing the development of generalizable, multi-modal strategies. In this paper, we introduce a multidimensional design space for intent communication structured along three dimensions: Transparency (what is communicated), Abstraction (when), and Modality (how). We apply this design space to three distinct human-agent collaboration scenarios: (a) bystander interaction, (b) cooperative tasks, and (c) shared control, demonstrating its capacity to generate adaptable, scalable, and cross-domain communication strategies. By bridging the gap between intent content and communication implementation, our design space provides a foundation for designing safer, more intuitive, and more transferable agent-human interactions.

Technology increasingly shapes our social interactions, both online and in person. Strong social connections and face-to-face interactions are vital for wellbeing, especially with close relationships. In this context, technology can play an ambivalent role: whereas it often has a negative impact on the quality of these interactions, it carries potential to enrich conversations and improve social interactions if used in a meaningful way. We design a prototype that implements subtle intervention strategies to foster meaningful technology use, specifically aimed at enhancing close relationships during in-person interactions. We evaluate the prototype within an exploratory, two-week in-the-wild user study with 6 tandems ( N =12). Our findings suggest that the strategy of "us-reflection" – a social approach to reflection – contributes to mutual awareness of participants’ shared time. Our prototype encouraged more meaningful interactions by proposing conversation topics or suggesting activities, ultimately strengthening close relationships and fostering more intentional, engaging, and rewarding social experiences.

Smartphone overuse is hyper-prevalent in society, and developing tools to prevent this overuse has become a focus of HCI. However, there is a lack of work investigating smartphone overuse interventions over the long term. We collected usage data from N = 1, 039 users of one sec over an average of 13.4 weeks and qualitative insights from 249 of the users through an online survey. We found that users overwhelmingly choose to target Social Media apps. We found that the short design frictions introduced by one sec effectively reduce how often users attempt to open target apps and lead to more intentional app-openings over time. Additionally, we found that users take periodic breaks from one sec interventions, and quickly rebound from a pattern of overuse when returning from breaks. Overall, we contribute findings from a longitudinal investigation of design frictions in the wild and identify usage patterns from real users in practice.

Large language models (LLMs) like ChatGPT have been widely adopted in work contexts. We explore the impact of ChatGPT on young professionals’ perception of productivity and sense of accomplishment. We collected LLMs’ main use cases in knowledge work through a preliminary study, which served as the basis for a two-week diary study with 21 young professionals reflecting on their ChatGPT use. Findings indicate that ChatGPT enhanced some participants’ perceptions of productivity and accomplishment by enabling greater creative output and satisfaction from efficient tool utilization. Others experienced decreased perceived productivity and accomplishment, driven by a diminished sense of ownership, perceived lack of challenge, and mediocre results. We found that the suitability of task delegation to ChatGPT varies strongly depending on the task nature. It’s especially suitable for comprehending broad subject domains, generating creative solutions, and uncovering new information. It’s less suitable for research tasks due to hallucinations, which necessitate extensive validation.

Social interaction is a crucial part of what it means to be human. Maintaining a healthy social life is strongly tied to positive outcomes for both physical and mental health. While we use personal informatics data to reflect on many aspects of our lives, technology-supported reflection for social interactions is currently under-explored. To address this, we first conducted an online survey (N=124) to understand how users want to be supported in their social interactions. Based on this, we designed and developed an app for users to track and reflect on their social interactions and deployed it in the wild for two weeks (N=25). Our results show that users are interested in tracking meaningful in-person interactions that are currently untraced and that an app can effectively support self-reflection on social interaction frequency and social load. We contribute insights and concrete design recommendations for technology-supported reflection for social interaction.

The use of AI is becoming more common among the population every day; the use of generative AI, such as LLMs, empowers individuals by supporting daily life tasks. Yet, the user interaction with AI models is mostly constrained to chatbot interactions. However, we envision that in the near future, individuals will be able to integrate the use of these technologies into their daily activities without refocusing their attention. Consequently, we explores the impact of such integration on individuals’ conversations.

Large language models like GPT and Codex drastically alter many daily tasks, including programming, where they can rapidly generate code from natural language or informal specifications. Thus, they will change what it means to be a programmer and how programmers act during software development. This work explores how AI assistance for code generation impacts productivity. In our user study (N=24), we asked programmers to complete Python programming tasks supported by a) an auto-complete interface using GitHub Copilot, b) a conversational system using GPT-3, and c) traditionally with just the web browser. Aside from significantly increasing productivity metrics, participants displayed distinctive usage patterns and strategies, highlighting that the form of presentation and interaction affects how users engage with these systems. Our findings emphasize the benefits of AI-assisted coding and highlight the different design challenges for these systems.

Human-AI interaction in text production increases complexity in authorship. In two empirical studies (n1 = 30 & n2 = 96), we investigate authorship and ownership in human-AI collaboration for personalized language generation. We show an AI Ghostwriter Effect : Users do not consider themselves the owners and authors of AI-generated text but refrain from publicly declaring AI authorship. Personalization of AI-generated texts did not impact the AI Ghostwriter Effect , and higher levels of participants’ influence on texts increased their sense of ownership. Participants were more likely to attribute ownership to supposedly human ghostwriters than AI ghostwriters, resulting in a higher ownership-authorship discrepancy for human ghostwriters. Rationalizations for authorship in AI ghostwriters and human ghostwriters were similar. We discuss how our findings relate to psychological ownership and human-AI interaction to lay the foundations for adapting authorship frameworks and user interfaces in AI in text-generation tasks.

Artificial intelligence has enabled scalable auto-creation of context-aware personalized learning materials. However, it remains unclear how content personalization shapes the learners’ experience. We developed one personalized and two non-personalized, crowdsourced versions of a mobile language learning app: (1) with personalized auto-generated photo flashcards, (2) the same flashcards provided through crowdsourcing, and (3) manually generated flashcards based on the same photos. A two-week in-situ study (n = 64) showed that learners assessed the quality of the non-personalized auto-generated material to be on par with manually generated material, which means that auto-generation is viable. However, when the auto-generation was personalized, the learners’ quality rating was significantly lower. Further analyses suggest that aspects such as prior expectations and required efforts must be addressed before learners can actually benefit from context-aware personalization with auto-generated material. We discuss design implications and provide an outlook on the role of content personalization in AI-supported learning.

Allowing users of interactive systems to reflect on their task proficiency is often incidental. This is unfortunate, as communicating meaningful task-related proficiency feedback could improve users’ awareness of their abilities and their willingness to improve. To highlight the feasibility of this concept, we evaluated how different methods of readability feedback impacted users during a text production task. In general, our results showed that having access to readability feedback allowed participants to reflect on their task solving approach, facilitating the users’ understanding of their proficiency. Revision-based methods are less distracting for the user than continuous feedback methods, while still offering high efficacy. Further, feedback should be paired with a subtle form of gamification elements. We envision this reflection-oriented design to user proficiency to be applicable to a variety of interactive systems, allowing for an improved and engaging user experience.

Mastering skills which involve high dexterity, such as playing the piano, requires extensive guidance through personal teaching. Understanding how we can leverage data from sensor-based systems to improve the learning process, allows us to build interactive systems which effectively facilitate skill acquisition. To explore such possibilities, we developed EyePiano—a gaze-assisted tool for reflective piano playing. EyePiano guides the practice process of learning piano scores through analyzing the pianist’s gaze behavior. We based the design of EyePiano on requirements identified through interviews with piano teachers and a feasibility evaluation of gaze metrics. Our system illustrates that basic gaze metrics are sufficient to predict difficult regions for students. Thus, highlighting sections of the music piece which are particularly difficult for the pianist allows EyePiano to support piano rehearsals for students. Our work showcases the feasibility of using gaze data for reflective music education, enabling effective instrument practice.

Learning to read Braille is crucial to academic success for people with blindness or severe visual impairment. In our work, we investigate how we can support early learning of Braille with tangible computing. In a human-centered inclusive design process with interviews, six design iterations with prototypes, and feedback from experts, students, and teachers, we created BrailleBuddy. BrailleBuddy is a tangible user interface supporting children with visual impairments in learning Braille. We evaluated BrailleBuddy in a user study with children with blindness. Our results show that BrailleBuddy provides intrinsic motivation for learning Braille and can be used by children without supervision. BrailleBuddy complements the educational program as it allows children to play with and explore Braille characters at their own pace, thus lowering the challenge of learning to read Braille. In addition, an open-source toolkit is provided to enable educators and researchers to support individual requirements.

When we go for a walk with friends, we can observe an interesting effect: From step lengths to arm movements - our movements unconsciously align; they synchronize. Prior research found that this synchronization is a crucial aspect of human relations that strengthens social cohesion and trust. Generalizing from these findings in synchronization theory, we propose a dynamical approach that can be applied in the design of non-humanoid robots to increase trust. We contribute the results of a controlled experiment with 51 participants exploring our concept in a between-subjects design. For this, we built a prototype of a simple non-humanoid robot that can bend to follow human movements and vary the movement synchronization patterns. We found that synchronized movements lead to significantly higher ratings in an established questionnaire on trust between people and automation but did not influence the willingness to spend money in a trust game.

Social media platforms use short, highly engaging videos to catch users’ attention. While the short-form video feeds popularized by TikTok are rapidly spreading to other platforms, we do not yet understand their impact on cognitive functions. We conducted a between-subjects experiment (N = 60) investigating the impact of engaging with TikTok, Twitter, and YouTube while performing a Prospective Memory task (i.e., executing a previously planned action). The study required participants to remember intentions over interruptions. We found that the TikTok condition significantly degraded the users’ performance in this task. As none of the other conditions (Twitter, YouTube, no activity) had a similar effect, our results indicate that the combination of short videos and rapid context-switching impairs intention recall and execution. We contribute a quantified understanding of the effect of social media feed format on Prospective Memory and outline consequences for media technology designers to not harm the users’ memory and wellbeing.

Lifelogging is traditionally used for memory augmentation. However, recent research shows that users’ trust in the completeness and accuracy of lifelogs might skew their memories. Privacy-protection alterations such as body blurring and content deletion are commonly applied to photos to circumvent capturing sensitive information. However, their impact on how users remember memories remain unclear. To this end, we conduct a white-hat memory attack and report on an iterative experiment (N=21) to compare the impact of viewing 1) unaltered lifelogs, 2) blurred lifelogs, and 3) a subset of the lifelogs after deleting private ones, on confidently remembering memories. Findings indicate that all the privacy methods impact memories’ quality similarly and that users tend to change their answers in recognition more than recall scenarios. Results also show that users have high confidence in their remembered content across all privacy methods. Our work raises awareness about the mindful designing of technological interventions.

While walking meetings offer a healthy alternative to sit-down meetings, they also pose practical challenges. Taking notes is difficult while walking, which limits the potential of walking meetings. To address this, we designed the Walking Talking Stick—a tangible device with integrated voice recording, transcription, and a physical highlighting button to facilitate note-taking during walking meetings. We investigated our system in a three-condition between-subjects user study with thirty pairs of participants (N=60) who conducted 15-minute outdoor walking meetings. Participants either used clip-on microphones, the prototype without the button, or the prototype with the highlighting button. We found that the tangible device increased task focus, and the physical highlighting button facilitated turn-taking and resulted in more useful notes. Our work demonstrates how interactive artifacts can incentivize users to hold meetings in motion and enhance conversation dynamics. We contribute insights for future systems which support conducting work tasks in mobile environments.

Technologies that help users overcome their limitations and integrate with the human body are often termed “human augmentations”. Such technologies are now available on the consumer market, potentially supporting people in their everyday activities. To date, there is no systematic understanding of the perception of human augmentations yet. To address this gap and build an understanding of how to design positive experiences with human augmentations, we conducted a mixed-method study of the perception of augmented humans (AHs). We conducted two scenario-based studies: interviews (n = 16) and an online study (n = 506) with participants from four countries. The scenarios include one out of three augmentation categories (sensory, motor, and cognitive) and specify if the augmented person has a disability or not. Overall, results show that the type of augmentation and disability impacted user attitudes towards AHs. We derive design dimensions for creating technological augmentations for a diverse and global audience.

Providing users with a haptic sensation of the hardness and softness of objects in virtual reality is an open challenge. While physical props and haptic devices help, their haptic properties do not allow for dynamic adjustments. To overcome this limitation, we present a novel technique for changing the perceived stiffness of objects based on a visuo-haptic illusion. We achieved this by manipulating the hands’ Control-to-Display (C/D) ratio in virtual reality while pressing down on an object with fixed stiffness. In the first study (N=12), we determine the detection thresholds of the illusion. Our results show that we can exploit a C/D ratio from 0.7 to 3.5 without user detection. In the second study (N=12), we analyze the illusion’s impact on the perceived stiffness. Our results show that participants perceive the objects to be up to 28.1% softer and 8.9% stiffer, allowing for various haptic applications in virtual reality.

Most smart home devices have multiple sensors, such as cameras and microphones; however, most cannot be controlled individually. Tangible privacy mechanisms provide control over individual sensors and instill high certainty of privacy. Yet, it remains unclear how they can be used in future smart homes. We conducted three studies to understand how tangible privacy mechanisms scale across multiple devices and respond to user needs. First, we conducted a focus group (N=8) on speculative tangible control artifacts to understand the user perspective. Second, we ran a workshop at a human-computer interaction conference (N=8) on tangible privacy. Third, we conducted a six-week in-the-wild study with a tangible, static privacy dashboard across six households. Our findings help to contrast the need for tangible privacy mechanisms on the sensor level with user needs on a smart home level. Finally, we discuss our design implications for future smart homes through the lens of inclusive privacy.

We are constantly surrounded by technology that collects and processes sensitive data, paving the way for privacy violations. Yet, current research investigating technology-facilitated privacy violations in the physical world is scattered and focused on specific scenarios or investigates such violations purely from an expert’s perspective. Informed through a large-scale online survey, we first construct a scenario taxonomy based on user-experienced privacy violations in the physical world through technology. We then validate our taxonomy and establish mitigation strategies using interviews and co-design sessions with privacy and security experts. In summary, this work contributes (1) a refined scenario taxonomy for technology-facilitated privacy violations in the physical world, (2) an understanding of how privacy violations manifest in the physical world, (3) a decision tree on how to inform users, and (4) a design space to create notices whenever adequate. With this, we contribute a conceptual framework to enable a privacy-preserving technology-connected world.

Many modern household appliances are challenging to operate for people with visual impairment. Low-contrast designs and insufficient tactile feedback make it difficult to distinguish interface elements and to recognize their function. Augmented reality (AR) can be used to visually highlight such elements and provide assistance to people with residual vision. To realize this goal, we (1) created a dataset consisting of 13,702 images of interfaces from household appliances and manually labeled control elements; (2) trained a neural network to recognize control elements and to distinguish between PushButton, TouchButton, Knob, Slider, and Toggle; and (3) designed various contrast-rich and visually simple AR augmentations for these elements. The results were implemented as a screen-based assistive AR application, which we tested in a user study with six individuals with visual impairment. Participants were able to recognize control elements that were imperceptible without the assistive application. The approach was well received, especially for the potential of familiarizing oneself with novel devices. The automatic parsing and augmentation of interfaces provide an important step toward the independent interaction of people with visual impairments with their everyday environment.

Communication is crucial for interpersonal connection, but sometimes we simply cannot find the right words. Some data, such as complex emotions, are either hard to quantify or are otherwise difficult to communicate. We have access to numerous personal statistics from quantified self devices, but hidden data are either untracked or require abstraction. In this paper, we explore physicalizations to communicate hidden data between couples. We recruited six couples (N=12 participants, 163 telegram responses) to participate in a two-week sensitization diary study followed by two participatory co-design sessions. We then hosted a one-day expert prototyping workshop (N=5) to create tangible artifacts based on the findings of the participatory phase. By iterating on the topic in three ways, we contribute (i) a design framework for understanding and tangibly representing hidden data, (ii) a discussion on the appropriateness of these methodologies, and (iii) open research questions to guide future research in the field.

Smart home devices with their sophisticated sensing technologies raise many privacy concerns. In most cases, they only function when fully connected to the internet in which case privacy exposure is greatest. Users currently have to either accept these privacy risks or remove devices from the internet or power plug, rendering them useless. Our demo is based on recent work advocating for advanced smart device configuration options across a spectrum of connectivity control options. We introduce a model ecosystem with four connectivity levels and a privacy label that informs about connectivity-feature trade-offs across those four modes. The presented ecosystem introduces two functional smart devices and demonstrates intuitively how configuration decisions impact information flow.

Technologies have become an integral part of physical activity. Yet, the majority of popular programs do not focus on promoting a genuine understanding of how sport affects our bodies. As apps and trackers persuade users to exercise more, lack of body awareness can be detrimental to health. In this work, we propose and evaluate the concept of in-session reflective feedback as a means to support informed exercise routines by design. We designed and implemented REPLAY, a system which presents users with a visualization of physiological signals (heart rate, movement) from body-worn sensors during high-intensity interval training (HIIT). Our evaluation showed that participants gained a better understanding of how their body reacted to physical activity, allowing them to understand its effect and recognize own weaknesses. Further, our work demonstrates how the type of feedback can significantly moderate a user's perceived exhaustion. We highlight how in-session reflective feedback using bodily signals can promote healthy and effective workouts through creating a deeper understanding of one's own body physiology and limits.

Exploring the great outdoors offers physical and mental health benefits. Hiking is healthy, provides a sense of accomplishment, and offers an opportunity to relax. However, a nature trip is not always possible, and there is a lack of evidence showing how these beneficial experiences can be replicated in Virtual Reality (VR). In response, we recruited (N=24) participants to explore a virtual mountain landscape in a within-subjects study with different levels of exertion: walking, using a chairlift, and teleporting. We found that physical exertion when walking produced significantly more positive emotions and mindfulness than other conditions. Our research shows that physically demanding outdoor activities in VR can be beneficial for the user and that the achievement of hiking up a virtual mountain on a treadmill positively impacts wellbeing. We demonstrate how physical exertion can be used to add mindfulness and positive affect to VR experiences and discuss consequences for VR designers.

We present a systematic literature review of cryptocurrency and blockchain research in Human-Computer Interaction (HCI) published between 2014 and 2021. We aim to provide an overview of the field, consolidate existing knowledge, and chart paths for future research. Our analysis of 99 articles identifies six major themes: (1) the role of trust, (2) understanding motivation, risk, and perception of cryptocurrencies, (3) cryptocurrency wallets, (4) engaging users with blockchain, (5) using blockchain for application-specific use cases, and (6) support tools for blockchain. We discuss the focus of the existing research body and juxtapose it to the changing landscape of emerging blockchain technologies to highlight future research avenues for HCI and interaction design. With this review, we identify key aspects where interaction design is critical for the adoption of blockchain systems. Doing so, we provide a starting point for new scholars and designers and help them position future contributions.

The voluntary carbon market is an important building block in the fight against climate change. However, it is not trivial for consumers to verify whether carbon offset projects deliver what they promise. While technical solutions for measuring their impact are emerging, there is a lack of understanding of how to translate carbon offset data into trustworthy interface designs. With interaction between users and offset projects mainly happening online, it is critical to meet this design challenge. To this end, we designed and evaluated interfaces with varying trust cues for carbon offset projects in a randomized online experiment (n=244). Our results show that content design, particularly financial and forest-related quantitative data presented at the right detail level, increases the perceived trustworthiness, while images have no significant effect. We contribute the first specific guidance for interface designers for carbon offsets and discuss implications for interaction design.

We present MindPhone, a mindfulness-based intervention to tackle absentminded and excessive smartphone use. At unlock, MindPhone prompts one of two questions: what the user intends to do with the smartphone, or what the user intends to do in the real world after using their smartphone. Users may respond actively by writing, or passively by mentally reflecting. We evaluated the effectiveness of the two questions and two response modes in a mixed-method, 2x2 mixed field study with 28 participants over two weeks. Our results show that the real-world prompt significantly reduces absentminded use and encourages a quicker return to the real world, independent of the response mode. Asking about smartphone use intentions raises awareness of reasons for smartphone use. For everyday use of MindPhone, users wish to set the question and response mode based on context. We discuss including awareness of the physical world in future smartphone use interventions.

While traditional interfaces in human-computer interaction mainly rely on vision and audio, haptics becomes more and more important. Haptics cannot only increase the user experience and make technology more immersive, it can also transmit information that is hard to interpret only through vision and audio, such as the softness of a surface or other material properties. In this workshop, we aim at discussing how we could interact with technology if haptics is strongly supported and which novel research areas could emerge.

Modern games make creative use of First- and Third-person perspectives (FPP and TPP) to allow the player to explore virtual worlds. Traditionally, FPP and TPP perspectives are seen as distinct concepts. Yet, Virtual Reality (VR) allows for flexibility in choosing perspectives. We introduce the notion of a perspective continuum in VR, which is technically related to the camera position and conceptually to how users perceive their environment in VR. A perspective continuum enables adapting and manipulating the sense of agency and involvement in the virtual world. This flexibility of perspectives broadens the design space of VR experiences through deliberately manipulating perception. In a study, we explore users’ attitudes, experiences and perceptions while controlling a virtual character from the two known perspectives. Statistical analysis of the empirical results shows the existence of a perspective continuum in VR. Our findings can be used to design experiences based on shifts of perception.

While systems that use Artificial Intelligence (AI) are increasingly becoming part of everyday technology use, we do not fully understand how AI changes design processes. A structured understanding of how designers work with AI is needed to improve the design process and educate future designers. To that end, we conducted interviews with designers who participated in projects which used AI. While past work focused on AI systems created by experienced designers, we focus on the perspectives of a diverse sample of interaction designers. Our results show that the design process of an interactive system is affected when AI is integrated and that design teams adapt their processes to accommodate AI. Based on our data, we contribute four approaches adopted by interaction designers working with AI: a priori, post-hoc, model-centric, and competence-centric. Our work contributes a pragmatic account of how design processes for AI systems are enacted.

Users avoid engaging with privacy policies because they are lengthy and complex, making it challenging to retrieve relevant information. In response, research proposed contextual privacy policies (CPPs) that embed relevant privacy information directly into their affiliated contexts. To date, CPPs are limited to concept showcases. This work evolves CPPs into a production tool that automatically extracts and displays concise policy information. We first evaluated the technical functionality on the US’s 500 most visited websites with 59 participants. Based on our results, we further revised the tool to deploy it in the wild with 11 participants over ten days. We found that our tool is effective at embedding CPP information on websites. Moreover, we found that the tool’s usage led to more reflective privacy behavior, making CPPs powerful in helping users understand the consequences of their online activities. We contribute design implications around CPP presentation to inform future systems design.

Deep learning based electroencephalography (EEG) signal processing methods are known to suffer from poor test-time generalization due to the changes in data distribution. This becomes a more challenging problem when privacy-preserving representation learning is of interest such as in clinical settings. To that end, we propose a multi-source learning architecture where we extract domain-invariant representations from dataset-specific private encoders. Our model utilizes a maximum-mean-discrepancy (MMD) based domain alignment approach to impose domain-invariance for encoded representations, which outperforms state-of-the-art approaches in EEG-based emotion classification. Furthermore, representations learned in our pipeline preserve domain privacy as dataset-specific private encoding alleviates the need for conventional, centralized EEG-based deep neural network training approaches with shared parameters.

The ubiquity of mobile devices has fueled the popularity of microlearning, namely informal self-directed learning during brief personal downtime. However, learner engagement is challenging to maintain, and microlearning habits are hard to establish. Scheduled reminders are ineffective as they do not match the users’ variable schedules and their intention or capacity to engage. In this paper, we propose a schedule-based and an activity-based trigger for microlearning. The first trigger is sensitive to the learners’ agenda and device status and includes a snooze mechanism. A four-week study (n=10) showed slightly lower response times when compared to triggers scheduled at a fixed time but did not improve learner engagement. The second trigger initiates audio-based microlearning when plugging in headphones. Thus, we minimize the access to personal data and capture a moment where learners engage with their device for a listening activity. In an exploratory user study (n=10), the plugin trigger achieved high compliance rates and was less likely to induce annoyance in users than lock screen notifications. We conclude that intelligent reminders with simple interaction options can contribute to learner engagement.

Books can be a valuable resource for language learners, providing entertainment and showcasing authentic language usage. To further support learners, e-book platforms already include interactive vocabulary aids. Similarly, research has started to investigate the feasibility and usability of grammar aids. However, grammar aids can hinder reading flow. In this paper, we design an interactive e-reading interface with different levels of grammar support. We perform a within-subject user study (n = 24) where we assess the relationship between the reading flow, usability, and usefulness with these designs in a fiction reading scenario. Our findings show that more detailed designs are considered more useful, but also more disruptive and less usable than simpler designs. Hence, grammar support interfaces need to balance the focus on learning and the reading experience.

Absent-minded smartphone use has become a catalyst for digital stress. It is often triggered by the mere presence of the device itself, inciting physical opt-out solutions with potentially severe drawbacks. Consequently, there is a need to explore the design dimensions and entailing limitations of physical opt-out methods, especially for everyday use. Informed through expert interviews and online questionnaires, we designed a proof of concept prototype that creates a physical barrier between users and their smartphones while maintaining ease of access. We evaluated our prototype concept in a two-week in-the-wild study, surfacing known benefits of physical opt-out strategies, such as encouraging reflection and supporting deliberate breaks. Although we reduced absent-minded use for users with a high predisposition, drawbacks such as Fear of Missing Out (FoMO) remain. Our work highlights that finding the appropriate amount of physical separation is essential when designing opt-out strategies for digital wellbeing.

There is a growing need for sparse representational formats of human affective states that can be utilized in scenarios with limited computational memory resources. We explore whether representing neural data, in response to emotional stimuli, in a latent vector space can serve to both predict emotional states as well as generate synthetic EEG data that are participant-and/or emotion-specific. We propose a conditional variational autoencoder based framework, EEG2Vec, to learn generative-discriminative representations from EEG data. Experimental results on affective EEG recording datasets demonstrate that our model is suitable for unsupervised EEG modeling, classification of three distinct emotion categories (positive, neutral, negative) based on the latent representation achieves a robust performance of 68.49%, and generated synthetic EEG sequences resemble real EEG data inputs to particularly reconstruct low-frequency signal components. Our work advances areas where affective EEG representations can be useful in e.g., generating artificial (labeled) training data or alleviating manual feature extraction, and provide efficiency for memory constrained edge computing applications.

Exploring and interacting with electronics is challenging as the internal processes of components are not visible. Further barriers to engagement with electronics include fear of injury and hardware damage. In response, Augmented Reality (AR) applications address those challenges to make internal processes and the functionality of circuits visible. However, current apps are either limited to abstract low-fidelity applications or entirely virtual environments. We present ElectronicsAR, a tangible high-fidelity AR electronics kit with scaled hardware components representing the shape of real electronics. Our evaluation with 24 participants showed that users were more efficient and more effective at naming components, as well as building and debugging circuits. We discuss our findings in the context of ElectronicsAR's unique characteristics that we contrast with related work. Based on this, we discuss opportunities for future research to design functional mobile AR applications that meet the needs of beginners and experts.

Novel input methods for game design often excite users, especially if they extend the way one interacts with the system. Electromyography (EMG) has the inherent potential to provide an intuitive - yet challenging - input channel for interactive systems. While this difficulty in control often limits the scope of applications for EMG in most systems, we argue that these qualities are especially relevant for games and playful interaction. The inherently challenging qualities of EMG input make the modality a prime candidate for designing body-centric playful experiences. Yet, we still need to understand its limitations to create engaging rather than frustrating experiences for users. In this work, we investigate EMG's potential to support playful interaction through exploratory studies, deriving feasible game interactions based on EMG's technical constraints, and study their application in game design. Based on our findings, we highlight design implications and pitfalls to avoid when creating EMG-based entertainment systems.

Mid-air haptics allow bare-hand tactile stimulation; however, it has a constrained workspace, making it unsuitable for room-scale haptics. We present a novel approach to rendering mid-air haptic sensations in a large rendering volume by turning a static array into a dynamic array following the user's hand. We used a 6DOF robot to drive a haptic ultrasound array over a large 3D space. Our system enables rendering room-scale mid-air experiences while preserving bare-hand interaction, thus, providing tangibility for virtual environments. To evaluate our approach, we performed three evaluations. First, we performed a technical system evaluation, showcasing the feasibility of such a system. Next, we conducted three psychophysical experiments, showing that the motion does not affect the user's perception with high likelihood. Lastly, we explored seven use cases that showcase our system's potential using a user study. We discuss challenges and opportunities in how large-scale mid-air haptics can contribute toward room-scale haptic feedback. Thus, with our system, we contribute to general haptic mid-air feedback on a large scale.

Private homes are increasingly becoming smart spaces. While smart homes promise comfort, they expose most intimate spaces to security and privacy risks. Unfortunately, most users today are not equipped with the right tools to assess the vulnerabilities or privacy practices of smart devices. Further, users might lose track of the devices installed in their homes or are unaware of devices placed by a partner or host. We developed SaferHome, an interactive digital-physical privacy framework, to provide smart home users with security and privacy assessments and a sense of device location. SaferHome includes a digital list view and physical and digital dashboards that map real floor plans. We evaluated SaferHome with eight households in the wild. We find that users adopted various strategies to integrate the dashboards into their understanding and interpretation of smart home privacy. We present implications for the design of future smart home privacy frameworks that are impacted by technical affinity, device types, device ownership, and tangibility of assessments.

In medicine, patients can obtain real benefits from a sham treatment. These benefits are known as the placebo effect. We report two experiments (Experiment I: N = 369; Experiment II: N = 100) demonstrating a placebo effect in adaptive interfaces. Participants were asked to solve word puzzles while being supported by no system or an adaptive AI interface. All participants experienced the same word puzzle difficulty and had no support from an AI throughout the experiments. Our results showed that the belief of receiving adaptive AI support increases expectations regarding the participant’s own task performance, sustained after interaction. These expectations were positively correlated to performance, as indicated by the number of solved word puzzles. We integrate our findings into technological acceptance theories and discuss implications for the future assessment of AI-based user interfaces and novel technologies. We argue that system descriptions can elicit placebo effects through user expectations biasing the results of user-centered studies.

Science faces a reproducibility crisis. There is a need to establish open science practices within the academic reputation economy. Open Science Badges address this issue by promoting and acknowledging research sharing and documentation. The generic design of currently awarded badges enabled their adoption across the sciences. Yet, their general nature makes it difficult to reflect individual practices and needs of distinct scientific fields. In this paper, we explore uses and effects of highly tailored badges in research data management. We implemented six science badges in a particle physics research preservation service. Our exploration showed that scientists were open to encouraging valuable scientific practices through tailored science badges. They described entirely new opportunities for interaction with research repositories. We present design implications for systems that promote reproducibility, related to meaningful criteria, repository navigation, and content discovery. Finally, we discuss the scope and uses of tailored science badges in modern science.

Engaging first-time users of mobile apps is challenging. Onboarding task flows are designed to minimize the drop out of users. To this point, there is little scientific insight into how to design these task flows. We explore this question with a specific focus on financial applications, which pose a particularly high hurdle and require significant trust. We address this question by combining two approaches. We first conducted semi-structured interviews (n=16) exploring users’ meaning-making when engaging with new mobile applications in general. We then prototyped and evaluated onboarding task flows (n=16) for two mobile cryptocurrency apps using the minimalist instruction framework. Our results suggest that well-designed onboarding processes can improve the perceived usability of first-time users for feature-rich mobile apps. We discuss how the expectations users voiced during the interview study can be met by applying instructional design principles and reason that the minimalist instruction framework for mobile onboarding insights presents itself as a useful design method for practitioners to develop onboarding processes and also to identify when not to.

Cryptocurrencies have increasingly gained interest in practice and research alike. Current research in the HCI community predominantly focuses on understanding the behavior of existing cryptocurrency users. Little attention has been given to early users and the challenges they encounter. However, understanding how interfaces of cryptocurrency systems support, impede, or even prevent adoption through new users is essential to develop better, more inclusive solutions. To close this gap, we conducted a user study (n=34) exploring challenges first-time cryptocurrency users face. Our analysis reveals that even popular wallets are not designed for novice users’ needs, stopping them when they would be ready to engage with the technology. We identify multiple challenges ranging from general user interface issues to finance and cryptocurrency-specific ones. We argue that these challenges can and should be addressed by the HCI community and present implications for building better cryptocurrency systems for novice users.

There are many situations where using personal devices is not socially acceptable, or where nearby people present a privacy risk. For these situations, we explore the concept of hidden interaction techniques through two prototype applications. HiddenHaptics allows users to receive information through vibrotactile cues on a smartphone, and HideWrite allows users to write text messages by drawing on a dimmed smartwatch screen. We conducted three user studies to investigate whether, and how, these techniques can be used without being exposed. Our primary findings are (1) users can effectively hide their interactions while attending to a social situation, (2) users seek to interact when another person is speaking, and they also tend to hide the interaction using their body or furniture, and (3) users can sufficiently focus on the social situation despite their interaction, whereas non-users feel that observing the user hinders their ability to focus on the social activity.

Physical exercises can benefit our health, but avoiding improper form and overexertion is essential. Facilitating bodily insights can encourage learning about exercise form, allowing users to gain a deeper understanding of their physiology. To investigate this, we conducted a lab experiment where amateur users performed bicep curls, and interviews with sports coaches. Participants were provided with FitBack—a system that monitors muscle activity during exercises via electromyography (EMG) and offers real-time biofeedback. Amateurs reported that they were successful in improving their exercise form and could acquire deeper bodily insights. Coaches reflected on how understanding muscle activity through EMG could be effectively used for increasing body awareness during coaching, highlighting that EMG-based biofeedback is beneficial for a diverse set of users. Our work contributes insights into using bodily sensing to help users understand their bodies. We contribute guidelines for designing systems that use EMG biofeedback effectively in physical activity.

While fitness trackers are increasingly popular among users, recent studies have shown that the health benefits of wearing a tracker are not apparent. The need to explicitly retrieve data can lead to limited benefits. Understanding how users can access, understand, and reflect on their data can lead to building systems that benefit our wellbeing. In this work, we explore the feasibility of using ubiquitous artifacts for unobtrusive feedback in health tracking. We evaluated a concept based on design dimensions for personal visualization on a smart mirror in a user study. Our design puts emphasis on the temporality of presented data. Participants found the visualizations comprehensive, rating cardiac and inertial data most useful as well as approved the different levels of temporal aggregation. Our work contributes findings on how to represent health-related data with ubiquitous artifacts to increase users’ awareness.

Detecting emotions while driving remains a challenge in Human-Computer Interaction. Current methods to estimate the driver’s experienced emotions use physiological sensing (e.g., skin-conductance, electroencephalography), speech, or facial expressions. However, drivers need to use wearable devices, perform explicit voice interaction, or require robust facial expressiveness. We present VEmotion (Virtual Emotion Sensor), a novel method to predict driver emotions in an unobtrusive way using contextual smartphone data. VEmotion analyzes information including traffic dynamics, environmental factors, in-vehicle context, and road characteristics to implicitly classify driver emotions. We demonstrate the applicability in a real-world driving study (N = 12) to evaluate the emotion prediction performance. Our results show that VEmotion outperforms facial expressions by 29% in a person-dependent classification and by 8.5% in a person-independent classification. We discuss how VEmotion enables empathic car interfaces to sense the driver’s emotions and will provide in-situ interface adaptations on-the-go.

Traditional meetings involve extensive sitting, which negatively impacts the health of attendees. Understanding how technology can facilitate integrating physical activity into the workplace, such as in walking meetings, is vital to improving workplace wellbeing. To that end, we applied a mixed-method approach to explore requirements and opportunities for walking meetings. We conducted an online questionnaire and a series of interviews with early adopters of walking meetings and created design fictions based on their feedback. We evaluated the design fictions with a second questionnaire and garnered additional feedback from the original early adopters. Based on our findings, we derived four dimensions associated with walking meetings: practical, environmental, social, and cognitive facets. We define attributes, challenges, and opportunities within these dimensions which are important for designing systems that support walking meetings. Our work identifies key considerations for developing systems that integrate physical activity into communication activities.

Virtual Reality (VR) experiences have massively improved in the mediation of feedback. However, the simulation of forces is still limited. This paper presents Odin's Helmet, a head-worn device to simulate g-forces that act on the human head in real-life situations. Odin's Helmet uses four head-mounted propellers as actuators to simulate g-forces through pushing and pulling the user's head while being immersed in VR. Odin's Helmet's goal is to increase presence and manipulate the user's perception of the otolith organ in the vestibular system. The user's perception will be tricked to experience a sensation of self-movement in VR. A technical evaluation shows Odin's Helmet's applicability to apply perceivable g-forces to the user's head. We conclude with future use cases of Odin's Helmet, such as redirected walking by controlling the user's head orientation, attention guidance, and wind simulations through Odin's Helmet.

Body movements, from a short smile to a marathon run, are driven by muscle activity. Despite the fact that measuring muscle activity with electromyography (EMG) is technically well established, it is highly complex and its use in interfaces has been limited. Easy access to muscle sensing can offer new opportunities to Human-Computer Interaction (HCI) research. Off-the-shelf sensors often only provide low-level access, hence requiring expertise in signal processing and widening the gulf of execution for users without engineering skills. To address this challenge, we introduce EMBody, a data-centric toolkit for EMG-based interface prototyping and experimentation. EMBody offers multiple levels of prototyping fidelity for EMG sensing, signal processing, and data interpretation. Our data-centric toolkit encapsulates the different data representation stages, offering a wide range of customization opportunities to experts while also allowing non-technical designers to focus on creating new interaction techniques. EMBody features a lightweight form factor and wireless connectivity. Additionally, the system leverages an exploration-centered workflow by allowing rapid access to measurement data via the accompanying software. Users define a set of motions to be recognized and interactively provide example data points. The toolkit then handles signal processing and classification. The recognized movements are streamed on the local network, ready to be used by interactive applications. This paper reports on how to use EMBody and its implementation. We iteratively developed the toolkit in a series of workshops and example applications. Users who had none or very limited knowledge of EMG could rapidly create engaging functional prototypes, while experts appreciated the modularity of the software component allowing for a high degree of customization. We contribute the software and hardware components of EMBody as a tool for the research community to stimulate creative exploration of EMG systems.

We investigate opportunities and challenges of running virtual reality (VR) studies remotely. Today, many consumers own head-mounted displays (HMDs), allowing them to participate in scientific studies from their homes using their own equipment. Researchers can benefit from this approach by being able to recruit study populations normally out of their reach, and to conduct research at times when it is difficult to get people into the lab (cf. the COVID pandemic). In an initial online survey ( N = 227), we assessed HMD owners’ demographics, their VR setups and their attitudes toward remote participation. We then identified different approaches to running remote studies and conducted two case studies for an in-depth understanding. We synthesize our findings into a framework for remote VR studies, discuss strengths and weaknesses of the different approaches, and derive best practices. Our work is valuable for Human-Computer Interaction (HCI) researchers conducting VR studies outside labs.

Workplace stress is a growing problem, which is often not identified by sufferers and those around them until it becomes chronic. Moreover, admitting to feel stressed is a highly private and therefore sensitive topic, particularly at work. Informed by the findings from six in-depth interviews, we designed the manually self-adjusted wearable BuSiNec: a Busyness Signifying Necklace. 18 participants wore it at their workplace and reported on their experiences and their usage behavior through a diary and in a focus group interview. Our findings indicate that BuSiNec supports self-reflection on stress and further stimulates valuable discussions among co-users which in turn increased mutual consideration. Participants were conscious of their their displayed busyness status towards others and used the display to encourage or discourage discussions and interruptions, or as a warning of upcoming hurry. From these findings, we derive recommendations for the integration of affective state information in wearable displays.

Improvising on the piano keyboard requires extensive skill development, which may reduce the feeling of immersion and flow for amateur players. However, being able to add simple musical effects greatly boosts a player's ability to express their unique playing style. To simplify this process, we designed an electromyography-based (EMG) system which integrates seamlessly into normal play by allowing musicians to modulate sound pitch using their thumb. We conducted an exploratory user study where users played a predefined melody and improvised using our system and a standard pitch wheel. Interview responses and survey answers showed that the EMG-based system supported the players' musical flow. Additionally, interviews indicated the system's capabilities to foster player creativity, and that players enjoyed experimenting with the effect. Our work illustrates how EMG can support seamless integration into existing systems to extend the range of interactions provided by a given interface.

Artificial Intelligence (AI) has become the buzzword of the last decade. Advances so far have been largely technical with a focus on machine learning (ML). Only recently have we begun seeing a shift towards focusing on the human aspects of artificial intelligence, centered on the narrow view of making AI interactive and explainable. In this paper I suggest a definition for "Interactive Human Centered Artificial Intelligence and outline the required properties. Staying in control is essential for humans to feel safe and have self-determination. Hence, we need to find ways for humans to understand AI based systems and means to allow human control and oversight. In our work, we argue that levels of abstractions and granularity of control are a general solution to this. Furthermore, it is essential that we make explicit why we want AI and what are the goals of AI research and development. We need to state the properties that we expect of future intelligent systems and who will benefit from a system or service. For me, AI and ML are very much comparable to raw materials (like stone, iron, or bronze). Historical periods are named after these materials as they fundamentally changed what humans can build and what tools humans can engineer. Hence, I argue that in the AI age we need to shift the focus from the material (e.g. the AI algorithms, as there will be plenty of material) towards the tools and infrastructures that are enabled which are beneficial to humans. It is apparent that AI will allow the automation of mental routine tasks and that it will extend our ability to perceive the world and foresee events. For me, the central question is how to create these tools for amplifying the human mind without compromising human values.

Augmented Reality (AR) provides a unique opportunity to situate learning content in one's environment. In this work, we investigated how AR could be developed to provide an interactive context-based language learning experience. Specifically, we developed a novel handheld-AR app for learning case grammar by dynamically creating quizzes, based on real-life objects in the learner's surroundings. We compared this to the experience of learning with a non-contextual app that presented the same quizzes with static photographic images. Participants found AR suitable for use in their everyday lives and enjoyed the interactive experience of exploring grammatical relationships in their surroundings. Nonetheless, Bayesian tests provide substantial evidence that the interactive and context-embedded AR app did not improve case grammar skills, vocabulary retention, and usability over the experience with equivalent static images. Based on this, we propose how language learning apps could be designed to combine the benefits of contextual AR and traditional approaches.

Virtual Reality experiences and games present believable virtual environments based on graphical quality, spatial audio, and interactivity. The interaction with in-game characters, controlled by computers (agents) or humans (avatars), is an important part of VR experiences. Pre-captured motion sequences increase the visual humanoid resemblance. However, this still precludes realistic social interactions (eye contact, imitation of body language), particularly for agents. We aim to make social interaction more realistic via social touch. Social touch is non-verbal, conveys feelings and signals (coexistence, closure, intimacy). In our research, we created an artificial hand to apply social touch in a repeatable and controlled fashion to investigate its effect on the perceived human-likeness of avatars and agents. Our results show that social touch is effective to further blur the boundary between computer- and human-controlled virtual characters and contributes to experiences that closely resemble human-to-human interactions.

Rapid Serial Visual Presentation (RSVP) has gained popularity as a method for presenting text on wearable devices with limited screen space. Nonetheless, it remains unclear how to calibrate RSVP display parameters, such as spatial alignments or presentation rates, to suit the reader's information processing ability at high presentation speeds. Existing methods rely on comprehension and subjective workload scores, which are influenced by the user's knowledge base and subjective perception. Here, we use electroencephalography (EEG) to directly determine how individual information processing varies with changes in RSVP display parameters. Eighteen participants read text excerpts with RSVP in a repeated-measures design that manipulated the Text Alignment and Presentation Speed of text representation. We evaluated how predictive EEG metrics were of gains in reading speed, subjective workload, and text comprehension. We found significant correlations between EEG and increasing Presentation Speeds and propose how EEG can be used for dynamic selection of RSVP parameters.

Field studies on public displays can be difficult, expensive, and time-consuming. We investigate the feasibility of using virtual reality (VR) as a test-bed to evaluate deployments of public displays. Specifically, we investigate whether results from virtual field studies, conducted in a virtual public space, would match the results from a corresponding real-world setting. We report on two empirical user studies where we compared audience behavior around a virtual public display in the virtual world to audience behavior around a real public display. We found that virtual field studies can be a powerful research tool, as in both studies we observed largely similar behavior between the settings. We discuss the opportunities, challenges, and limitations of using virtual reality to conduct field studies, and provide lessons learned from our work that can help researchers decide whether to employ VR in their research and what factors to account for if doing so.

Abstract Many individuals with visual impairments have residual vision that often remains underused by assistive technologies. Head-mounted augmented reality (AR) devices can provide assistance, by recoding difficult-to-perceive information into a visual format that is more accessible. Here, we evaluate symbolic and alphanumeric information representations for their efficiency and usability in two prototypical AR applications: namely, recognizing facial expressions of conversational partners and reading the time. We find that while AR provides a general benefit, the complexity of the visual representations has to be matched to the user’s visual acuity.

Augmented Reality (AR) and wearable sensors offer new possibilities to expand our senses and change how we interact with the world. Sensory augmentation can be integrated into everyday activities, but controls remain a challenge for user experience. In this paper, we investigate how users can control a futuristic interface that enables in-situ magnification. We designed an interactive system to enable users to zoom in on objects up to a microscopic level and implemented a prototype using the Microsoft Hololens. In a user-study, we compared full-screen to windowed visualizations and four interaction techniques for zooming: a clicker, two types of gestures, and voice. Our results indicate that the clicker enabled users to zoom at the fastest rate and lowered cognitive load. We also found a preference for windowed views. With our work, we provide insights for future augmented vision systems.

Human vision underlies several constraints in its spatial, temporal and spectral resolution. One limitation is the visual sensation of temperatures, which can be a useful addition for human sight in education or work scenarios. This demo presents how mixed reality applications can virtually superimpose objects’ thermal properties with the physical world into a coherent environment. We present a prototype that utilizes either a smartphone or a head-mounted display to mediate thermal flux. We show the feasibility of our prototype in the context of a physics lab experiment that fosters the understanding of heat conduction for different materials. We envision thermal vision as an addition to the human sight that can be accomplished with already available technologies. Finally, we discuss how our prototype can be designed as a context-aware in-situ interface that toggles between regular and thermal vision.

Augmented Reality (AR) is gaining increasing importance in science, education, and entertainment. A fundamental characteristic of AR is blending the virtual and physical world into a coherent environment. In this paper, we examine the effect of substituting the physical components of lab experiments with tangible replicas and virtual representations. We conducted a user study with thirty participants who carried out the experiment in three different abstraction levels (original lab equipment, non-functional tangible props, virtual representation). We compared the users’ performance regarding setup time, experienced workload, quality of measurements, and concept comprehension of the learning task. We found no effect on comprehension but significant differences in setup time and quality of measures. The results indicate that substitution reduces the experiment setup duration without affecting knowledge transfer. These results help to shape future AR learning environments, and we offer insights for creating complex mixed reality learning materials.

As science becomes increasingly data-intensive, the requirements for comprehensive Research Data Management (RDM) grow. This often overwhelms scientists, requiring more workload and training. The failure to conduct effective RDM leads to producing research artefacts that cannot be reproduced or reused. Past research placed high value on supporting data science workers, but focused mainly on data production, collection, processing, and sensemaking. In order to understand practices and needs of data science workers in relation to documentation, preservation, sharing, and reuse, we conducted a cross-domain study with 15 scientists and data managers from diverse scientific domains. We identified five core concepts which describe requirements, drivers, and boundaries in the development of commitment for RDM, essential for generating reproducible research artefacts: Practice, Adoption, Barriers, Education, and Impact. Based on those concepts, we introduce a stage-based model of personal RDM commitment evolution. The model can be used to drive the design of future systems that support a transition to open science. We discuss infrastructure, policies, and motivations involved at the stages and transitions in the model. Our work supports designers in understanding the constraints and challenges involved in designing for reproducibility in an age of data-driven science.

Reproducibility should be a cornerstone of scientific research and is a growing concern among the scientific community and the public. Understanding how to design services and tools that support documentation, preservation and sharing is required to maximize the positive impact of scientific research. We conducted a study of user attitudes towards systems that support data preservation in High Energy Physics, one of science's most data-intensive branches. We report on our interview study with 12 experimental physicists, studying requirements and opportunities in designing for research preservation and reproducibility. Our findings suggest that we need to design for motivation and benefits in order to stimulate contributions and to address the observed scalability challenge. Therefore, researchers' attitudes towards communication, uncertainty, collaboration and automation need to be reflected in design. Based on our findings, we present a systematic view of user needs and constraints that define the design space of systems supporting reproducible practices.

The need for data preservation and reproducible research is widely recognized in the scientific community. Yet, researchers often struggle to find the motivation to contribute to data repositories and to use tools that foster reproducibility. In this paper, we explore possible uses of gamification to support reproducible practices in High Energy Physics. To understand how gamification can be effective in research tools, we participated in a workshop and performed interviews with data analysts. We then designed two interactive prototypes of a research preservation service that use contrasting gamification strategies. The evaluation of the prototypes showed that gamification needs to address core scientific challenges, in particular the fair reflection of quality and individual contribution. Through thematic analysis, we identified four themes which describe perceptions and requirements of gamification in research: Contribution, Metrics, Applications and Scientific practice. Based on these, we discuss design implications for gamification in science.

While Virtual Reality continues to increase in fidelity, it remains an open question how to effectively reflect the user's movements and provide congruent feedback in virtual environments. We present VRsneaky, a system for producing auditory movement feedback, which helps participants orient themselves in a virtual environment by providing footstep sounds. The system reacts to the user's specific gait features and adjusts the audio accordingly. In a user study with 28 participants, we found that VRsneaky increases users' sense of presence as well as awareness of their own posture and gait. Additionally, we find that increasing auditory realism significantly influences certain characteristics of participants' gait. Our work shows that gait-aware audio feedback is a means to increase presence in virtual environments. We discuss opportunities and design requirements for future scenarios where users walk through immersive virtual worlds.

While we usually have no trouble with orientation, our sense of direction frequently fails in the absence of a frame of reference. Open-water swimmers raise their heads to look for a reference point, since disorientation might result in exhaustion or even drowning. In this paper, we report on Clairbuoyance - a system that provides feedback about the swimmer's orientation through lights mounted on swimming goggles. We conducted an experiment with two versions of Clairbuoyance: Discrete signals relative to a chosen direction, and continuous signals providing a sense of absolute direction. Participants swam to a series of targets. Proficient swimmers preferred the discrete mode; novice users the continuous one. We determined that both versions of Clairbuoyance enabled reaching the target faster than without the help of the system, although the discrete mode increased error. Based on the results, we contribute insights for designing directional guidance feedback for swimmers.

While Augmented Reality (AR) technologies are becoming increasingly available, our understanding of AR is primarily limited to controlled experiments which address use at work or for entertainment. Little is known about how it could enhance everyday interaction from a user's perspective. Personal use of AR at home may improve how users' interface with information on a daily basis. Through an online survey, we investigated attitudes towards domestic AR. We further explored the opportunities for AR at home in a technology probe. We first introduced the users to AR by offering an AR experience presented through mixed reality smart glasses. We then used a tailor-made tablet application to elicit photos illustrating how users imagine future AR experiences. Finally, we conducted semi-structured interviews based on elicited photos. Our results show that users are eager to benefit from on-demand information, assistance, enhanced sensory perception, and play offered by AR across many locations at home. We contribute insights for future AR systems designed for domestic environments.

Recent work demonstrated the opportunities of thermal imaging in the development of novel interactive systems. However, the exploration is limited to controlled lab setups. Hence, little we know about how thermal imaging could be useful for a broader range of daily applications by novice users. To investigate the potential of domestication of thermal imaging, we conducted an exploration with a technology-cultural probe. Ten households (26 individuals) used a mobile thermal camera in their daily life. We collected thermal photos taken by the participants and conducted interviews after using the camera. We found that the users were excited about using thermal cameras in their everyday lives and found many practical uses for them. Our study provides insights into how novice users wish to use thermal imaging technology to augment their vision in daily setups, as well as identifying and classifying common thermal imaging use cases. Our work contributes implications for designing thermal imaging devices targeted towards novice users.

Reading foreign language texts is a frequently used strategy for language learning. Visual text augmentation methods further support the learning experience, e.g., by annotating vocabulary or grammar. Common approaches are integrated dictionaries or static grammar highlights. This work investigates how we can further support grammar learning with the dynamic visualization and interaction opportunities offered by digital reading devices. In collaboration with teachers and potential learners, we identify difficulties learners experience with English grammar and gather ideas for suitable interactive text augmentations. Based on this, we design four different concepts that augment adjectives and adverbs in English-language texts using typographic cues and interactive information displays. The concepts are evaluated in a within-subject study (N = 16). Results show that participants preferred concepts that presented case-specific support, did not distract too much from the text, and gave details on demand. We conclude with design recommendations for designing text augmentation for language learning.

We present DronOS, a rapid prototyping framework that can track, control, and automate drone routines. Previous research in the domain of Human-Drone Interaction relied on hardware or proprietary vendor-dependent libraries that had to be exclusively programmed for specific use cases. This forces users to stick with a drone manufacturer or model as well as limiting users in transferring their drone control logic to other drones. To overcome the aforementioned issues, our framework uses low-cost off-the-shelf hardware and applies to a variety of already available or self-crafted drones. To assess the usability of DronOS, we evaluate three drone programming modes: Unity Scripting, Vive Scripting, and Vive Realtime. We find that Vive Scripting required the least subjective workload in programming drone routines while Unity Scripting yielded the highest accuracy and Vive Realtime the least task completion time. We anticipate requirements for drone prototyping frameworks that target novice and expert users as operators.

Uncertainty is common when working with data and becomes more important as processing big data gains attention. However, no standard tangible interface element exists for inputting uncertain data. In this article, we extend the input space of two traditional TUIs: dial and slider. We present five designs that are based on dials and sliders and support uncertain input. We conduct focus group interviews to evaluate the designs. The interviews allow us to extend existing design requirements for parameter control UIs to support uncertain input.

As architects usually decide on the shape and look of windows during the design of buildings, opportunities for interactive windows have not been systematically explored yet. In this work, we extend the vision of sustainable and comfortable adaptive buildings using interactive smart windows. We systematically explore the design space of interactive windows to chart requirements, constraints, and challenges. To that end, we built proof-of-concept prototypes of smart windows with fine-grained control of transparency. In two studies, we explored user attitudes towards interactive windows and elicited control methods. We found that users understand and see potential for interactive windows at home. We provide specific usage contexts and specify interactions that may facilitate domestic applications. Our work illustrates the concept of interactive smart windows and provides insights regarding their design, development, and user controls for adaptive walls. We identify design dimensions and challenges to stimulate further development in the domain of adaptive buildings.

Mastering fine motor tasks, such as playing the guitar, takes years of time-consuming practice. Commonly, expensive guidance by experts is essential for adjusting the training program to the student's proficiency. In our work, we showcase the suitability of Electromyography to detect fine-grained hand and finger postures in an exemplary guitar tutor scenario. We present EMGuitar, an interactive guitar tutoring system, that assists students by reporting on play correctness and adjusts playback tempi automatically. We report person-dependent classification utilizing a ring of electrodes around the forearm with an F1 score of up to 0.89 on recorded calibration data. Furthermore, our system was received well by neither diminishing ease of use nor being disruptive for the participants. Based on the received comments, we identified the need for detailed play accuracy feedback down to individual chords, for which we suggest an adapted visualization and an algorithmic approach.

Tactile overlays for touch-screen devices are an opportunity to display content for users with visual impairments. However, when users switch tactile overlays, the displayed content on the touch-screen devices still correspond to the previous overlay. Currently, users have to change the displayed content on the touch-screen devices manually which hinders a fluid user interaction. In this paper, we introduce self-identifying overlays - an automated method for touch-screen devices to identify tactile overlays placed on the screen and to adapt the displayed content based on the applied tactile overlay. We report on a pilot study with two participants with visual impairments to evaluate this approach with a functional content exploration application based on an adapted textbook.

The number of sensors in our surroundings that provide the same information steadily increases. Since sensing is prone to errors, sensors may disagree. For example, a GPS-based tracker on the phone and a sensor on the bike wheel may provide discrepant estimates on traveled distance. This poses a user dilemma, namely how to reconcile the conflicting information into one estimate. We investigated whether visualizing the uncertainty associated with sensor measurements improves the quality of users' inference. We tested four visualizations with increasingly detailed representation of uncertainty. Our study repeatedly presented two sensor measurements with varying degrees of inconsistency to participants who indicated their best guess of the "true" value. We found that uncertainty information improves users' estimates, especially if sensors differ largely in their associated variability. Improvements were larger for information-rich visualizations. Based on our findings, we provide an interactive tool to select the optimal visualization for displaying conflicting information.

Fitness trackers not just provide easy means to acquire physiological data in real-world environments due to affordable sensing technologies, they further offer opportunities for physiology-aware applications and studies in HCI; however, their performance is not well understood. In this paper, we report findings on the quality of 3 sensing technologies: PPG-based wrist trackers (Apple Watch, Microsoft Band 2), an ECG-belt (Polar H7) and reference device with stick-on ECG electrodes (Nexus 10). We collected physiological (heart rate, electrodermal activity, skin temperature) and subjective data from 21 participants performing combinations of physical activity and stressful tasks. Our empirical research indicates that wrist devices provide a good sensing performance in stationary settings. However, they lack accuracy when participants are mobile or if tasks require physical activity. Based on our findings, we suggest a textitDesign Space for Wearables in Research Settings and reflected on the appropriateness of the investigated technologies in research contexts.

Individuals with cognitive impairments currently leverage extensive human resources during their transitions from assisted living to independent living. In Western Europe, many government-supported volunteer organizations provide sheltered living facilities; supervised environments in which people with cognitive impairments collaboratively learn daily living skills. In this paper, we describe communal cooking practices in sheltered living facilities and identify opportunities for supporting these with interactive technology to reduce volunteer workload. We conducted two contextual observations of twelve people with cognitive impairments cooking in sheltered living facilities and supplemented this data through interviews with four employees and volunteers who supervise them. Through thematic analysis, we identified four themes to inform design requirements for communal cooking activities: Work organization, community, supervision, and practicalities. Based on these, we present five design implications for assistive systems in kitchens for people with cognitive deficiencies.

We present LYRA, a modular in-flight system that enhances service and assists flight attendants during their work. LYRA enables passengers to browse and order services from their smartphones. Smart glasses and a smart shoe-clip with RFID reader module provides flight attendants with situated information. We gained first insights into how flight attendants and passengers use of the system during a long distance flight from Frankfurt to Houston.

Human vision is limited to a small band of the electromagnetic spectrum. It has been shown that extending visual perception can be beneficial, but it is unclear if this is useful for a broader range of applications. In this paper, we explore user reactions to extended visual perception. We describe the design and implementation of TriSight, a head-mounted display that allows the user to perceive the environment in the visual spectrum, the thermal spectrum, and through depth maps. In a study, we asked participants to perform everyday tasks in a created home, kitchen, office and basement with TriSight. Through analyzing videos, interviews, questionnaires and logs we chart the users' feedback towards the augmented visual perception. Our findings, imply the positive impact of extended vision, as well as the potential of thermal vision as alternative perception mode in performing daily tasks. Finally, we discuss the implications for designing devices that extend human's visual perception beyond the visual spectrum.

To help deal with daily reading volumes, we present Reading Scheduler, a smartphone application linked to people's reading list, which triggers reading reminders throughout the day. The app suggests articles according to their length, complexity, and the time available for reading as indicated by the user. In a field study, we collected usage data from ten participants over the course of two weeks. During this time, we recorded mobile sensor data and trained a classifier to detect opportune moments for reading. Participants read 182 articles while we collected 787,752 sensor data points. Together with an assessment of the feasibility of proactive reading suggestions, we present a prediction model with close to 73% accuracy, that can be used to build mobile recommender systems for utilizing idle moments for reading throughout the user's day.

We present VRHapticDrones, a system utilizing quadcopters as levitating haptic feedback proxy. A touchable surface is attached to the side of the quadcopters to provide unintrusive, flexible, and programmable haptic feedback in virtual reality. Since the users' sense of presence in virtual reality is a crucial factor for the overall user experience, our system simulates haptic feedback of virtual objects. Quadcopters are dynamically positioned to provide haptic feedback relative to the physical interaction space of the user. In a first user study, we demonstrate that haptic feedback provided by VRHapticDrones significantly increases users' sense of presence compared to vibrotactile controllers and interactions without additional haptic feedback. In a second user study, we explored the quality of induced feedback regarding the expected feeling of different objects. Results show that VRHapticDrones is best suited to simulate objects that are expected to feel either light-weight or have yielding surfaces. With VRHapticDrones we contribute a solution to provide unintrusive and flexible feedback as well as insights for future VR haptic feedback systems.

Manual assembly at production is a mentally demanding task. With rapid prototyping and smaller production lot sizes, this results in frequent changes of assembly instructions that have to be memorized by workers. Assistive systems compensate this increase in mental workload by providing "just-in-time" assembly instructions through in-situ projections. The implementation of such systems and their benefits to reducing mental workload have previously been justified with self-perceived ratings. However, there is no evidence by objective measures if mental workload is reduced by in-situ assistance. In our work, we showcase electroencephalography (EEG) as a complementary evaluation tool to assess cognitive workload placed by two different assistive systems in an assembly task, namely paper instructions and in-situ projections. We identified the individual EEG bandwidth that varied with changes in working memory load. We show, that changes in the EEG bandwidth are found between paper instructions and in-situ projections, indicating that they reduce working memory compared to paper instructions. Our work contributes by demonstrating how design claims of cognitive demand can be validated. Moreover, it directly evaluates the use of assistive systems for delivering context-aware information. We analyze the characteristics of EEG as real-time assessment for cognitive workload to provide insights regarding the mental demand placed by assistive systems.

We often base our decisions on uncertain data - for instance, when consulting the weather forecast before deciding what to wear. Due to their uncertainty, such forecasts can differ by provider. To make an informed decision, many people compare several forecasts, which is a time-consuming and cumbersome task. To facilitate comparison, we identified three aggregation mechanisms for forecasts: manual comparison and two mechanisms of computational aggregation. In a survey, we compared the mechanisms using different representations. We then developed a weather application to evaluate the most promising candidates in a real-world study. Our results show that aggregation increases users' confidence in uncertain data, independent of the type of representation. Further, we find that for daily events, users prefer to use computationally aggregated forecasts. However, for high-stakes events, they prefer manual comparison. We discuss how our findings inform the design of improved interfaces for comparison of uncertain data, including non-weather purposes.

Obtaining information about audience engagement in presentations is a valuable asset for presenters in many domains. Prior literature mostly utilized explicit methods of collecting feedback which induce distractions, add workload on audience and do not provide objective information to presenters. We present EngageMeter - a system that allows fine-grained information on audience engagement to be obtained implicitly from multiple brain-computer interfaces (BCI) and to be fed back to presenters for real time and post-hoc access. Through evaluation during an HCI conference (Naudience=11, Npresenters=3) we found that EngageMeter provides value to presenters (a) in real-time, since it allows reacting to current engagement scores by changing tone or adding pauses, and (b) in post-hoc, since presenters can adjust their slides and embed extra elements. We discuss how EngageMeter can be used in collocated and distributed audience sensing as well as how it can aid presenters in long term use.

We are often confronted with information interfaces designed in an unfamiliar language, especially in an increasingly globalized world, where the language barrier inhibits interaction with the system. In our work, we explore the design space for building interfaces that can detect the user's language proficiency. Specifically, we look at how a user's gaze properties can be used to detect whether the interface is presented in a language they understand. We report a study (N=21) where participants were presented with questions in multiple languages, whilst being recorded for gaze behavior. We identified fixation and blink durations to be effective indicators of the participants' language proficiencies. Based on these findings, we propose a classification scheme and technical guidelines for enabling language proficiency awareness on information displays using gaze data.

People use more and more applications and devices that quantify daily behavior such as the step count or phone usage. Purely presenting the collected data does not necessarily support users in understanding their behavior. In recent research, concepts such as learning by reflection are proposed to foster behavior change based on personal data. In this paper, we introduce user-made predictions to help users understand personal behavior patterns. Therefore, we developed an Android application that tracks users' screen-on and unlock patterns on their phone. The application asks users to predict their daily behavior based on their former usage data. In a user study with 12 participants, we showed the feasibility of leveraging user-made predictions in a quantified self approach. By trying to improve their predictions over the course of the study, participants automatically discovered new insights into personal behavior patterns.

Smartphones, wearables, and other mobile devices often use tactile feedback for notifying users. This feedback type proved to be beneficial since it does not occupy the visual or auditory channel. However, it still can be distracting in other situations such as when users are already stressed. To investigate tactile feedback patterns which do not increase the user's stress level, we developed two wrist-worn prototypes capable of providing tactile feedback (i.e., vibrotactile and pressure-based feedback). Further, we conducted a user-study with 14 participants comparing both feedback types. The results suggest that vibrotactile feedback increases the user's stress level more, compared to pressure-based feedback particularly applied when the user currently has a low stress level. Consequently, we present implications for designing notifications for mobile and wearable devices.

Regular physical activity reduces the risks of illness, obesity, and falling. Thus, many personal devices support users in monitoring their physical activity to initiate behavior changes. However, activity data is prone to measurement errors; for example a user is seated but typing increases the step count. Such false positives could be easily detected if data from multiple sensors would be connected. We envision a system that combines data from multiple sensors in our surroundings to reduce such measurement errors. Furthermore, the system shows the user's historical data for the past seven days, the activity level of the current day, and a forecast about physical activity for the next seven days in an artistic and configurable digital wall painting. We argue that this strengthens connectedness and privacy. We are convinced that our system can help to increase user's trust in activity data and raise awareness for behavior change regarding physical activity.

Although more and more data is collected automatically, many interfaces still require manual input. When we, for example, enter our daily calorie intake or calculate our ecological footprint, we often have to guess the weight of the food or what distance we have covered with our car. In this paper, we propose a solution to overcome the problem of forcing users to enter a single value when they are unsure about the actual input. On the basis of a slider, we designed four input controls which allow the input of uncertain data in the form of probability distribution functions. To evaluate our input controls, we conducted two studies collecting subjective and objective feedback. Based on the evaluation, we derived implications for their usage. We additionally provide an open-source toolkit with the evaluated input controls that can be included in web applications and customized for different contexts and tasks.

More and more industrial manufacturing companies are outsourcing assembly tasks to sheltered work organizations where cognitively impaired workers are employed. To facilitate these assembly tasks assistive systems have been introduced to provide cognitive assistance. While previous work found that these assistive systems have a great impact on the workers' performance in giving assembly instructions, these systems are further capable of detecting errors and notifying the worker of an assembly error. However, the topic of how assembly errors are presented to cognitively impaired workers has not been analyzed scientifically. In this paper, we close this gap by comparing tactile, auditory, and visual error feedback in a user study with 16 cognitively impaired workers. The results reveal that visual error feedback leads to a significantly faster assembly time compared to tactile error feedback. Further, we discuss design implications for providing error feedback for workers with cognitive impairments.

In this paper, we present a proof-of-concept approach to estimating mental workload by measuring the user's pupil diameter under various controlled lighting conditions. Knowing the user's mental workload is desirable for many application scenarios, ranging from driving a car, to adaptive workplace setups. Typically, physiological sensors allow inferring mental workload, but these sensors might be rather uncomfortable to wear. Measuring pupil diameter through remote eye-tracking instead is an unobtrusive method. However, a practical eye-tracking-based system must also account for pupil changes due to variable lighting conditions. Based on the results of a study with tasks of varying mental demand and six different lighting conditions, we built a simple model that is able to infer the workload independently of the lighting condition in 75% of the tested conditions.

With increasing complexity of assembly tasks and an increasing number of product variants, instruction systems providing cognitive support at the workplace are becoming more important. Different instruction systems for the workplace provide instructions on phones, tablets, and head-mounted displays (HMDs). Recently, many systems using in-situ projection for providing assembly instructions at the workplace have been proposed and became commercially available. Although comprehensive studies comparing HMD and tablet-based systems have been presented, in-situ projection has not been scientifically compared against state-of-the-art approaches yet. In this paper, we aim to close this gap by comparing HMD instructions, tablet instructions, and baseline paper instructions to in-situ projected instructions using an abstract Lego Duplo assembly task. Our results show that assembling parts is significantly faster using in-situ projection and locating positions is significantly slower using HMDs. Further, participants make less errors and have less perceived cognitive load using in-situ instructions compared to HMD instructions.

Due to the rise of lifelog cameras, we have personal video data that is too large to be watched. Video indexing has the potential to provide meta-information for faster video search. This work aims to support lifelog video indexing through automated face priority rating. In a user study, we identified parameters that allow for rating the importance of persons in a video. We implemented these findings to automatically predict the person's importance in video. We show that our algorithm predicts similar person priority ratings like the participants had given. Hence, we contribute to video-based lifelogging through indicating, implementing, and testing face indexing rules that predict how important a person in a video is perceived. Our findings can help to build video players that support users navigating through their large video data and reviewing sequences that recall important moments of life.

With the advent of lifelogging cameras the amount of personal video material is massively growing to an extent that easily overwhelms the user. To efficiently review lifelog data, we need well designed video navigation tools. In this paper, we analyze which cues are most beneficial for lifelog video navigation. We show that the information kind determines the most appropriate cue in single cue systems, but that multicue approaches are more appreciated. These findings can inspire to design video players with multiple navigation cues, including time, place, persons, and events for easier and more efficient lifelog video retrieval.

Today's working society tries to integrate more and more impaired workers into everyday working processes. One major scenario for integrating impaired workers is in the assembly of products. However, the tasks that are being assigned to cognitively impaired workers are easy tasks that consist of only a small number of assembly steps. For tasks with a higher number of steps, cognitively impaired workers need instructions to help them with assembly. Although supervisors provide general support and assist new workers while learning new assembly steps, sheltered work organizations often provide additional printed pictorial instructions that actively guide the workers. To further improve continuous instructions, we built a system that uses in-situ projection and a depth camera to provide context-sensitive instructions. To explore the effects of in-situ instructions, we compared them to state-of-the-art pictorial instructions in a user study with 15 cognitively impaired workers at a sheltered work organization. The results show that using in-situ instructions, cognitively impaired workers can assemble more complex products up to 3 times faster and with up to 50% less errors. Further, the workers liked the in-situ instructions provided by our assistive system and would use it for everyday assembly.

Recent work demonstrated the exciting opportunities that thermal imaging offers for the development of interactive systems. It was shown that a thermal camera can sense when a user touches a surface, performs gestures in the camera's direct field of view and, in addition, performs gestures outside the camera's direct field of view through thermal reflection. In this paper, we investigate the material properties that should be considered for detecting interaction using thermal imaging considering both in- and outdoor settings. We conducted a study to analyze the recognition performance for different gestures and different surfaces. Using the results, we derive guidelines on material properties of surfaces for detecting on-surface as well as mid-air interaction using a thermal camera. We discuss the constrains that should be taken into account using thermal imaging as the sensing technology. Finally, we present a material space based on our findings. The space depicts surfaces and the required properties that enable the different interaction techniques.

The advent of wearables (e.g., smartwatches, smartglasses, and digital jewelry) anticipates the need for text entry methods on very small devices. We conduct fundamental research on this topic using 3 qwerty-based soft keyboards for 3 different screen sizes, motivated by the extensive training that users have with qwerty keyboards. In addition to ZoomBoard (a soft keyboard for diminutive screens), we propose a callout-based soft keyboard and ZShift, a novel extension of the Shift pointing technique. We conducted a comprehensive user study followed by extensive analyses on performance, usability, and short-term learning. Our results show that different small screen sizes demand different types of assistance. In general, manufacturers can benefit from these findings by selecting an appropriate qwerty soft keyboard for their devices. Ultimately, this work provides designers, researchers, and practitioners with new understanding of qwerty soft keyboard design space and its scalability for tiny touchscreens.

Order Picking is not only one of the most important but also most mentally demanding and error-prone tasks in the industry. Both stationary and wearable systems have been introduced to facilitate this task. Existing stationary systems are not scalable because of the high cost and wearable systems have issues being accepted by the workers. In this paper, we introduce a mobile camera-projector cart called OrderPickAR, which combines the benefits of both stationary and mobile systems to support order picking through Augmented Reality. Our system dynamically projects in-situ picking information into the storage system and automatically detects when a picking task is done. In a lab study, we compare our system to existing approaches, i.e, Pick-by-Paper, Pick-by-Voice, and Pick-by-Vision. The results show that using the proposed system, order picking is almost twice as fast as other approaches, the error rate is decreased up to 9 times, and mental demands are reduced up to 50%.

With an increasing number of new camera devices entering the market, lifelogging has turned into a viable everyday practice. The promise of comprehensively capturing our life's happenings has caused adoption rates to grow, but approaches to do so greatly differ. In this paper we evaluate existing visual lifelogging capture approaches through a user study with two main capture dimensions: (1) comparing the body position where a lifelogging camera is worn: head versus chest (2) comparing the media captures: video versus stills. We equipped 30 participants with cameras on their heads and chests. That data was evaluated by subjective user ratings as well as by objective image processing analysis. Our findings indicate that (1) chest-worn devices are more stable and contain less motion blur through which feature detection by image processing algorithms works better than from head-worn cameras; 2) head-worn video cameras, however, seem to be the better choice for lifelogging as they capture more important autobiographical cues than chest-worn devices, e.g., faces that have been shown to be most relevant for recall.

With the opportunity to customize ordered products, assembly tasks are becoming more and more complex. To meet these increased demands, a variety of interactive instruction systems have been introduced. Although these systems may have a big impact on overall efficiency and cost of the manufacturing process, it has been difficult to optimize them in a scientific way. The challenge is to introduce performance metrics that apply across different tasks and find a uniform experiment design. In this paper, we address this challenge by proposing a standardized experiment design for evaluating interactive instructions and making them comparable with each other. Further, we introduce a General Assembly Task Model, which differentiates between task-dependent and task-independent measures. Through a user study with 12 participants, we evaluate the experiment design and the proposed task model using an abstract pick-and-place task and an artificial industrial task. Finally, we provide paper-based instructions for the proposed task as a baseline for evaluating Augmented Reality instructions.

In this paper, we identify design guidelines for stereoscopic 3D (S3D) user interfaces (UIs) and present the MirrorBox and the FrameBox, two UI prototyping tools for S3D displays. As auto-stereoscopy becomes available for the mass market we believe the design of S3D UIs for devices, for example, mobile phones, public displays, or car dashboards, will rapidly gain importance. A benefit of such UIs is that they can group and structure information in a way that makes them easily perceivable for the user. For example, important information can be shown in front of less important information. This paper identifies core requirements for designing S3D UIs and derives concrete guidelines. The requirements also serve as a basis for two depth layout tools we built with the aim to overcome limitations of traditional prototyping when sketching S3D UIs. We evaluated the tools with usability experts and compared them to traditional paper prototyping.

The entertainment industry is going through a transformation, and technology development is affecting how we can enjoy and interact with the entertainment media content in new ways. In our work, we explore how to enable interaction with content in the context of 3D cinemas. This allows viewers to use their mobile phone to retrieve, for example, information on the artist of the soundtrack currently playing or a discount coupon on the watch the main actor is wearing. We are particularly interested in the user experience of the interactive 3D cinema concept, and how different interactive elements and interaction techniques are perceived. We report on the development of a prototype application utilizing smart phones and on an evaluation in a cinema context with 20 participants. Results emphasize that designing for interactive cinema experiences should drive for holistic and positive user experiences. Interactive content should be tied together with the actual video content, but integrated into contexts where it does not conflict with the immersive experience with the movie.

Thermal cameras have recently drawn the attention of HCI researchers as a new sensory system enabling novel interactive systems. They are robust to illumination changes and make it easy to separate human bodies from the image background. Far-infrared radiation, however, has another characteristic that distinguishes thermal cameras from their RGB or depth counterparts, namely thermal reflection. Common surfaces reflect thermal radiation differently than visual light and can be perfect thermal mirrors. In this paper, we show that through thermal reflection, thermal cameras can sense the space beyond their direct field-of-view. A thermal camera can sense areas besides and even behind its field-of-view through thermal reflection. We investigate how thermal reflection can increase the interaction space of projected surfaces using camera-projection systems. We moreover discuss the reflection characteristics of common surfaces in our vicinity in both the visual and thermal radiation bands. Using a proof-of-concept prototype, we demonstrate the increased interaction space for hand-held camera-projection system. Furthermore, we depict a number of promising application examples that can benefit from the thermal reflection characteristics of surfaces.

Notifications are a core feature of mobile phones. They inform users about a variety of events. Users may take immediate action or ignore them depending on the importance of a notification as well as their current context. The nature of notifications is manifold, applications use them both sparsely and frequently. In this paper we present the first large-scale analysis of mobile notifications with a focus on users' subjective perceptions. We derive a holistic picture of notifications on mobile phones by collecting close to 200 million notifications from more than 40,000 users. Using a data-driven approach, we break down what users like and dislike about notifications. Our results reveal differences in importance of notifications and how users value notifications from messaging apps as well as notifications that include information about people and events. Based on these results we derive a number of findings about the nature of notifications and guidelines to effectively use them.

Touch-enabled user interfaces have become ubiquitous, such as on ATMs or portable devices. At the same time, authentication using touch input is problematic, since finger smudge traces may allow attackers to reconstruct passwords. We present SmudgeSafe, an authentication system that uses random geometric image transformations, such as translation, rotation, scaling, shearing, and flipping, to increase the security of cued-recall graphical passwords. We describe the design space of these transformations and report on two user studies: A lab-based security study involving 20 participants in attacking user-defined passwords, using high quality pictures of real smudge traces captured on a mobile phone display; and an in-the-field usability study with 374 participants who generated more than 130,000 logins on a mobile phone implementation of SmudgeSafe. Results show that SmudgeSafe significantly increases security compared to authentication schemes based on PINs and lock patterns, and exhibits very high learnability, efficiency, and memorability.

Computing technology is currently adopted in emerging countries. Especially mobile phones and smart phones become widely used - with a much higher penetration than traditional computers. In our work we investigate how computing technologies and particularly mobile devices can support education. While previous work focused on controlled experiments, in this paper we present the results of a 20 weeks long study of mobile phone usage in an emerging region. Our aim was not only to investigate how the phones are used for education but also to learn how they are adopted by children in daily life. By logging screenshots, we used an unsupervised approach that allowed to unobtrusively observe usage patterns without the presence of researchers. Instead of offering tailored teaching applications, we used general-purpose applications to support teaching and found that the phone itself was an empowering technology similar to pen and paper. Based on a detailed analysis of actual use in a natural setting, we derived a set of typical use cases for mobile phones in education and describe how they change learning. From in-depth interviews with a teacher, selected guardians and pupils we show that introducing mobiles phones has great potential for supporting education in emerging regions.

Many embedded platforms that support the creation of interactive smart objects have become available over the last years. Arduino, Raspberry Pi, electric imp, mbed, MSP430, and .NET Gadgeteer are examples of hardware platforms with very different properties and capabilities. In order to make interactive artifacts additional sensors, actuators, and networking elements are available for the different platforms. Additionally there are different software environments and development tools that support developers in creating custom applications for embedded systems. It is apparent that one size doesn't fit all and that choosing the right platform and tools is an important step towards an effective solution. In this TEI2014 studio we first provide an overview of available platforms and tools that allow developers to create novel and tangible interactive systems. We will present and discuss developing environments, with a specific focus on browser based programming tools and social coding. In a second step we will have 2 hands-on sessions, where in each we use a different platform and create an initial functional prototype. The aim is to provide the participants with an overview of existing embedded development tools suitable for creating interactive artifacts and to provide some hands on-experience with different new platforms.

Calling while driving a car has become very common since the rise of mobile phones. Drivers use their phone despite the fact that calling in the car is potentially distracting and dangerous. Prohibiting communication while driving is not a good idea as there are also positive effects of calling (e.g., ability to notify about a delay, staying awake, preventing fatigue, guidance at foreign places).

Driving a car is becoming increasingly complex. Many new features (e.g., for communication or entertainment) that can be used in addition to the primary task of driving a car increase the driver's workload. Assessing the driver's workload, however, is still a challenging task. A variety of means are explored which rather focus on experimental conditions than on real world scenarios (e.g., questionnaires). We focus on physiological data that may be assessed in an non-obtrusive way in the future and is therefore applicable in the real world.

Falling hardware prices and ever more displays being connected to the Internet will lead to large public display networks, potentially forming a novel communication medium. We envision that such networks are not restricted to display owners and advertisers anymore, but allow also passersby (e.g., customers) to exchange content, similar to traditional public notice areas, such as bulletin boards. In this context it is crucial to understand emerging practices and provide easy and straight forward interaction techniques to be used for creating and exchanging content. In this paper, we present Digifieds, a digital public notice area we built to investigate and compare possible interaction techniques. Based on a lab study we show that using direct touch at the display as well as using the mobile phone as a complementing interaction technology are most suitable. Direct touch at the display closely resembles the interaction known from classic bulletin boards and provides the highest usability. Mobile phones preserve the users' privacy as they exchange (sensitive) data with the display and at the same time allow content to be created on-the-go or to be retrieved.

It has become extremely easy for developers to build custom software for smartphone and tablet computers. However, it is still hard to extend those devices with external electronics, e.g. additional sensors and actuators. In the moment when external hardware can be easily attached to phones and tablets a wide new application space will be opened up. With WebClip we present a device offering digital and analogue I/O ports that can be controlled and monitored by just clipping the device onto a capacitive touch screen. A web page in the browser of the touch screen device is used to control the bi-directional communication. Data from the WebClip to the device is sent by emulating touches on the screen whereas the reverse direction uses light sensors on the bottom side of the clip to receive light sequences emitted by the web page. A simple Javascript API is offered to build custom web applications. We have successfully tested our prototype with a variety of phones and tablet computers and report on performance and limitations.

Drawing and handwriting play a central role in primary schools. So far handwriting is practiced mainly on paper and blackboards. Providing tasks on paper can be challenging in developing countries. With the potential availability of mobile phones in classrooms, there is a new medium that can be used. We determined the effect of different touch technologies on children's handwriting for 18 third grade and 20 sixth grade participants. Children drew and wrote using different input techniques. We measured their performance and asked teachers to assess the legibility. We show that writing on touchscreens is less legible and slower than on paper. Further, the comparison of touchscreen technologies indicates that capacitive screens operated with a stylus yield the highest readability and are faster to use for writing than resistive screens. In contrast to these quantitative findings participants from third grade indicated that they prefer resistive screens with a thin stylus compared to using capacitive screens with a stylus or fingers.

In this paper, we present results from an ongoing participatory design research project that focuses on the design and evaluation of a tagging-system and corresponding visualizations to support actors and visitors of an urban gardening project. The contributions of this work are threefold. First, it addresses the yet underexplored field of integrating context information beyond location and time into mobile tagging systems and folksonomies. Second, it suggests novel visualizations to explore tagged data and folksonomies beyond tag clouds and simple map representations on mobile phones. And third, it gives novel insights into supporting the embedded practices of a DIY (Do-It-Yourself) urban gardening community with interactive systems based on ethnographic fieldwork and participatory design.

In this paper, we investigate how smartphone applications, in particular web browsers, are used on mobile phones. Using a publicly available widget for smart phones, we recorded app usage and the phones' acceleration and orientation from 1,330 devices. Combining app usage and sensor data we derive the device's typical posture while different apps are used. Analyzing motion data shows that devices are moved more while messaging and navigation apps are used as opposed to browser and other common applications. The time distribution between landscape and portrait depicts that most of the landscape mode time is used for burst interaction (e.g., text entry), except for Media apps, which are mostly used in landscape mode. Additionally, we found that over 31% of our users use more than one web browser. Our analysis reveals that the duration of mobile browser sessions is longer by a factor of 1.5 when browsers are explicitly started through the system's launcher in comparison to being launched from within another app. Further, users switch back and forth between apps and web browsers, which suggest that a tight and smooth integration of web browsers with native apps can improve the overall usability. From our findings we derive design guidelines for app developers.

In this paper, we present the novel concept of ephemeral user interfaces. Ephemeral user interfaces contain at least one user interface (UI) element that is intentionally created to last for a limited time only and typically incorporate materials that evoke a rich and multisensory perception, such as water, fire, soap bubbles or plants. We characterize the term "ephemeral user interface" and, based on a review of existing user interfaces that fall into this research area but have not been discussed under one common term before, we present a design space for ephemeral user interfaces providing a terminology for (a) materials for ephemeral UI elements, (b) interaction and (c) aspects of ephemerality. This paper contributes to the ongoing research on materiality of user interfaces as well as on conceptualizing visionary interaction styles with novel materials.

This paper introduces the toolkit ASED: Assistive System Experiment Designer. Combining a specially constructed assembly table and new software it allows measuring the performance of impaired persons when using assistive systems for production environments (ASiPE). The ASiPE design tested using ASED transgresses the state of the art by three enhancements. With the help of ASED we are able to quantify and rank their effects on work quality and performance. The ASED toolkit, however, is not confined to the design tested but can be used for the experimental analysis of every kind of manual process.

In modern cars users need to interact with safety and comfort functions, driver assistance systems, and infotainment devices. Basic requirements include the perception of the current status and of information items as well as the control of functions. Handling that myriad amount of information while driving requires an appropriate interaction design, structure and visualization of the data. This paper investigates potentials and limitations of stereoscopic 3D for visualizing an in-vehicle information system. We developed a spatial in-car visualization concept that exploits three dimensions for the system's output. Based on a prototype, that implements the central functionality of our concept, we evaluate the 3D representation. A laboratory study with 32 users indicates that stereoscopic 3D is the better choice as it improves the user experience, increases the attractiveness, and helps the user in recognizing the current state of the system. The study shows no significant differences between non-stereoscopic and stereoscopic representations in the users' workload. This indicates that stereoscopic visualizations have no negative impact on the primary driving task.

Implementing controls in the car becomes a major challenge: The use of simple physical buttons does not scale to the increased number of assistive, comfort, and infotainment functions. Current solutions include hierarchical menus and multi-functional control devices, which increase complexity and visual demand. Another option is speech control, which is not widely accepted, as it does not support visibility of actions, fine-grained feedback, and easy undo of actions. Our approach combines speech and gestures. By using speech for identification of functions, we exploit the visibility of objects in the car (e.g., mirror) and simple access to a wide range of functions equaling a very broad menu. Using gestures for manipulation (e.g., left/right), we provide fine-grained control with immediate feedback and easy undo of actions. In a user-centered process, we determined a set of user-defined gestures as well as common voice commands. For a prototype, we linked this to a car interior and driving simulator. In a study with 16 participants, we explored the impact of this form of multimodal interaction on the driving performance against a baseline using physical buttons. The results indicate that the use of speech and gesture is slower than using buttons but results in a similar driving performance. Users comment in a DALI questionnaire that the visual demand is lower when using speech and gestures.

With computers being used ever more ubiquitously in situations where privacy is important, secure user authentication is a central requirement. Gaze-based graphical passwords are a particularly promising means for shoulder-surfing-resistant authentication, but selecting secure passwords remains challenging. In this paper, we present a novel gaze-based authentication scheme that makes use of cued-recall graphical passwords on a single image. In order to increase password security, our approach uses a computational model of visual attention to mask those areas of the image that are most likely to attract visual attention. We create a realistic threat model for attacks that may occur in public settings, such as filming the user's interaction while drawing money from an ATM. Based on a 12-participant user study, we show that our approach is significantly more secure than a standard image-based authentication and gaze-based 4-digit PIN entry.

Secure user authentication on mobile phones is crucial, as they store highly sensitive information. Common approaches to authenticate a user on a mobile phone are based either on entering a PIN, a password, or drawing a pattern. However, these authentication methods are vulnerable to the shoulder surfing attack. The risk of this attack has increased since means for recording high-resolution videos are cheaply and widely accessible. If the attacker can videotape the authentication process, PINs, passwords, and patterns do not even provide the most basic level of security. In this project, we assessed the vulnerability of a magnetic gestural authentication method to the video-based shoulder surfing attack. We chose a scenario that is favourable to the attack-er. In a real world environment, we videotaped the interactions of four users performing magnetic signatures on a phone, in the presence of HD cameras from four different angles. We then recruited 22 participants and asked them to watch the videos and try to forge the signatures. The results revealed that with a certain threshold, i.e, th=1.67, none of the forging attacks was successful, whereas at this level all eligible login attempts were successfully recognized. The qualitative feedback also indicated that users found the magnetic gestural signature authentication method to be more secure than PIN-based and 2D signature methods.

High-speed Internet connectivity makes browsing a convenient task. However, there are many situations in which surfing the web is still slow due to limited bandwidth, slow servers, or complex queries. As a result, loading web pages can take several seconds, making (mobile) browsing cumbersome. We present an approach which makes use of the time spent on waiting for the next page, by bridging the wait with extra cached or preloaded content. We show how the content (e.g., news, Twitter) can be adapted to the user's interests and to the context of use, hence making mobile surfing more comfortable. We compare two approaches: in time-multiplex mode, the entire screen displays bridging content until the loading is finished. In space-multiplex mode, content is displayed alongside the requested content while it loads. We use an HTTP proxy to intercept requests and add JavaScript code, which allows the bridging content from websites of our choice to be inserted. The approach was evaluated with 15 participants, assessing suitable content and usability.

In public places we can observe that many conventional displays are replaced by digital displays, a lot of them networked. These displays mainly show advertising in a similar way to television's commercial break not exploiting the opportunities of the new medium [1]. Several approaches of interaction between mobile devices and public displays have been investigated over the last 15 years. In this demo we concentrate on challenges that are specific to public displays used for advertising. In particular we focus on how new approaches for interaction with content, means for content creation, and tools for follow-ups can be implemented based on mobile devices. With Digifieds we present a research system that has been used to explore different research questions and to showcase the potential of interactive advertising in public space.

Mobile phones have become pervasive in the developing world and are much more affordable and accessible than computers. For many children and teenagers, phones have become a familiar tool for communication as well as for multimedia creation and consumption. We run a two-week field study with 78 students and 6 teachers, where the use of mobile phones was integrated to the regular school days in the rural Panama. Based on our findings, we present recommendations how the applications and the general phone user interface design could be improved to better support teaching and learning tasks. The results show that there is potential for a tighter integration of applications to improve the learning experience.

Natural user interfaces by means of gesture and speech interaction have become a hot topic in research as well as already for real products. Most use cases currently center around consumer electronics devices like smart phones, TV sets, gaming, or other large screens like tabletops.

During the last decade, the number of functions of automotive user interfaces has increased rapidly. Besides traditional controls to drive a car, driver assistance, infotainment, entertainment, and comfort systems need to be controlled while driving. This does not only affect the driver's cognitive workload but also leads to increased complexity in designing automotive user interfaces. In this paper, we provide models and tools for rapid prototyping and the evaluation of user interfaces in this context. Usually, functional prototypes of user interfaces are implemented that allow the usability and quality to be assessed with time-consuming user studies. In contrast, in our approach we use an adapted Keystroke-Level Model (KLM) that is based on empirically collected data for typical operations in the car. It takes into account the aspect of attention switching in the car between primary tasks and other tasks. We present KLM operator times that we determined in a user study as well as a formula for estimating the task completion time. The presented model is the foundation for the MI-AUI prototyping tool that we implemented for permitting the creation of automotive interfaces using tangible controls. By demonstrating a typical operation with the MI-AUI prototype, the estimated task completion time can be calculated. MI-AUI is an evaluation tool that can be quickly and easily applied in early stages of the design process without the need to involve real drivers.

Providing good education is one of the major challenges for humanity. In many developing regions in the world improving educational standards is seen as a central building block for improving socio-economic situation of society. Based on our research in Panama we report on how mobile phones can be used as educational tools. In contrast to personal computers mobile phones are widely available and in Panama over 80% of the children have access to phones. We report on four different studies building on one another. We conducted surveys, focus groups, and group interviews with several hundred teachers and pupils to assess opportunities, needs, and threads for using phones in teaching and learning. Based on the feedback received we created a set of use cases and finally evaluated these in a field study in a rural multigrade school in Panama. Our findings suggest that current phones with multimedia capabilities provide a valuable resource for teaching and learning across many subjects. In particular recording of audio and video, programs for drawing, and taking photos were used in very creative and constructive ways beyond the use cases envisioned by us and initial skepticism of parents turned into support.

Non-planar screens, such as columns, have been a popular means for displaying information for a long time. In con-trast to traditional displays their digital counterparts are mainly flat and rectangular due to current technological constraints. However, we envision bendable displays to be available in the future, which will allow for creating new forms of displays with new properties. In this paper we ex-plore cylindrical displays as a possible form of such novel public displays. We present a prototype and report on a user study, comparing the influence of the display shape on user behavior and user experience between flat and cylindrical displays. The results indicate that people move more in the vicinity of cylindrical displays and that there is no longer a default position when it comes to interaction. As a result, such displays are especially suitable to keep people in motion and to support gesture-like interaction.

Cars offer an increasing number of infotainment systems as well as comfort functions that can be controlled by the driver. In our research, we investigate new interaction techniques that aim to make it easier to interact with these systems while driving. We suggest utilizing the steering wheel as an additional interaction surface. In this paper, we present two user studies conducted with a working prototype of a multi-touch steering wheel. In the first, we developed a user-defined steering wheel gesture set, and in the second, we applied the identified gestures and compared their application to conventional user interaction with infotainment systems in terms of driver distraction. The main outcome was that driver's visual demand is reduced significantly by using gestural interaction on the multi-touch steering wheel.

Even with the rise of the World Wide Web, TV has remained the most pervasive entertainment medium and is nowadays often used together with other media, which allow for active participation. The idea of connecting non-collocated TV viewers via telecommunication technologies, referred to as Social TV, has recently received considerable attention. Such systems typically include set-top boxes for supporting collaboration. In this research we investigate if real-time opinion sharing about TV shows through a nonverbal (non-textual) iconic UI on mobile phones is reasonable. For this purpose we developed a mobile app, made it available to a large number of users through the Android Market, and conducted an uncontrolled user study in the wild during the soccer world cup 2010. The results of the study indicate that TV viewers who used the app had more fun and felt more connected to other viewers. We also show that by monitoring this channel it is possible to collect sentiments relevant to the broadcasted content in real-time. The collected data exemplify that the aggregated sentiments correspond to important moments, and hence can be used to generate a summary of the event.

"In all the wonderful worlds that writing opens, the spoken word still resides and lives. Written texts all have to be related somehow, directly or indirectly, to the world of sound, the natural habitat of language, to yield their meanings."

Currently, the development of mobile applications heavily relies on using conventional computers as development platform. MobiDev enables people in emerging countries without access to a computer but to a cell phone to develop their own locally relevant applications. The goal of the Mo-biDev project is to simplify development and deployment of applications directly on mobile phones. As a first step, we focus on the design of applications and try to support the computer science curriculum in developing countries to bootstrap the mobile developer culture and community. MobiDev allows the creation of graphical user interfaces (GUI) using various concepts. We present the results of a first system evaluation that show how people perceive the concepts for UI creation of MobiDev.

In past years, ubiquitous computing technologies have been successfully deployed for supporting children with special needs. One focus of current research has been on post-hoc behavior analysis based on video footage where one or multiple cameras were used to review situations in which children behaved in a certain way. As miniaturized cameras as well as portable devices are becoming available at low costs, we envision a new quality in supporting the diagnosis, observation, and education of children with special needs. In contrast to existing approaches that use cameras in fix locations, we suggest to use multiple mobile camera perspectives. In this way observation data from fellow classmates, teachers, and caregivers can be considered, even in highly dynamic outdoor situations. In this paper we present MuPerBeAn, a platform that allows multi-perspective video footage from mobile cameras to be collected, synchronously reviewed, and annotated. We report on interviews with caregivers and parents and present a qualitative study based on two scenarios involving a total of seven children with autism (CWA). Our findings show that observing multiple mobile perspectives can help children as well as teachers to better reflect on situations, particularly during education.

Traditional public notice areas (PNAs) are nowadays a popular means to publicly exchange information and reach people of a local community. The high usability led to a wide-spread use in stores, cafes, supermarkets, and public institutions. With public displays permeating public spaces and with display providers and owners being willing to share parts of their display space we envision traditional PNAs to be complemented or even replaced by their digital counterparts in the future, hence contributing to making public displays a novel communication medium. In this paper we report on the design and development of Digifieds (derived from digital classified), a digital public notice area. We deployed and evaluated Digifieds in an urban environment in the context of the UbiChallenge 2011 in Oulu, Finland over the course of 6 months. The deployment allowed the users' view to be studied with regard to the envisioned content, preferred interaction techniques, as well as privacy concerns, and to compare them against traditional PNAs.

People spend a significant amount of time in their cars (US: 86 minutes/day, Europe: 43 minutes/day) while commuting, shopping, or traveling. Hence, the variety of entertainment in the car increases, and many vehicles are already equipped with displays, allowing for watching news, videos, accessing the Internet, or playing games. At the same time, the urbanization caused a massive increase of traffic volume, which led to people spending an ever-increasing amount of their time in front of red traffic lights. An observation of the prevailing forms of entertainment in the car reveals that content such as text, videos, or games are often a mere adaptation of content produced for television, public displays, PCs, or mobile phones and do not adapt to the situation in the car. In this paper we report on a web survey assessing which forms of entertainment and which types of content are considered to be useful for in-car entertainment by drivers. We then introduce an algorithm, which is capable of learning standing times in front of traffic lights based on GPS information only. This, on one hand, allows for providing content of appropriate length, on the other hand, for directing the attention of the driver back to-wards the street at the right time. Finally, we present a prototype implementation and a qualitative evaluation.

Interaction with communication and infotainment systems in the car is common while driving. Our research investigates modalities and techniques that enable interaction with interactive applications while driving without compromising safety. In this paper we present the results of an experiment where we use eye-gaze tracking in combination with a button on the steering wheel as explicit input substituting the interaction on the touch screen. This approach combines the advantages of direct interaction on visual displays without the drawbacks of touch screens. In particular the freedom of placement for the screen (even out of reach from the user) and that both hands remain on the steering wheel are the main advantages. The results show that this interaction modality is slightly slower and more distracting than a touch screen but it is significantly faster than automated speech interaction.

In this paper, we explore the potential of combining shared and interactive displays (e.g. a multi-touch table) with personal devices (e.g. mobile phones) as an important class of heterogeneous multi-display environments. Within six case studies applications and interactions were invented and implemented that utilize the potential of such heterogeneous multi-display environments. We were in particular interested how to design systems that include interaction across different displays and how to manage public and private information in a group setting. One case study, a digital card game, highlights these design challenges. We explore different natural ways of interaction, including touching the table as well as motion gestures with mobile phones. With this application we provide a use case to discuss gestures combining mobile phones with tabletop surfaces, as well as to explore a private-public display setting. First results showed that combining tables and mobile phones provide a suitable and understandable way for interaction in these settings.

Many tasks require attention switching. For example, searching for information on one sheet of paper and then entering this information onto another one. With paper we see that people use fingers or objects as placeholders. Using these simple aids, the process of switching attention between displays can be simplified and speeded up. With large or multiple visual displays we have many tasks where both attention areas are on the screen and where using a finger as a placeholder is not suitable. One way users deal with this is to use the mouse and highlight their current focus. However, this also has its limitations -- in particular in environments where there is no pointing device. Our approach is to utilize the user's gaze position to provide a visual placeholder. The last area where a user fixated on the screen (before moving their attention away) is highlighted; we call this visual reminder a Gazemark. Gazemarks ease orientation and the resumption of the interrupted task when coming back to this display. In this paper we report on a study where the effectiveness of using Gazemarks was investigated, in particular we show how they can ease attention switching. Our results show faster completion times for a resumed simple visual search task when using this technique. The paper analyzes relevant parameters for the implementation of Gazemarks and discusses some further application areas for this approach.

Despite the availability of other mobile messaging applications, SMS has kept its position as a heavily used communication technology. However, there are many situations in which it is inconvenient or inappropriate to check a message's content immediately. In this paper, we introduce the concept of audio previews of SMS. Based on a real-time analysis of the content of a message, we provide auditory cues in addition to the notification tone upon receiving an SMS. We report on a field trial with 20 participants and show that the use of audio-enhanced SMS affects the reading and writing behavior of users. Our work is motivated by the results of an online survey among 347 SMS users of whose we analyzed 3400 text messages.

Research has shown that product reviews on the Internet not only support consumers when shopping, but also lead to increased sales for retailers. Recent approaches successfully use smart phones to directly relate products (e.g. via barcode or RFID) to corresponding reviews, making these available to consumers on the go. However, it is unknown what modality (star ratings/text/video) users consider useful for creating reviews and using reviews on their mobile phone, and how the preferred modalities are different from those on the Web. To shed light on this we conduct two experiments, one of them in a quasi-realistic shopping environment. The results indicate that, in contrast to the known approaches, stars and pre-structured text blocks should be implemented on mobile phones rather than long texts and videos. Users prefer less and rather well-aggregated product information while on the go. This accounts both for entering and, surprisingly, also for using product reviews.

Digital immersion is moving into public space. Interactive screens and public displays are deployed in urban environments, malls, and shop windows. Inner city areas, airports, train stations and stadiums are experiencing a transformation from traditional to digital displays enabling new forms of multimedia presentation and new user experiences. Imagine a walkway with digital displays that allows a user to immerse herself in her favorite content while moving through public space. In this paper we discuss the fundamentals for creating exciting public displays and multimedia experiences enabling new forms of engagement with digital content. Interaction in public space and with public displays can be categorized in phases, each having specific requirements. Attracting, engaging and motivating the user are central design issues that are addressed in this paper. We provide a comprehensive analysis of the design space explaining mental models and interaction modalities and we conclude a taxonomy for interactive public display from this analysis. Our analysis and the taxonomy are grounded in a large number of research projects, art installations and experience. With our contribution we aim at providing a comprehensive guide for designers and developers of interactive multimedia on public displays.

While there is more to context than location, localization and positioning must continue to be improved. Location-aware applications, such as Google Latitude, are enjoying great popularity. Location-based applications and services are widely used on platforms such as the iPhone. The localization itself thus remains an issue, especially in indoor scenarios. In this paper we discuss Digital Enhanced Cordless Telecommunications (DECT) and its potential for positioning and localization. While the technology itself is not new, the application of DECT for localization has so far not been thoroughly investigated, especially for combined indoor and outdoor usage. We have collected and analyzed large data sets of DECT and WLAN fingerprints in urban, sub-urban and rural areas, both for indoor and outdoor situations. We show that DECT could improve the accuracy and robustness of existing localization schemes based on WLAN or GSM.

In this paper we inquire material qualities for tangible user interfaces and introduce the term ephemeral user interface. We present an example user interface that employs fragile soap bubbles for human-computer interaction: the user has to move or blow bubbles over a dark liquid surface in order to interact with a computer system. Our installation uses liquid, smoke, and soap bubbles as unusual and transient materials that demand a very cautious and calm handling, and furthermore elude the complete control of users, as the bubbles inevitably burst after a short while. Thus, the soap bubble interface provokes thoughts about durability, control, and materiality of tangible user interfaces.

Over the last 100 years it has become much easier to operate a car. However in recent years the number of functions a user can control while driving has greatly increased. Infotainment, entertainment and comfort systems as well as driver assistance contribute to this trend. Interaction with these systems plays an important role, as on one hand this can improve the user experience while driving but on the other hand it may distract from the primary task of driving. User interfaces in cars differ regarding the number of input and output devices and their placement in the car to a great extent. In this paper, we introduce a first design space for driver-based automotive user interfaces that allows a comprehensive description of input and output devices in a car with regard to placement and modality. This design space is intended to provide a basis for analyzing and discussing different user interface arrangements in cars, to compare alternative user interface setups, and to identify new opportunities for interaction and placement of controls. We present a graphical representation of the design space and discuss its usage in detail based on several examples. To assess the completeness of the proposed design space we used it to classify and compare user interfaces from more than 100 cars shown at IAA2007, cars from the BMW museum, and from the A2Mac1 image database.

Context-awareness is one of the rising trends of future mobile technology, and due to advances in technology development, new application and service concepts are being developed and demonstrated in an ever-increasing manner. This workshop brings together researchers and practitioners working on humancomputer interaction (HCI) aspects of context-aware mobile technology and communities to present their insights and research on new concepts, interaction design for mobile context-awareness, usability challenges, collaborative context-aware services and applications for supporting communities, and other topics related to HCI with mobile context-aware technology.

In their role as personal communication devices, mobile phones are a natural choice for sharing and communicating emotions. However, their functionalities are currently very limited in power to express affective messages. In this paper, we describe the design of a system that allows users to easily compose melodies and share them via mobile phones. We show that by using these melodies information about the current emotional state of the sender can be expressed and recognized synchronously by the receiver in a simple, quick, and unobtrusive way. Further, we reveal that self-composed melodies have a stronger impact than pre-composed or downloaded messages, similar to crafted pieces of art offered to a beloved person. We then present findings from a user study that assesses the implementation of a functional prototype and the adequacy of the system for emotional communication.

In this paper, we introduce the design and implementation of a digital card game that combines mobile phone gestures and multi-touch tabletop interactions. Conceptually based on interactions in traditional card games, our Poker game application explores different natural ways of interaction, including touching the table as well as tilting, throwing, and shaking a mobile phone. By exemplarily translating traditional gestures into the digital domain, we provide a use case to discuss useful gestures combing mobile phones with tabletop surfaces, as well as to explore a private-public display setting for digital card games at interactive tables. Additionally, the mobile phone provides a tangible feeling similar to physical cards. The preliminary user study showed that users preferred using mobile phones for interaction compared to direct interaction on the multi-touch table.

In this paper, we introduce Flashlight interaction, a new approach to light-based interaction between mobile phones and large screens by using the phone camera flashlight. Using a mobile phone as an interaction device with large screens has been explored in various projects. Mobile phones are enhanced with different sensors and provide opportunities to be used as an interaction device with other devices such as a large display. Our approach supports to set up a private-public display setting and provide simple means for interaction without a wireless connectivity. Our user study results indicate that the interaction is easy to perform and understand due to the direct mapping between the phone movement and the response on the screen.

Customer reviews and recommendations for products are provided by almost all e-business platforms, supporting consumers when shopping on the web. Mobile and ubiquitous computing provide extended means to sense input data for recommendations and to make recommendations available for consumers when shopping in traditional stores. This work contributes a comprehensive design space that outlines design options for product recommendation systems using mobile and ubiquitous technologies. A visual notation for the design space is proposed, based on which existing systems are categorized. Blank spaces are identified and concrete possible extensions are proposed by the example of an existing mobile product recommendation system. Finally, general options for future research on product recommendation systems using UbiComp technologies are discussed.

Increasing functionality, growing media volumes and dynamic data in today's in-vehicle information systems bear new challenges for user interaction design. Traditional hierarchical and menu-based interaction can only provide limited support while new search-based approaches are promising. In this work we assess different search techniques and search-based user interfaces. In particular we compare free search across all data items with categorized search. Our experiments with functional prototypes show that free search is more efficient and easier to use than searching within categories. Tests in a driving simulator show promising results regarding safety and workload. Means for alphanumeric input appear to be essential for an efficient and safe search interaction while driving.

Wearable computing and smart clothing have attracted a lot of attention in the last years. For a variety of applications, it can be seen as potential future direction of mobile user interfaces. In this paper, we concentrate on usability and applicability issues concerned with capacitive touch input on clothing. To be able to perform user studies, we built a generic platform for attaching, e.g., capacitive sensors of different types. On top of that, several prototypes of wearable accessories and clothing and implemented various application scenarios. We report on two studies we undertook with these implementations with a user group randomly sampled at a shopping mall. We provide a significant set of guidelines and lessons learned that emerged from our experiences and those studies. Thus, developers of similar projects have to put major efforts into minimizing the delay between button activation and feedback and to make location and identification of controls and their function as simple and quick as possible. Issues that have to be treated in all designs include the requirement of one-handed interaction and that, even for minimal functionality, to find a general solution with regard to layout and button-to-function mapping is hardly possible. Additionally, in order to generate a satisfactory user experience good usability must be combined with aesthetical factors.

The last few years have seen the emergence of two parallel trends: the first of such trends is set by technologies such as Near Field Communication, 2D Bar codes and RFID that support the association of digital information with virtually every object. By using these technologies, ordinary objects such as coffee mugs or advertisement posters provide digital information that can be easily processed. The second trend is set by (semantic) Web services that provide a way to automatically invoke functionalities across the Internet lowering interoperability barriers. The PERCI system, discussed in the article, provides a bridge between these two technologies allowing the invocation of Web services using the information gathered from the tags to effectively transform every object in a service proxy.

In this paper, we introduce an implementation for detailed monitoring of user actions on web pages. It addresses the problem that the log data recorded by standard web servers is not sufficient for the tracking of users on AJAX websites, e.g. to conduct a usability test. Using standard web technologies, our HTTP proxy can record very detailed usage information, such as mouse movements, clicks, key presses and scrolling, together with the exact HTML DOM tree objects involved. As we show in several case studies, the tracking also works across multiple websites, none of which needs to be under our control. This approach is much less invasive than previous efforts: The test person does not need to install software on her computer, and in certain operation modes, no configuration changes at all are required on her computer. Our research indicates that if the technology described in this paper is employed, arbitrary visitors of a website are more likely to take part in a usability test offered by that site -- this facilitates recruiting test participants over the Internet.

The design of applications using mobile devices needs a different quality assessment than those known for desktop applications. Of the many aspects that have to be taken into account, one important criterion is the average time users need to complete a task. For interactions with the mouse, keyboard or touch screens, there exist models that predict interaction times like Fitts' law or the Keystroke-Level Model (KLM). This paper shows parallels to these models for advanced interactions with mobile phones targeted at pervasive services, including near field communication as well as built-in cameras and sensors. Applications can be evaluated with respect to user performance time without having a prototype running on the phone. To accomplish that, we extend the known KLM by identifying basic interaction elements for mobile phones and give estimates for expert user performance derived from several user tests.

The proposed demonstration is based on the work performed in the PERCI project and presents a generic framework to access and interact with Web Services through mobile interaction with real world objects. The demonstration will put a focus on the front-end of the framework that comprises augmented posters for mobile ticketing as well as a mobile client application for the interaction with associated services. By supporting Physical Mobile Interaction techniques such as Touching or Pointing, the framework tries to make mobile service interaction more intuitive and shift its focus from the menus of mobile devices to augmented physical objects.

The proposed demonstration offers the possibility to access and interact with (web) services through Physical Mobile Interaction (PMI) with tag-based dynamic physical objects such as papers. Our demonstration supports several direct manipulation gestures in the real world and deals with physical objects able to provide system feedback to the user during the course of the interaction.

Although mobile services can be used ubiquitously, their employment and the interaction with them are still restricted by the constraints of mobile devices. In order to facilitate and leverage mobile interaction with services, we present a generic framework that combines semantic Web service technology and physical mobile interaction. This interaction paradigm uses mobile devices to extract information from augmented physical objects and use it for a more intuitive and convenient invocation of associated services. For that purpose, the presented framework exploits Web service descriptions for the automatic and dynamic generation of customizable user interfaces that support and facilitate physical mobile interaction. This generic approach to mobile interaction with services through the interaction with physical objects promises to meet the complementary development of the Internet of things. A user study with a prototype application for mobile ticketing confirms our concept and shows its limits

In this paper, we report on a user-centered, iterative design process for augmenting sports equipment with ubiquitous computing technology. In several design iterations, a fully working system for training and physiotherapy has been developed and deployed using hard- and software components of ubiquitous computing technology. We report on the design and development process that led to this system. Based on our experience we generalize the specific processes to general ubicomp systems. The validity of our approach has been verified by a larger user study

Bringing ubiquitous computing applications to home environments is a great challenge. In our research we investigate how applications can be conceived, designed, and implemented that fit into people's lives. In particular we describe our experiments on how methods of user centered design and participatory design can be appropriated to find users' requirements and design ideas for ubiquitous computing applications for the home. In particular we focus in the study reported on information presentation using display appliances. In a participatory design process, enhanced with technology probes, we discussed potential solutions for a specific home with 14 participants individually. Based on these individual solutions, fitting a single persona each, we specified the prototypes that would accommodate the user's needs and that are generic in its applicability. This allows drawing a first set of guidelines for the design of display appliances in the home environment

In this paper we present a toolkit for realizing capacitive sensing applications for human-computer interaction in pervasive computing systems. We argue that capacitive sensors - due to their unique properties - are well suited for many pervasive and ubiquitous computing applications and scenarios. We describe the CapToolKit designed to rapidly realize prototypes and systems that are able to detect the presence of humans and objects. Our toolkit also allows the integration of 3D interaction with everyday objects as well as instrumented environments. We illustrate its capabilities by presenting several applications implemented using CapToolKit. The entire system will be open-sourced to allow utilization of our technology within other research projects. By building on the existing toolkit researchers are provided a foundation for developing their own sensor systems, algorithms, and applications

We report on approaches for context-awareness in a kitchen environment. Two devices, an augmented cutting board and a sensor-enriched knife, enable the environment to determine the type of food handled during the preparation of meals.

Design tools that integrate hardware and software components facilitate product design work across aspects of physical form and user interaction, but at the cost of requiring designers to work with other than their accustomed programming tools. In this paper we introduce VoodooFlash, a tool designed to build on the widespread use of Flash while facilitating design work across physical and digital components. VoodooFlash extends the existing practice of authoring interactive applications in terms of arranging components on a virtual stage, and provides a physical stage on which controls can be arranged, linked to software components, and appropriated with other physical design materials.

In this paper, we investigate how detailed tracking of user interaction can be monitored using standard web technologies. Our motivation is to enable implicit interaction and to ease usability evaluation of web applications outside the lab. To obtain meaningful statements on how users interact with a web application, the collected information needs to be more detailed and fine-grained than that provided by classical log files. We focus on tasks such as classifying the user with regard to computer usage proficiency or making a detailed assessment of how long it took users to fill in fields of a form. Additionally, it is important in the context of our work that usage tracking should not alter the user's experience and that it should work with existing server and browser setups. We present an implementation for detailed tracking of user actions on web pages. An HTTP proxy modifies HTML pages by adding JavaScript code before delivering them to the client. This JavaScript tracking code collects data about mouse movements, keyboard input and more. We demonstrate the usefulness of our approach in a case study.

This second workshop on Multi-User and Ubiquitous User Interfaces aims at further investigating two major issues identified at last year's MU3I: control and consistency. The former relates to how a user gains control of devices in a ubiquitous computing environment, how control is passed, and how it is shared in such a setting. The second one concerns interfaces that span multiple devices or move from one set of devices to another. Both issues will be discussed in this year's workshop (with a focus on consistency.

Movements of interfaces can be analyzed in terms of whether they are expected, sensed, and desired. Expected movements are those that users naturally perform; sensed are those that can be measured by a computer; and desired movements are those that are required by a given application. We show how a systematic comparison of expected, sensed, and desired movements, especially with regard to how they do not precisely overlap, can reveal potential problems with an interface and also inspire new features. We describe how this approach has been applied to the design of three interfaces: pointing flashlights at walls and posters in order to play sounds; the Augurscope II, a mobile augmented reality interface for outdoors; and the Drift Table, an item of furniture that uses load sensing to control the display of aerial photographs. We propose that this approach can help to build a bridge between the analytic and inspirational approaches to design and can help designers meet the challenges raised by a diversification of sensing technologies and interface forms, increased mobility, and an emerging focus on technologies for everyday life.

This paper presents a low-cost and practical approach to achieve basic input using a tactile cube-shaped object, augmented with a set of sensors, processor, batteries and wireless communication. The algorithm we propose combines a finite state machine model incorporating prior knowledge about the symmetrical structure of the cube, with maximum likelihood estimation using multivariate Gaussians. The claim that the presented solution is cheap, fast and requires few resources, is demonstrated by implementation in a small-sized, microcontroller-driven hardware configuration with inexpensive sensors. We conclude with a few prototyped applications that aim at characterizing how the familiar and elementary shape of the cube allows it to be used as an interaction device.

All in-text\treferences\tunderlined\tin\tblue\tare\tlinked\tto\tpublications\ton\tResearchGate, letting you\taccess\tand\tread\tthem\timmediately.