Human Motion Simulation (HMS) predicts and visualizes how virtual agents move and behave in various scenarios, aiding decision-making. However, it does not communicate the experience behind agents' behavior. To address this, we present Embodied Spatial Simulation (ESS), a concept combining augmented reality(AR) and HMS to support in-situ decision-making. It enables an XR user to create "what-if" scenarios, simulate virtual agents' behavior in situ, and experience the scenarios from their perspectives through embodiment. To demonstrate ESS, we developed a proof-of-concept prototype featuring two scenarios (e.g., fire evacuation planning). From expert walkthroughs (N=4) and a survey (N=38) with different professions, we found that incorporating embodiment into simulations can change users' intentions to use simulation outcomes. Rather than directly obtaining solutions from the results, users showed more interest in understanding the experience behind simulated behavior and using this insight to shape decisions. We discuss the potential of ESS to support future human-centered design practices.

With the recent advancements in Large Language Models (LLMs), web developers increasingly apply their code-generation capabilities to website design. However, since these models are trained on existing designerly knowledge, they may inadvertently replicate bad or even illegal practices, especially deceptive designs (DD). This paper examines whether users can accidentally create DD for a fictitious webshop using GPT-4. We recruited 20 participants, asking them to use ChatGPT to generate functionalities (product overview or checkout) and then modify these using neutral prompts to meet a business goal (e.g., “increase the likelihood of us selling our product”). We found that all 20 generated websites contained at least one DD pattern (mean: 5, max: 9), with GPT-4 providing no warnings. When reflecting on the designs, only 4 participants expressed concerns, while most considered the outcomes satisfactory and not morally problematic, despite the potential ethical and legal implications for end-users and those adopting ChatGPT’s recommendations.

AI-powered systems pose unknown challenges for designers, policymakers, and users, making it more difficult to assess potential harms and outcomes. Although understanding risks is a requirement for building trust in technology, users are often excluded from risk assessments and explanations in policy and design. To address this issue, we conducted three workshops with 18 participants and discussed the EU AI Act, which is the European proposal for a legal framework for AI regulation. Based on results of these workshops, we propose a user-centered conceptual model with five risk dimensions (Design and Development, Operational, Distributive, Individual, and Societal) that includes 17 key risks. We further identify six criteria for categorizing use cases. Our conceptual model (1) contributes to responsible design discourses by connecting the risk assessment theories with user-centered approaches, and (2) supports designers and policymakers in more strongly considering a user perspective that complements their own expert views.

Dark patterns are deceptive designs that influence a user’s interactions with an interface to benefit someone other than the user. Prior work has identified dark patterns in windows, icons, menus, and pointer (WIMP) interfaces and ubicomp environments, but how dark patterns can manifest in Augmented and Virtual Reality (collectively XR) requires more attention. We therefore conducted 10 co-design workshops with 20 experts in XR and deceptive design. Our participants co-designed 42 scenarios containing dark patterns, based on application archetypes presented in recent HCI/XR literature. In the co-designed scenarios, we identified 10 novel dark patterns in addition to 39 existing ones, as well as 10 examples in which specific characteristics associated with XR potentially amplified the effect dark patterns could have on users. Based on our findings and prior work, we present a classification of XR-specific properties that facilitate dark patterns: perception, spatiality, physical/virtual barriers, and XR device sensing. We also present the experts’ assessments of the likelihood and severity of the co-designed scenarios and highlight key aspects they considered for this evaluation, for example, technological feasibility, ease of upscaling and distributing malicious implementations, and the application’s context of use. Finally, we discuss means to mitigate XR dark patterns and support regulatory bodies to reduce potential harms.

Foreseeing the impact of augmented and virtual reality applications on users and society is challenging. Thus, efforts to establish an ethical mindset and include technology assessment techniques in HCI education are increasing. However, XR educational courses fostering students’ reasoning and perceived responsibility in designing ethical applications are lacking. We, therefore, developed the explorative design method Reality Composer to investigate and foster the students’ assessment of the ethical impact of and responsibilities in XR application design. We conducted a workshop with 40 international HCI master students applying this method and analyzed the resulting application concepts regarding the students’ ethical assessment. Our findings show that they critically discussed their concepts’ impact and identified potential countermeasures for negative social implications. However, they overestimated the users’ responsibility to securely use XR applications as well as a positive design intention. We contribute with our findings and developed method to understand students’ potential and derive future course design implications.

Abstract Although Voice Assistants are ubiquitously available for some years now, the interaction is still monotonous and utilitarian. Sound design offers conceptual and methodological research to design auditive interfaces. Our work aims to complement and supplement voice interaction withsonic overlaysto enrich the user experience. Therefore, we followed a user-centered design process to develop a sound library for weather forecasts based on empirical results from a user survey of associative mapping. After analyzing the data, we created audio clips for seven weather conditions and evaluated the perceived combination of sound and speech with 15 participants in an interview study. Our findings show that supplementing speech with soundscapes is a promising concept that communicates information and induces emotions with a positive affect for the user experience of Voice Assistants. Besides a novel design approach and a collection of sound overlays, we provide four design implications to support voice interaction designers.

Intelligent Personal Assistants (IPA) are advertised as reliable companions in the everyday life to simplify household tasks. Due to speech-based usability issues, users struggle to deeply engage with current systems. The capabilities of newer generations of standalone devices are even extended by a display, also to address some weaknesses like memorizing auditive information. So far, it is unclear how the potential of a multimodal experience is realized by designers and appropriated by users. Therefore, we observed 20 participants in a controlled setting, planning a dinner with the help of an audio-visual-based IPA, namely Alexa Echo Show. Our study reveals ambiguous mental models of perceived and experienced device capabilities, leading to confusion. Meanwhile, the additional visual output channel could not counterbalance the weaknesses of voice interaction. Finally, we aim to illustrate users’ conceptual understandings of IPAs and provide implications to rethink audiovisual output for voice-first standalone devices.

Current research in augmented, virtual, and mixed reality (XR) reveals a lack of tool support for designing and, in particular, prototyping XR applications. While recent tools research is often motivated by studying the requirements of non-technical designers and end-user developers, the perspective of industry practitioners is less well understood. In an interview study with 17 practitioners from different industry sectors working on professional XR projects, we establish the design practices in industry, from early project stages to the final product. To better understand XR design challenges, we characterize the different methods and tools used for prototyping and describe the role and use of key prototypes in the different projects. We extract common elements of XR prototyping, elaborating on the tools and materials used for prototyping and establishing different views on the notion of fidelity. Finally, we highlight key issues for future XR tools research.

Augmented/Virtual Reality (AR/VR) is still a fragmented space to design for due to the rapidly evolving hardware, the interdisciplinarity of teams, and a lack of standards and best practices. We interviewed 26 professional AR/VR designers and developers to shed light on their tasks, approaches, tools, and challenges. Based on their work and the artifacts they generated, we found that AR/VR application creators fulfill four roles: concept developers, interaction designers, content authors, and technical developers. One person often incorporates multiple roles and faces a variety of challenges during the design process from the initial contextual analysis to the deployment. From analysis of their tool sets, methods, and artifacts, we describe critical key challenges. Finally, we discuss the importance of prototyping for the communication in AR/VR development teams and highlight design implications for future tools to create a more usable AR/VR tool chain.

Over the last decades, different kinds of design guides have been created to maintain consistency and usability in interactive system development. However, in the case of spatial applications, practitioners from research and industry either have difficulty finding them or perceive such guides as lacking relevance, practicability, and applicability. This paper presents the current state of scientific research and industry practice by investigating currently used design recommendations for mixed reality (MR) system development. We analyzed and compared 875 design recommendations for MR applications elicited from 89 scientific papers and documentation from six industry practitioners in a literature review. In doing so, we identified differences regarding four key topics: Focus on unique MR design challenges, abstraction regarding devices and ecosystems, level of detail and abstraction of content, and covered topics. Based on that, we contribute to the MR design research by providing three factors for perceived irrelevance and six main implications for design recommendations that are applicable in scientific and industry practice.

This paper reports on the central sterile services department (CSSD) as a potential new design space for future research in Human-Computer Interaction (HCI) and Augmented Reality (AR). Within the last 2 years, we explored processes, tools and user needs in this field to identify use cases with the capability of enhancing everyday work using AR head-mounted displays (HMD). The conducted research was focused on the potential of applying AR technology as a proof of concept in which 8 problem-driven use cases were identified. These use cases enable interesting aspects for future investigation and utilization of new technologies. In addition to that, this paper describes the insights into user groups, their tasks, challenges, and needs for work support in this specific domain. Furthermore, a sample application is introduced which demonstrates the possibilities of HMD-based AR in the CSSD.