AI agents are increasingly explored for supporting reflection and well-being. However, we know little about how AI agents participate in reflective practices, particularly through co-creation. We presented MIRA, a co-creative AI agent that engages users in transforming abstract squiggles into concrete drawings while offering reflective feedback. Through a three-group comparative study, we examine how AI-mediated co-creation shapes reflective experience. We found that MIRA operates as a scaffold that introduces external reflective perspectives, helping users reinterpret experiences and surface emotions through visual expression. These findings highlight how AI agents can foster engaging co-creative experiences that encourage reflection among university students, while providing insights for designing AI-human co-creation that can extend to broader populations.

Generative AI (GenAI) is rapidly transforming design education, but universities primarily govern GenAI through macro-level policies that may fail to match micro-level course practices, leaving students and instructors without practical guidance. We reviewed 23 top-ranked university GenAI policy documents and 48 empirical studies of GenAI-integrated design courses, using inductive analysis to build paired thematic frameworks and then comparing policies and practices. Our findings reveal strong alignment around academic integrity, transparency, and the emphasis on critical thinking and human judgment. However, gaps appear with policies prioritizing institutional risk governance and academic writing support, whereas practices emphasize pragmatic benefits and iterative multimodal production. Additionally, policies stress independent learning and lifelong skill development, while practices highlight creativity and efficiency alongside persistent inquiry bottlenecks, overreliance, and fixation. We provide a macro-micro analytical lens linking policy to practice, integrative frameworks grounded in student and instructor perspectives, and actionable implications for university governance and course design.

The overreliance on large language models (LLMs)-generated answers poses risks to the development of learners’ critical thinking. Socratic instruction, which follows “tutor asks, student answers” approach, could mitigate overreliance by engaging learners with LLM-generated questions rather than passively seeking answers from LLMs. However, learners without effective response strategies often produce superficial answers and therefore undermine Socratic instruction. To bridge the gap, we first conducted a formative study (N=20) to analyze learners’ dialogue logs and interviews, deriving 18 Scaffolding Cards as response strategies to guide learners in framing their answers. A subsequent mixed-methods study (N=34) demonstrated that Scaffolding Cards improved critical thinking, optimized cognitive load allocation, and increased learning satisfaction compared to that without scaffolds. Our work reconfigures scaffolding by incorporating state-aware, agency-preserving, and function-transparent support. We further provide actionable implications for designing responsive and personalized scaffolding to facilitate learner-LLM interaction, introducing innovative perspectives for reclaiming learner agency in LLM-driven education.

Environmental User Experience (UX) data collection is essential for user research, enabling evidence-based design decisions. However, traditional retrospective methods like micro-phenomenological interviews suffer from recall inaccuracies and memory distortions. Concurrent UX data collection methods with environmental contexts are promising but lack in-depth investigation. To examine this potential, we conducted a formative study with 34 participants, identifying design goals such as natural interaction, in-situ annotation, and spatial-temporal coupling. We developed JourneyCapturer, an interactive tool that fulfills these goals to integrate concurrent annotation within Immersive Virtual Environment (IVE), enabling real-time UX data capture within contextual scenarios. Using a mixed-method design, we comparatively evaluated concurrent IVE annotations, retrospective interviews, and the combined method with 20 participants, demonstrating how JourneyCapturer improves UX collection processes and outcomes. Our findings suggest that a consciously proactive concurrent IVE method with a first-person perspective advances UX research, offering implications for expert collaboration, multi-modal analytics, and IVE-based field studies.

Large language model (LLM) based AI applications are being rapidly adopted by various creativity-related sectors, including UX design. However, as newly emerged applications, how UX designers work with LLMs and how to optimize the benefits of LLMs in UX design remain unclear. Leveraging the widely adopted LLM application ChatGPT as a case study, we recruited 12 experienced frontline UX practitioners from IT companies to use ChatGPT in their everyday design activities for 4 to 6 weeks. Through use logs, questionnaires, and interviews, we collected participants’ usage and reflection data. Our analysis reveals the different functions of ChatGPT in facilitating UX design, such as offering design guidelines, constructing user profiles, and simulating stakeholders. Challenges also emerged, such as limitations in understanding complex design problems and prototyping design ideas. Based on the findings, we reflect on the working dynamics between human workers and AI tools and suggest implications for designing more usable LLM-based AI applications.

Engaging the younger generation is crucial to the preservation and innovation of intangible cultural heritage (ICH). However, the technical barriers to ICH participation often deter youth generation. Generative Artificial Intelligence (GenAI) provides opportunities to lower barriers but risks undermining ICH authenticity and leading to over-reliance on automation driven by GenAI. To address these challenges, we aim to develop a human-GenAI co-creation system to support the transmission and innovation of ICH. Using Canton Porcelain as a case study, we collaborated with experts to design EnamelAI Painter that simplifies the participation process while promoting creativity. In a study with 19 participants, this system demonstrated advantages over existing AI and non-AI tools, enhancing creative engagement and user experience. EnamelAI Painter balanced GenAI automation with young novices’ engagement, preserving cultural authenticity and fostering interest in ICH. This work contributes a human-GenAI co-creation framework that provides actionable recommendations for the digital preservation of ICH.

AI-powered technologies are becoming increasingly integral to children’s digital experiences through devices like interactive toys, home automation systems, and apps, offering rich, personalized, and dynamic interactions. Despite their growing prevalence, how these AI-powered platforms can be designed to address the unique needs of children remains largely underexplored. Leveraging family interactions with Smart Voice Assistants (VAs) as a case study, we aim to explore how to approach child-centered AI (CCAI) design from a family perspective in this work. Specifically, we interviewed 20 parents and observed children’s VA interactions in eight households in a non-Western context. Using the theoretical lenses of agency and family functioning, we provide empirical insights into family dynamics when interacting with VAs in a less studied cultural setting, such as variations in family interaction types around VAs, the autonomy exercised by different parties, and the family functional roles VAs played. Based on these findings, we argue that CCAI design should be understood as balancing children’s agency, the roles and goals of other involved actors, and the contexts in which AI is used, and that it should focus on creating AI technologies that support positive outcomes for children in ethical ways while thoughtfully considering other stakeholders and their varying purposes for engaging with AI. In doing so, we offer a reconceptualization of CCAI and point to design directions for AI technologies that more meaningfully center child users in family contexts.

Waiting for system loading is a common scenario that often diminishes user experience, leading to dissatisfaction. Well-established visual indicators like progress bars can not directly apply to the interactions with voice assistants (VAs) like Siri. As VAs continue to rise in popularity, this research aims to explore the design of auditory indicators, particularly human speech, for optimizing waiting experiences in Voice User Interfaces (VUIs). We first organized focus groups (N=35) to identify design considerations for speech indicators, uncovering design opportunities in integrating explanations and humor. Subsequently, we conducted an empirical study (N=30) to evaluate the effects of speech indicators with two levels of explanation and humor on the waiting experience, measured by attention, perceived time, pleasure, and overall satisfaction, during both short and long loading durations. Our findings suggest significant potential for incorporating explanations and humor into VUIs, offering actionable insights for designing effective speech indicators that improve waiting experiences.

Immersive Virtual Environment (IVE) have demonstrated substantial potential in user research. However, the integration of IVE into Journey Map (JM) creation remains unexplored, as traditional JM creation still predominantly relies on 2D data. To bridge this research gap, we integrated IVE into the JM creation process, aiming to explore the potential of applying IVE in the JM creation process. Through a formative study with design experts ($\mathrm{N}=14$), six design goals were identified, reflecting designers' expectations for incorporating IVE into JM creation. Based on these goals, we developed ImmerJM, a tool that enables designers to analyze and document user journeys directly within immersive 3D environments. We conducted a comparative evaluation with User Experience (UX) participants ($\mathrm{N}=20$) to assess the JM creation process of ImmerJM against a traditional screen-based method. The quantitative and qualitative analyses revealed key improvements (e.g., phase-based JM mapping) in journey process and outputs. Our findings suggest that ImmerJM advances JM methodologies by leveraging spatial immersion, offering crucial implications for future JM creation and broader UX design practices involving IVE technologies.

Achieving high-quality output alongside enhanced controllability is crucial in video-to-music generation, especially for optimizing user experience in real-life application scenarios. Most existing studies emphasize generative quality, but often overlooking the vital aspect of controllability. Therefore, the generated music cannot be easily fine-tuned or modified to meet users' expectations. In this paper, we delve into the spatial-temporal decomposition and alignment in controllable video-to-music generation. We first introduce a novel video-music decomposition and transformation approach in both spatial and temporal domain, and enhance the cross-modal correspondence through feature alignment and flow-matching based alignment. Furthermore, our method attains unsupervised controllability during training via feature-free guidance. Experimental results demonstrate that our model achieves state-of-the-art results in overall generative quality. Moreover, its controllability significantly outperforms existing models, making it exceptionally well-suited to accommodate users' flexible and diverse control requirements.

Customized interiors enhance quality of life and self-expression, driving demand for VR-based design solutions. However, scant research exists on exploiting contextual cues in VR to aid decision making. Consequently, we propose CONDA, a context-aware assistant which leveraging LLMs to support interior renovation decisions. Specifically, we reconstruct users’ homes in VR and provide CONDA with stylistic details and spatial layouts, allowing it to predict furniture labels based on the decision scenario. Besides, we devise various modes to comprehensively express users’ purchasing preferences. Finally, CONDA recommend compatible items based on the label matching algorithm, and generate multi-dimensional explanations. A 30-user study reveals contextual completeness and preference diversity critically influence recommendation quality and decision behaviors, with 90% praising CONDA’s performance and all expressing daily-use intent. Overall, we validated the efficacy and practicality of CONDA, deriving universal design insights for VR decision-support systems and establishing new research directions.CCS ConceptsHuman-centered computing → Virtual realityComputing methodologies → Natural language generationApplied computing → Computer-aided design

The interaction between autonomous vehicles (AVs) and pedestrians has gained significant attention, leading to the exploration of external human-machine interfaces (eHMIs) equipped on AVs to facilitate effective communication. While existing research suggests that perceptions of vehicle appearances may influence interactions between pedestrians and AVs, a comprehensive study on the eHMIs related to AV appearance remains lacking. Therefore, we conducted a virtual reality (VR) experiment to investigate how AV appearances affect pedestrians’ acceptance regarding Awareness, Intent, and Harmony during interactions with various eHMIs. Leveraging the fuzzy set qualitative comparative analysis (fsQCA) method, we identified specific combinations of AV appearances and eHMIs that yield either high or low-performance interactions. For example, our findings reveal that text displays exhibit high performance in terms of awareness on AVs that are aggressive and ordinary. Furthermore, we distilled design guidelines to provide actionable suggestions for the design of eHMIs, fostering the acceptance of AVs among pedestrians.

Advances in artificial intelligence have led to breakthroughs in large language models (LLMs), like ChatGPT, opening up exciting possibilities for conceptual design. However, it’s essential to gain an in-depth understanding of how LLMs impact conceptual design output, process, and human designers’ perception. To this end, we chose ChatGPT as an example and conducted the investigation with 30 participants divided into Human-LLMs groups and human-human groups. The results indicated that there was no significant difference between their outputs, but the incorporation of LLMs shortened the completion time with fewer design steps and less time allocated to the late stages of design. Despite being perceived as less efficient and trusted, LLMs can still be viewed as potential collaborators, with humans holding the leadership. These findings offer the HCI community a thorough comprehension of how LLMs influence creativity-related practices, providing valuable insights for designing future interactions with LLMs.

Since traditional gaze-tracking methods rely on line-of-sight estimation, spatial attention modeling from neural activity offers an alternative perspective to gaze estimation. This paper presents a proof-of-concept study on attention-guided gaze estimation with Electroencephalography (EEG), investigating whether brain signals can be leveraged to estimate attentional focus within a controlled 3D environment. We first conducted a preliminary survey to gather public opinions, revealing a generally positive attitude towards EEG-driven gaze tracking. Building on this insight, we collected an EEG dataset in VR, where participants engaged with stimuli presented at predefined spatial locations. We introduce a deep learning model that estimates the relative saliency of candidate positions, enabling gaze estimation through optimization within the learned representation. Our results demonstrate that attentional focus was successfully mapped in a 3D coordinate space from 5 participants, and low-frequency oscillations contributed more significantly to predictive performance. The model achieved robust accuracy in distinguishing gaze locations, highlighting the potential of EEG-based gaze estimation for attention tracking in 3D environments.

The development of AR and VR technologies is enhancing users' online shopping experiences in various ways. However, in existing VR shopping applications, shopping contexts merely refer to the products and virtual malls or metaphorical scenes where users select products. This leads to the defect that users can only imagine rather than intuitively feel whether the selected products are suitable for their real usage scenes, resulting in a significant discrepancy between their expectations before and after the purchase. To address this issue, we propose PaRUS, a VR shopping approach that focuses on the context between products and their real usage scenes. PaRUS begins by rebuilding the virtual scenario of the products' real usage scene through a new semantic scene reconstruction pipeline (manual operation needed), which preserves both the structured scene and textured object models in the scene. Afterwards, intuitive visualization of how the selected products fit the reconstructed virtual scene is provided. We conducted two user studies to evaluate how PaRUS impacts user experience, behavior, and satisfaction with their purchase. The results indicated that PaRUS significantly reduced the perceived performance risk and improved users' trust and expectation with their results of purchase.

Interactive videos have been applied to various areas due to their engagement potential and efficiency improvement of information communication. However, creating interactive videos can be challenging because of a lack of novice-oriented guidance in current platforms, and the logic-building process when authoring interactive videos. To address these challenges, we obtained insights from four creativity support tool designers, proposed a series of hierarchical interactive video structures based on existing narrative structures, and presented the HierVid system. The system is designed as a Template-Module-Unit Mode-based hierarchical authoring platform grounded on three design requirements, and we conducted two user studies to evaluate HierVid. The results showed that novice users could get started to use and understand the functions easily, and the system allowed users to use and explore freely, with an enhanced efficiency compared to the bilibili platform. In conclusion, our research and design of HierVid offer guidance and support for novice users, making interactive video authoring quicker and more accessible.

With the increasing prevalence of artificial intelligence (AI) agents, the transparency of agents has become vital in addressing interaction issues (e.g., trust, usefulness, and understandability). However, determining the transparency of AI agents requires a systematic consideration of complex related factors, including stakeholders, algorithms, context, etc. Thus, in our study, we presented an overview of studies on the transparency of AI agents through multiple-stage bibliometric analysis, and identified an ontological framework of the key concepts relevant to transparent AI. We then built a Transparent-AI Blueprint prototype which is a diagram that visualizes the ontological framework of design concepts. In the subsequent pilot test, we updated Blueprint to the final version, and validated it in a workshop. Our work structurally summarized the design concepts related to the transparency of AI agents, and proposed a useful and practical conceptual design tool that effectively guides designers to operationalize the transparency of AI agents.

As a particular type of artificial intelligence technology, machine learning (ML) is widely used to empower user experience (UX). However, designers, especially the novice designers, struggle to integrate ML into familiar design activities because of its ever-changing and growable nature. This paper proposes a design method called Material Lifecycle Thinking (MLT) that considers ML as a design material with its own lifecycle. MLT encourages designers to regard ML, users, and scenarios as three co-creators who cooperate in creating ML-empowered UX. We have developed ML Lifecycle Canvas (Canvas), a conceptual design tool that incorporates visual representations of the co-creators and ML lifecycle. Canvas guides designers to organize essential information for the application of MLT. By involving design students in the “research through design” process, the development of Canvas was iterated through its application to design projects. MLT and Canvas have been evaluated in design workshops, with completed proposals and evaluation results demonstrating that our work is a solid step forward in bridging the gap between UX and ML.