This paper presents AIrt Guide, a conversational human-AI system designed to scaffold interpretive curiosity and creative meaning-making during art museum visits. Art engagement is inherently interpretive and creative, and AIrt Guide structures dialogue to support sustained observation, evidence-based reasoning, and elaborative perspective shifting during the act of viewing. Grounded in a mini-c framework of everyday creative cognition, the system implements two pedagogical conversational variations: Visual Thinking Strategies (VTS) and Formal Analysis. Rather than delivering definitive interpretations, the system supports iterative meaning construction while preserving viewer agency and productive ambiguity. Findings from in-person workshops suggest that this form of conversational scaffolding promotes analytic depth and more explicit articulation of visual evidence. Overall, this work demonstrates the potential of conversational AI as a creativity support tool in cultural settings, augmenting everyday interpretive practice by expanding associative space without resolving meaning.

The integration of Generative AI (GenAI) into creative communities, like fanfiction, is reshaping how stories are created, shared, and valued. This study investigates the perceptions of 157 active fanfiction members, both readers and writers, regarding AI-generated content in fanfiction. Our research explores the impact of GenAI on community dynamics, examining how AI affects the participatory and collaborative nature of these spaces. The findings reveal responses ranging from cautious acceptance of AI’s potential for creative enhancement to concerns about authenticity, ethical issues, and the erosion of human-centered values. Participants emphasized the importance of transparency and expressed worries about losing social connections. Our study highlights the need for thoughtful AI integration in creative platforms using design interventions that enable ethical practices, promote transparency, increase engagement and connection, and preserve the community’s core values.

In this study, we focus on different strategies drivers use in terms of interleaving between driving and non-driving related tasks (NDRT) while taking back control from automated driving. We conducted two driving simulator experiments to examine how different cognitive demands of texting, priorities, and takeover time budgets affect drivers’ takeover strategies. We also evaluated how different takeover strategies affect takeover performance. We found that the choice of takeover strategy was influenced by the priority and takeover time budget but not by the cognitive demand of the NDRT. The takeover strategy did not have any effect on takeover quality or NDRT engagement but influenced takeover timing.

The growing availability of generative AI technologies such as large language models (LLMs) has significant implications for creative work. This paper explores twofold aspects of integrating LLMs into the creative process – the divergence stage of idea generation, and the convergence stage of evaluation and selection of ideas. We devised a collaborative group-AI Brainwriting ideation framework, which incorporated an LLM as an enhancement into the group ideation process, and evaluated the idea generation process and the resulted solution space. To assess the potential of using LLMs in the idea evaluation process, we design an evaluation engine and compared it to idea ratings assigned by three expert and six novice evaluators. Our findings suggest that integrating LLM in Brainwriting could enhance both the ideation process and its outcome. We also provide evidence that LLMs can support idea evaluation. We conclude by discussing implications for HCI education and practice.

Remote scientific collaborations have been pivotal in generating scientific discoveries and breakthroughs that accelerate research in many fields. Emerging VR applications for remote work, which utilize commercially available head-mounted displays (HMDs), offer the promise to enhance collaboration, through spatial and embodied experiences. However, there is little evidence on how professionals in general, and scientists in particular, could use existing commercial VR applications to support their cognitive and creative collaborative processes while exploring real-world data as part of day-to-day collaborative work. In this paper, we present findings from an empirical study with 14 coral reef scientists, examining how they chose to utilize available resources in existing virtual environments for their ongoing data-driven collaborative research. We shed light on scientists’ data organization practices, identify affordances unique to VR for supporting cognition in a collaborative setting, and highlight design requirements for supporting cognitive and creative collaboration processes in future tools.

1. The advent of the COVID-19 pandemic has forced millions of workers to suddenly shift their activity out of their offices and into their homes: 5–15% of Americans worked from home before the pand...

We explore the transfer of control from an automated vehicle to the driver. Based on data from N=19 participants who participated in a driving simulator experiment, we find evidence that the transfer of control often does not take place in one step. In other words, when the automated system requests the transfer of control back to the driver, the driver often does not simply stop the non-driving task. Rather, the transfer unfolds as a process of interleaving the non-driving and driving tasks. We also find that the process is moderated by the length of time available for the transfer of control: interleaving is more likely when more time is available. Our interface designs for automated vehicles must take these results into account so as to allow drivers to safely take back control from automation.

Recent technological advancements in automation have sparked interest in how automation will affect truck drivers and the trucking industry. However, there is a gap in the literature addressing how truck drivers perceive automation and how they believe it will impact trucking. This study aims to understand truck drivers’ perspectives on automation in the trucking industry. Extending a preliminary study, we conducted a broader analysis of comments discussing automation in the r/Truckers subreddit from February 2017 to March 2021. In general, the community had negative sentiments towards automation in the trucking industry. Participants speculated when automation would become mainstream in trucking and discussed the feasibility of automation in the context of executing non-driving tasks and having accommodating infrastructure. Our findings indicate that truck drivers seek to participate in conversations about the future and to prepare themselves for when automation is more prominent in the trucking industry.

Recent years are seeing a sharp increase in the availability of personal omic (e.g. genomes, microbiomes) data to non-experts through direct-to-consumer testing kits. While the scientific understanding of human -omic information is evolving, the interpretation of the data may impact well-being of users and relevant others, and therefore poses challenges and opportunities for CSCW research. We identify the information, interaction, and sense-making needs of microbiomic data users, within the broader context of social omics - the sharing and collaborative engagement with data and interpretation. Analyzing users' discussions on Reddit's r/HumanMicrobiome, we identified seven user needs for microbiome data: reviewing an annotated report, comparing microbiome data, tracking changes, receiving personalized actionable information, curating and securing information, documenting and sharing self experiments, and enhancing the communication between patients and health-care providers. We highlight the ways in which users interact with each other to collaboratively make sense of the data. We conclude with design implications, including tools for better communication with care providers, and for symptom-centered sharing and discussion.

Frameworks such as Direct Manipulation or Instrumental Interaction have been an important force in HCI research. Evaluating the impact of frameworks can identify whether and how a framework was used, how it has evolved, and what trends have developed over time. However, studying the impact of such theoretical contributions requires consideration of various perspectives and level of impact. As a case study for investigating the impact of theoretical work in HCI, we present our evaluation of the impact of the Reality Based Interaction (RBI) framework, introduced by the authors in 2008. We provide our findings about the impact of the framework both on contemporary research, through content-based citation analysis, and in HCI education, through a survey we conducted on emerging interaction frameworks. The article contributes a comprehensive methodology for evaluating the impact of frameworks through our twofold approach: content-based citation analysis, including the design of a new citation typology; and a survey on the use of frameworks in education using a taxonomy of learning goals. We also consider the role of frameworks in HCI as well as the future of the RBI framework.

Crimson Wave is a personal tangible user interface that generates and displays information about its user's menstrual cycle. Basal Body Temperature (BBT) is an individual's temperature at rest, which fluctuates throughout menstruation. Crimson Wave tracks the user's BBT through the wearable armband and then displays the data visually on a separate smart mirror. Specifically, the mirror lights up a color corresponding to the menstrual stage the user is experiencing. Crimson Wave offers a novel method of keeping track of one's health and integrating one's own menstrual information seamlessly into daily life. The combination of live and aggregated personalized data helps users optimize each day. Crimson Wave is helpful to people who are interested in being more informed about their cycles, especially when they are irregular. In this paper, we describe the concept behind Crimson Wave, as well as its implementation and iterations.

Technological advances offer new methods of representing physical objects in tangible and virtual forms. This study compares learning outcomes from 61 students as they interact with ancient Egyptian sculptures using three increasingly popular educational technologies: HoloLens AR headset, 3D model viewing website (SketchFab), and plastic extrusion 3D prints. We explored how differences in interaction styles affect the learning process, quantitative and qualitative learning outcomes, and critical analysis.

We present CRISPEE, a novel tangible user interface designed to engage young elementary school children in bioengineering concepts. Using CRISPEE, children assume the role of a bioengineer to create a genetic program that codes for a firefly's bioluminescent light. This is accomplished through sequencing tangible representations of BioBricks, which code for the primary colors of light (red, green, and blue) to be turned on or off. The interface and curricular supplement expose children in early elementary school to concepts traditionally taught much later in school curricula through playful interaction and exploration. We discuss CRISPEE's concept and design, and share findings from its preliminary evaluation with children and adults.

We present BacToMars, an educational video game that engages elementary school children in a bio-design activity. We describe its design and implementation, and discuss design considerations for developing playful and developmentally-appropriate interactive activities, which enable children to grasp complex scientific concepts and apply bio-design to solve problems. We share results from preliminary evaluation, and describe next steps in our investigation.

Recent trends in gaming diversification have shown that women are both an increasingly significant pool of consumers and game producers, and regular victims of misogynistic harassment. Such observations stress the importance of investigating the complex relationships of women and gaming. In this paper, we draw upon perspectives from Feminist HCI to extend the current knowledge of issues in gaming that are specific to women. We present results from a mixed-methods study with 327 participants who are students and alumnae of a women's college. Our findings shed light on the complex relationships of women with games, with other gamers, and with gaming culture and industry. The results also indicate that in some cases gender-related negative experiences of gaming have lasting impact on the participation and self-confidence of young women. We conclude by discussing the implications of our findings for the design of games, game development education, and for the study of gaming.

We present BacToMars, a collaborative educational video game that engages elementary school children in creative bio-design. We describe the game, as well as how our iterative and participatory design process is shaping the game's development. We also discuss plans for on-site evaluations at local elementary schools to answer research questions pertaining to collaboration, attitudes, and the potential of video games as a tool for teaching biological engineering to young children.

The Microsoft HoloLens keeps track of its location and rotation relative to the environment but lacks the ability to capture eye gaze data. We assess a novel method to extend the HoloLens with a head mounted eye-tracker. Using a combination of eye gaze data and head rotation we compared gaze behavior between real and virtual objects. Results indicate that eye-tracking plays an important role in accurately determining a user's gaze for real objects in contrast to virtual objects.

We present BacPack, a tangible museum exhibit for exploring bio-design. BacPack utilizes tangible tokens on a large multitouch table display to allow visitors the opportunity to participate in a playful bio-design activity-engineering bacteria for sustaining life on Mars. To understand the role of tangible tokens in facilitating engagement and learning with the exhibit, we developed and evaluated two versions of BacPack: one with tangible tokens and one that consists of only multitouch interaction. Results from an evaluation in the Tech Museum of Innovation indicate that tangible tokens provide additional opportunities for collaborative problem solving and impact learning through support for tinkering and experimentation. We discuss design considerations for exhibits that facilitate creative engagement and exploration with biology.

We present HoloMuse, an AR application for the HoloLens wearable device, which allows users to actively engage with archaeological artifacts from a museum collection in ways that are otherwise not possible. We designed HoloLens to facilitate learning and engagement with museum collections without taking away from the experience of viewing an original artifact within the context of an exhibit. HoloMuse can be used inside the gallery or in the classroom. It enables users to pick up, rotate, scale, and alter a hologram of an original archeological artifact using in-air gestures. Users can also curate their own exhibit or customize an existing one by selecting artifacts from a virtual gallery and placing them within the physical world so that they are viewable only using the device. We intend to study the impact of HoloMuse on learning and engagement with college-level art history and archeology students.

We present findings from an empirical study of how groups of eight users collaborate on a decision-making task around an interactive tabletop. To our knowledge, this is the first study to examine co-located collaboration in larger groups (of 8-12 users) seated around a large-scale high-resolution multi-touch horizontal display. Our findings shed light on: 1) the effect of collaboration patterns of larger groups on equity of participation; 2) the role of participants' position around the tabletop in forming collaborations; and 3) the mechanisms, which facilitate coordination and collaboration in larger group interacting around large-scale tabletops; We also contribute computational methods that leverage image processing to analyze interaction around large-scale tabletops. Finally, we discuss implications for the design of large-scale tabletop systems for supporting co-located collaboration in larger groups.

The increase in the availability of personal genomic data to lay consumers using online services poses a challenge to HCI researchers: such data are complex and sensitive, involve multiple dimensions of uncertainty, and can have substantial implications for individuals' well-being. Personal genomic data are also unique because unlike other personal data, which constantly change, genomic data are largely stable during a person's lifetime; it is their interpretation and implications that change over time as new medical research exposes relationships between genes and health. In this paper, we present a novel tool for self exploration of personal genomic data. To evaluate the usability and utility of the tool, we conducted the first study of a genome interpretation tool to date, in which users used their own personal genomic data. We conclude by offering design implications for the development of interactive personal genomic reports.

In this study we explore the impact of exposure to social annotation, embedded in online consent forms, on individuals' beliefs and decisions in the context of informed consent. In this controlled between-subjects experiment, participants were presented with an online consent form for a personal genomics study. Individuals were randomly assigned to either a social annotation condition that exposed them to previous users' comments on-screen, or to a traditional consent form without social input. We compared participants' perceptions about their consent decision, their trust in the organization seeking the consent, and their actual consent across conditions. While no significant difference was observed between actual consent rates, we found that on average individuals exposed to social annotation felt that their decision was more informed, and furthermore, that the effect of the exposure to social annotation was stronger among users characterized by relatively lower levels of prior privacy preserving behaviors.

We are developing ways to teach computational thinking through interaction with tangible digital tools for synthetic biology. Inspired in part by Jeannette Wing's (Wing 2008) notion of the essence and pervasiveness of computational thinking, a growing community within SIGCSE is exploring tools that encourage computational thinking that don't use traditional computer science terminology or computer programming (Basawapna 2013, Daily 2014, Freeman 2014, Miller 2014, Wolz 2011). However, that community has not yet explored how to teach computational thinking through bio-design. Synthetic biology is a relatively new research field where scientists engineer living organisms to exhibit new behaviors. Biologists arrange and combine BioBricks, sequences of DNA that exhibit a behavior, sense an environmental factor, or indicate the beginning or end of a larger sequence. Bio-design not only incorporates traditional principles of computational thinking (abstraction, modularity, design/build/test), but also introduces new challenges that force the user to apply these principles in new ways. Interacting with biology can thus 1) help deepen understanding for people who are proficient in computational thinking, 2) provide a platform for introduction to computational thinking for the first time. Because most bio-design is performed in a wetlab by experienced biologists, we focus now on designing and developing low-barrier digital tools for interacting with biology. We present SynFlo (Okerlund 2016), a tangible interactive system for bio-design that mimics a real-life biological wetlab protocol and we discuss the properties of SynFlo and bio-design in general that encourage computational thinking.

We present BacPack for New Frontiers, an interactive museum exhibit that introduces core synthetic biology concepts to visitors through tangible, multi-touch, and physical interaction. The exhibit engages users in the design and engineering of bacteria, which are helpful in extreme environments like Mars. This project aims to allow museum visitors to play and tinker with biology through tangible and multi-touch interactions that bridge size and time scales. Here we present the design, implementation, and preliminary evaluation of this exhibit.

We present SynFlo, a tangible museum exhibit for exploring bio-design. SynFlo utilizes active and concrete tangible tokens to allow visitors to experience a playful biodesign activity through complex interactivity with digital biological creations. We developed two versions of SynFlo: one that combines active tokens with real concrete objects (i.e. labware) and one that consists of only abstract active tokens. Results from an evaluation in a museum indicate that both systems support learning. We discuss design choices for biology education tools to overcome confounders of biology and facilitate positive engagement and learning.

EmotiSphere is an interactive sensor-based musical instrument that generates music based on a user's current emotional state. Interactions with EmotiSphere draw upon everyday interactions with physical spherical objects, as well as on familiar interactions with music players. EmotiSphere offers a novel way to understand the relationship between emotion and music, and is aimed at people who want to create music and express themselves but do not necessarily possess skills in music composition. We describe the conceptualization and context of EmotiSphere, as well as its technical implementation.

We present a case study of applying TEI research to a data-intense scientific workflow that requires the exploration of large datasets through the construction of complex queries. We describe our two-year-long effort and design iterations of Eugenie, an interface for helping synthetic biologists through the collaborative and intricate process of bio-design. We introduce new interaction techniques for browsing large data sets and for constructing complex queries with active tangible tokens and an interactive tabletop. We also discuss challenges and opportunities for applying TEI to support data-driven inquiry.

Multi-touch and tangible interfaces provide unique opportunities for enhancing learning and discovery with big data. However, existing interaction techniques have limitations when manipulating large data sets. Our goal is to define novel interaction techniques for multi-touch and tangible interfaces, which support the construction of complex queries for big data. In this paper, we present results from a study which investigates the use of gestural interaction with active tokens for manipulating large data sets. In particular, we studied user expectations of a hybrid tangible and gestural language engaging this space. Our main results include a vocabulary of user-defined gestures for interaction with active tokens, which extends beyond familiar multi-touch gestures; characterization of the design space of gestural interaction with active tokens; and insight into participants' mental models, including common metaphors. We also present implications for the design of multi-touch and tangible interfaces with active tokens.

The combination of advanced genomic technologies and computational tools enables researchers to conduct large-scale experiments that answer biological questions in unprecedented ways. However, interaction tools in this area currently remain immature. We propose that tangible, embedded, and embodied interaction (TEI) offers unique opportunities for enhancing discovery and learning in genomics. Also, designing for problems in genomics can help move forward the theory and practice of TEI. We present challenges and key questions for TEI research in genomics, lessons learned from three case studies, and potential areas of focus for TEI research and design.

SynFlo is an interactive installation that utilizes tangible interaction to make core concepts of synthetic biology accessible to the public. This playful installation allows users to create useful virtual life forms from standardized genetic components through the manipulation of augmented object that mirror scientific instruments in order to explore synthetic biology concepts and protocols. These virtual E. coli, can serve as environmental biosensors that when deployed into an environment represented by a tabletop computer, detect toxins and change their color as an indicator. The goal of this project is to explore ways to develop effective interactive activities for outreach in STEM without the limitations of access to a lab bench while communicating the excitement of cutting-edge research.

Phylogenetic trees are representations of evolutionary relationships amongst species. Interviews of instructors and students have revealed that novice biologists have difficulty understanding phylogenetics. Moreover, misinterpretations of phylogenetics are common among college-level students. In this paper we present Phylo-Genie, a tabletop interface for fostering collaborative learning of phylogenetics. We conducted an experimental study with 56 participants, comparing students' conceptual learning and engagement using Phylo-Genie as: 1) a multi-touch tabletop interface and 2) a pen and paper activity. Our findings show that the tabletop implementation fosters collaborative learning by engaging users in the activity. We also shed light on the way in which our design principles facilitated engagement and collaborative learning in a tabletop environment.

We present G-nome Surfer 2.0, a tabletop interface for fostering inquiry-based learning of genomics. We conducted an experimental study with 48 participants that compared students' learning of genomic concepts using existing bioinformatics tools and using two alternative implementations of G-nome Surfer: a collaborative multi-mouse GUI and a tabletop interface. Our findings indicate that G-nome Surfer improves students' performance, reduces workload, and increases enjoyment. The comparison of tabletop and multi-mouse implementations further shows that the tabletop condition results in four educational benefits: 1) increasing physical participation, 2) encouraging reflection, 3) fostering effective collaboration, and 4) facilitating more intuitive interaction.

Molecular and computational biologists develop new insights by gathering heterogeneous data from genomic databases and leveraging bioinformatics tools. Through a qualitative study with 17 participants, we found that molecular and computational biologists experience difficulties interpreting, comparing, annotating, sharing, and relating this vast amount of biological information. We further observed that such interactions are critical for forming new scientific hypotheses. These observations motivated the creation of G-nome Surfer, a tabletop interface for collaborative exploration of genomic data that implements multi-touch and tangible interaction techniques. G-nome Surfer was developed in close collaboration with domain scientists and is aimed at lowering the threshold for using bioinformatics tools. A first-use study with 16 participants found that G-nome Surfer enables users to gain biological insights that are based on multiple forms of evidence with minimal overhead.

Tangible interaction shows promise to significantly enhance computer-mediated support for activities such as learning, problem solving, and design. However, tangible user interfaces are currently considered challenging to design and build. Designers and developers of these interfaces encounter several conceptual, methodological, and technical difficulties. Among others, these challenges include: the lack of appropriate interaction abstractions, the shortcomings of current user interface software tools to address continuous and parallel interactions, as well as the excessive effort required to integrate novel input and output technologies. To address these challenges, we propose a specification paradigm for designing and implementing Tangible User Interfaces (TUIs), that enables TUI developers to specify the structure and behavior of a tangible user interface using high-level constructs which abstract away implementation details. An important benefit of this approach, which is based on User Interface Description Language (UIDL) research, is that these specifications could be automatically or semi-automatically converted into concrete TUI implementations. In addition, such specifications could serve as a common ground for investigating both design and implementation concerns by TUI developers from different disciplines. Thus, the primary contribution of this article is a high-level UIDL that provides developers from different disciplines means for effectively specifying, discussing, and programming a broad range of tangible user interfaces. There are three distinct elements to this contribution: a visual specification technique that is based on Statecharts and Petri nets, an XML-compliant language that extends this visual specification technique, as well as a proof-of-concept prototype of a Tangible User Interface Management System (TUIMS) that semi-automatically translates high-level specifications into a program controlling specific target technologies.

introduction Share on Introduction to the special issue on UIDL for next-generation user interfaces Editors: Orit Shaer Wellesley College Wellesley CollegeView Profile , Robert J. K. Jacob Tufts University Tufts UniversityView Profile , Mark Green University of Ontario Institute of Technology University of Ontario Institute of TechnologyView Profile , Kris Luyten Hasselt University Hasselt UniversityView Profile Authors Info & Claims ACM Transactions on Computer-Human InteractionVolume 16Issue 4Article No.: 16pp 1–3https://doi.org/10.1145/1614390.1614391Published:27 November 2009Publication History 8citation870DownloadsMetricsTotal Citations8Total Downloads870Last 12 Months3Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We are in the midst of an explosion of emerging human-computer interaction techniques that redefine our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Based on this concept of RBI, we provide a framework that can be used to understand, compare, and relate current paths of recent HCI research as well as to analyze specific interaction designs. We believe that viewing interaction through the lens of RBI provides insights for design and uncovers gaps or opportunities for future research.

We created a tangible user interface that allows children to create musical compositions through constructive play. Our Marble Track Audio Manipulator (MTAM) is an augmented marble tower construction kit where marbles represent sound clips and tracks represent different sound effects. To create musical compositions, children collaboratively build a marble tower and then play their compositions by dropping marbles into the tower. As marbles roll through the tower children can interact with the marbles and thus improvise and alter their musical compositions. By augmenting a popular toy, physically representing sound clips and effects as well as allowing improvisation, the MTAM system provides children with a creative, playful, and engaging encounter with music.

We created Smart Blocks, an augmented mathematical manipulative that allows users to explore the concepts of volume and surface area of 3-dimensional (3D) objects. This interface supports physical manipulation for exploring spatial relationships and it provides continuous feedback for reinforcing learning. By leveraging the benefits of physicality with the advantages of digital information, this tangible interface provides an engaging environment for learning about surface area and volume of 3D objects.

In this paper we introduce the Tangible Video Editor (TVE), a multi-user, tangible interface for sequencing digital video. We present a new approach to tabletop interaction by using multiple handheld computers embedded in plastic tokens. Drawing from the rich physical experience of tradition film editing techniques we designed the TVE to engage multiple users in a collaborative process and encourage the exploration of narrative ideas. We used active tokens to provide a malleable interface, enabling users to organize the interface components in unspecified ways. Our implementation improves upon common projection-based tabletop interfaces in a number of ways including a design for use beyond dedicated two dimensional spaces and a naturally scaling screen resolution.

Reality-based interfaces (RBIs) offer the promise of natural interfaces that are intuitive to use. By allowing users to leverage their innate skills and knowledge about the real-physical world, RBIs could benefit disadvantage populations that rely on physical I/O modalities and tangible representations. We present our research which is aimed at simplifying the development of RBIs by providing a technology-independent visual language for modeling and programming these interfaces. Such a language will enable developers to systematically analyze and compare interaction designs as part of the development process while addressing issues such as users' skills and needs. Ultimately, we expect this research to lower the barriers for developing RBIs