Collaborative problem solving (CPS) is a fundamental practice in middle-school mathematics education; however, student groups frequently stall or struggle without ongoing teacher support. Recent work has explored how Generative AI tools can be designed to support one-on-one tutoring, but little is known about how AI can be designed as peer learning partners in collaborative learning contexts. We conducted a participatory design study with 24 middle school students, who first engaged in mathematics CPS tasks with AI peers in a technology probe, and then collaboratively designed their ideal AI peer. Our findings reveal that students envision an AI peer as competent in mathematics yet explicitly deferential, providing progressive scaffolds such as hints and checks under clear student control. Students preferred a tone of friendly expertise over exaggerated personas. We also discuss design recommendations and implications for AI peers in middle school mathematics CPS.

Therapeutic homework (i.e., tasks assigned by therapists for clients to complete between sessions) is essential for effective psychotherapy, yet therapists often interpret fragmented client logs, assessments, and reflections within limited preparation time. Our formative study with licensed therapists revealed three critical design requirements: support for interpreting unstructured client self-reports, customization aligned with clinical objectives, and seamless integration across multiple data sources. We then designed and developed TheraTrack, a customizable, therapist-facing tool that integrates multi-dimensional data and leverages large language models to generate traceable summaries and support natural-language queries, to streamline between-session homework tracking. Our pilot study with 14 therapists showed that TheraTrack reduced their cognitive load, enabled verification through direct navigation from AI summaries to original data entries, and was adapted differently for private analysis compared to in-session use, with dependence varying based on therapist experience and usage duration. We also discuss design implications for clinician-centered AI for mental health.

Social anxiety (SA) has become increasingly prevalent. Traditional coping strategies often face accessibility challenges. Generative AI (GenAI), known for their knowledgeable and conversational capabilities, are emerging as alternative tools for mental well-being. With the increased integration of GenAI, it is important to examine individuals' attitudes and trust in GenAI chatbots' support for SA. Through a mixed-method approach that involved surveys (n = 159) and interviews (n = 17), we found that individuals with severe symptoms tended to trust and embrace GenAI chatbots more readily, valuing their non-judgmental support and perceived emotional comprehension. However, those with milder symptoms prioritized technical reliability. We identified factors influencing trust, such as GenAI chatbots' ability to generate empathetic responses and its context-sensitive limitations, which were particularly important among individuals with SA. We also discuss the design implications and use of GenAI chatbots in fostering cognitive and emotional trust, with practical and design considerations.

Prevalent in biomedical applications (e.g., human phenotype research), multimodal datasets can provide valuable insights into the underlying physiological mechanisms. However, current machine learning (ML) models designed to analyze these datasets often lack interpretability and identifiability guarantees, which are essential for biomedical research. Recent advances in causal representation learning have shown promise in identifying interpretable latent causal variables with formal theoretical guarantees. Unfortunately, most current work on multimodal distributions either relies on restrictive parametric assumptions or yields only coarse identification results, limiting their applicability to biomedical research that favors a detailed understanding of the mechanisms. In this work, we aim to develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets. Theoretically, we consider a nonparametric latent distribution (c.f., parametric assumptions in previous work) that allows for causal relationships across potentially different modalities. We establish identifiability guarantees for each latent component, extending the subspace identification results from previous work. Our key theoretical contribution is the structural sparsity of causal connections between modalities, which, as we will discuss, is natural for a large collection of biomedical systems. Empirically, we present a practical framework to instantiate our theoretical insights. We demonstrate the effectiveness of our approach through extensive experiments on both numerical and synthetic datasets. Results on a real-world human phenotype dataset are consistent with established biomedical research, validating our theoretical and methodological framework.

Generative AI (GenAI), especially Large Language Models (LLMs), is rapidly reshaping both programming workflows and computer science education. Many programmers now incorporate GenAI tools into their workflows, including for collaborative coding tasks such as pair programming. While prior research has demonstrated the benefits of traditional pair programming and begun to explore GenAI-assisted coding, the role of LLM-based tools as collaborators in pair programming remains underexamined. In this work, we conducted a mixed-methods study with 39 undergraduate students to examine how GenAI influences collaboration, learning, and performance in pair programming. Specifically, students completed six in-class assignments under three conditions: Traditional Pair Programming (PP), Pair Programming with GenAI (PAI), and Solo Programming with GenAI (SAI). They used both LLM-based inline completion tools (e.g., GitHub Copilot) and LLM-based conversational tools (e.g., ChatGPT). Our results show that students in the PAI condition achieved the highest assignment scores, whereas those in the SAI condition attained the lowest. Additionally, students' attitudes toward LLMs' programming capabilities improved significantly after collaborating with LLM-based tools, and preferences were largely shaped by the perceived usefulness for completing assignments and learning programming skills, as well as the quality of collaboration. Our qualitative findings further reveal that while students appreciated LLM-based tools as valuable pair programming partners, they also identified limitations (e.g., contextual constraints and possibly outdated knowledge bases), and had different expectations compared to human teammates. Students in our study primarily relied on LLM-based tools for syntax clarification and conceptual guidance, while turning to human partners for idea exchanges. Our study provides one of the first empirical evaluations of GenAI as a pair programming collaborator through a comparison of three conditions (PP, PAI, and SAI). We also discuss the design implications and pedagogical considerations for future GenAI-assisted pair programming approaches.

Significant advancements have occurred in the application of Large Language Models (LLMs) for social simulations. Despite this, their abilities to perform teaming in task-oriented social events are underexplored. Such capabilities are crucial if LLMs are to effectively mimic human-like social behaviors and form efficient teams to solve tasks. To bridge this gap, we introduce MetaAgents , a social simulation framework populated with LLM-based agents. MetaAgents facilitates agent engagement in conversations and a series of decision making within social contexts, serving as an appropriate platform for investigating interactions and interpersonal decision-making of agents. In particular, we construct a job fair environment as a case study to scrutinize the team assembly and skill-matching behaviors of LLM-based agents. We take advantage of both quantitative metrics evaluation and qualitative text analysis to assess their teaming abilities at the job fair. Our evaluation demonstrates that LLM-based agents perform competently in making rational decisions to develop efficient teams. However, we also identify limitations that hinder their effectiveness in more complex team assembly tasks. Our work provides valuable insights into the role and evolution of LLMs in task-oriented social simulations.

In the era of digital communication, misinformation on social media threatens the foundational trust in these platforms. While myriad measures have been implemented to counteract misinformation, the complex relationship between these interventions and the multifaceted dynamics of trust and distrust on social media remains underexplored. To bridge this gap, we surveyed 1,769 participants in the U.S. to gauge their trust and distrust in social media and examine their experiences with anti-misinformation features. Our research demonstrates how trust and distrust in social media are not simply two ends of a spectrum; but can also co-exist, enriching the theoretical understanding of these constructs. Furthermore, participants exhibited varying patterns of trust and distrust across demographic characteristics and platforms. Our results also show that current misinformation interventions helped heighten awareness of misinformation and bolstered trust in social media, but did not alleviate underlying distrust. We discuss theoretical and practical implications for future research.

Large language models (LLMs) have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs present many challenges, particularly in the realm of trustworthiness. This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions. Specifically, we first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. We then present a study evaluating 16 mainstream LLMs in TrustLLM, consisting of over 30 datasets. Our findings firstly show that in general trustworthiness and capability (i.e., functional effectiveness) are positively related. Secondly, our observations reveal that proprietary LLMs generally outperform most open-source counterparts in terms of trustworthiness, raising concerns about the potential risks of widely accessible open-source LLMs. However, a few open-source LLMs come very close to proprietary ones, suggesting that open-source models can achieve high levels of trustworthiness without additional mechanisms like moderator, offering valuable insights for developers in this field. Thirdly, it is important to note that some LLMs may be overly calibrated towards exhibiting trustworthiness, to the extent that they compromise their utility by mistakenly treating benign prompts as harmful and consequently not responding. Besides these observations, we’ve uncovered key insights into the multifaceted trustworthiness in LLMs. We emphasize the importance of ensuring transparency not only in the models themselves but also in the technologies that underpin trustworthiness. We advocate that the establishment of an AI alliance between industry, academia, the open-source community to foster collaboration is imperative to advance the trustworthiness of LLMs.

Emotion significantly impacts our daily behaviors and interactions. While recent generative AI models, such as large language models, have shown impressive performance in various tasks, it remains unclear whether they truly comprehend emotions and why. This paper aims to address this gap by incorporating psychological theories to gain a holistic understanding of emotions in generative AI models. Specifically, we propose three approaches: 1) EmotionPrompt to enhance AI model performance, 2) EmotionAttack to impair AI model performance, and 3) EmotionDecode to explain the effects of emotional stimuli, both benign and malignant. Through extensive experiments involving language and multi-modal models on semantic understanding, logical reasoning, and generation tasks, we demonstrate that both textual and visual EmotionPrompt can boost the performance of AI models while EmotionAttack can hinder it. More importantly, EmotionDecode reveals that AI models can comprehend emotional stimuli akin to the mechanism of dopamine in the human brain. Our work heralds a novel avenue for exploring psychology to enhance our understanding of generative AI models, thus boosting the research and development of human-AI collaboration and mitigating potential risks.

Large language models (LLMs) have been widely used as agents to complete different tasks, such as personal assistance or event planning. Although most of the work has focused on cooperation and collaboration between agents, little work explores competition, another important mechanism that promotes the development of society and economy. In this paper, we seek to examine the competition dynamics in LLM-based agents. We first propose a general framework for studying the competition between agents. Then, we implement a practical competitive environment using GPT-4 to simulate a virtual town with two types of agents, including restaurant agents and customer agents. Specifically, the restaurant agents compete with each other to attract more customers, where competition encourages them to transform, such as cultivating new operating strategies. Simulation experiments reveal several interesting findings at the micro and macro levels, which align well with existing market and sociological theories. We hope that the framework and environment can be a promising testbed to study the competition that fosters understanding of society. Code is available at: https://github.com/microsoft/competeai.

The integration of AI assistants, especially through the development of Large Language Models (LLMs), into computer science education has sparked significant debate, highlighting both their potential to augment student learning and the risks associated with their misuse. An emerging body of work has looked into using LLMs in education, primarily focusing on evaluating the performance of existing models or conducting short-term human subject studies. However, very little work has examined the impacts of LLM-powered assistants on students in entry-level programming courses, particularly in real-world contexts and over extended periods. To address this research gap, we conducted a semester-long, between-subjects study with 50 students using CodeTutor, an LLM-powered assistant developed by our research team. Our study results show that students who used CodeTutor (the "CodeTutor group" as the experimental group) achieved statistically significant improvements in their final scores compared to peers who did not use the tool (the "control group"). Within the CodeTutor group, those without prior experience with LLM-powered tools demonstrated significantly greater performance gain than their counterparts. We also found that students expressed positive feedback regarding CodeTutor's capability to comprehend their queries and assist in learning programming language syntax. However, they had concerns about CodeTutor's limited role in developing critical thinking skills. Over the course of the semester, students' agreement with CodeTutor's suggestions decreased, with a growing preference for support from traditional human teaching assistants. Our findings also show that students turned to CodeTutor for different tasks, including programming task completion, syntax comprehension, and debugging, particularly seeking help for programming assignments. Our analysis further reveals that the quality of user prompts was significantly correlated with CodeTutor's response effectiveness. Building upon these results, we discuss the implications of our findings for the need to integrate Generative AI literacy into curricula to foster critical thinking skills, and turn to examining the temporal dynamics of user engagement with LLM-powered tools. We further discuss the discrepancy between the anticipated functions of tools and students' actual capabilities, which sheds light on the need for tailored strategies to improve educational outcomes.

Do people trust social media? If so, why, in what contexts, and how does that trust impact their lives? Researchers, companies, and journalists alike have increasingly investigated these questions, which are fundamental to understanding social media interactions and their implications for society. However, trust in social media is a complex concept, and there is conflicting evidence about the antecedents and implications of trusting social media content, users, and platforms. More problematic is that we lack basic agreement as to what trust means in the context of social media. Addressing these challenges, we conducted a systematic review to identify themes and challenges in this field. Through our analysis of 70 papers, we contribute a synthesis of how trust in social media is defined, conceptualized, and measured, a summary of trust antecedents in social media, an understanding of how trust in social media impacts behaviors and attitudes, and directions for future work.

Large language models have abilities in creating high-volume human-like texts and can be used to generate persuasive misinformation. However, the risks remain under-explored. To address the gap, this work first examined characteristics of AI-generated misinformation (AI-misinfo) compared with human creations, and then evaluated the applicability of existing solutions. We compiled human-created COVID-19 misinformation and abstracted it into narrative prompts for a language model to output AI-misinfo. We found significant linguistic differences within human-AI pairs, and patterns of AI-misinfo in enhancing details, communicating uncertainties, drawing conclusions, and simulating personal tones. While existing models remained capable of classifying AI-misinfo, a significant performance drop compared to human-misinfo was observed. Results suggested that existing information assessment guidelines had questionable applicability, as AI-misinfo tended to meet criteria in evidence credibility, source transparency, and limitation acknowledgment. We discuss implications for practitioners, researchers, and journalists, as AI can create new challenges to the societal problem of misinformation.

Abstract We stumble upon new and repeating information daily. As information comes from many sources, social media continues to play a predominant role in disseminating information, ultimately impacting individuals’ perceptions and behaviors. A prime example of this impact was observed during the COVID-19 pandemic, in which social media use was influencing willingness to receive the COVID-19 vaccine. While studies on this relationship between social media use and vaccination intent have been widely investigated, less is known about the mechanisms that link these two variables, specifically the types of information seen on social media platforms and the effects of these different types of information. In this exploratory study, we demonstrate the mediator role of information exposure (to include both types of information and frequency) between social media use and vaccination intent. Our results show that different types of information mediate this relationship differently and demonstrate how these relationships were further moderated by the income level of the participant. We conclude with the implications of these findings and how our findings can inform the direction of future research within the field of human–computer interaction.

During the COVID-19 pandemic, a number of data visualizations were created to inform the public about the rapidly evolving crisis. Data dashboards, a form of information dissemination used during the pandemic, have facilitated this process by visualizing statistics regarding the number of COVID-19 cases over time. Prior work on COVID-19 visualizations has primarily focused on the design and evaluation of specific visualization systems from technology-centered perspectives. However, little is known about what occurs behind the scenes during the visualization creation processes, given the complex sociotechnical contexts in which they are embedded. Yet, such ecological knowledge is necessary to help characterize the nuances and trajectories of visualization design practices in the wild, as well as generate insights into how creators come to understand and approach visualization design on their own terms and for their own situated purposes. In this research, we conducted a qualitative interview study among dashboard creators from federal agencies, state health departments, mainstream news media outlets, and other organizations that created (often widely-used) COVID-19 dashboards to answer the following questions: how did visualization creators engage in COVID-19 dashboard design, and what tensions, conflicts, and challenges arose during this process? Our findings detail the trajectory of design practices-from creation to expansion, maintenance, and termination-that are shaped by the complex interplay between design goals, tools and technologies, labor, emerging crisis contexts, and public engagement. We particularly examined the tensions between designers and the general public involved in these processes. These conflicts, which often materialized due to a divergence between public demands and standing policies, centered around the type and amount of information to be visualized, how public perceptions shape and are shaped by visualization design, and the strategies utilized to deal with (potential) misinterpretations and misuse of visualizations. Our findings and lessons learned shed light on new ways of thinking in visualization design, focusing on the bundled activities that are invariably involved in human and nonhuman participation throughout the entire trajectory of design practice.

The world population is projected to rapidly age over the next 30 years. Given the increasing digital technology adoption amongst older adults, researchers have investigated how technology can support aging populations. However, little work has examined how technology can support older adults during crises, despite increasingly common natural disasters, public health emergencies, and other crisis scenarios in which older adults are especially vulnerable. Addressing this gap, we conducted focus groups with older adults residing in coastal locations to examine to what extent they felt technology could support them during emergencies. Our findings characterize participants’ desire for tools that enhance community resilience-local knowledge, preparedness, community relationships, and communication, that help communities withstand disasters. Further, older adults’ crisis technology preferences were linked to their sense of control, social relationships, and digital readiness. We discuss how a focus on community resilience can yield crisis technologies that more effectively support older adults.

During crises like COVID-19, individuals are inundated with conflicting and time-sensitive information that drives a need for rapid assessment of the trustworthiness and reliability of information sources and platforms. This parallels evolutions in information infrastructures, ranging from social media to government data platforms. Distinct from current literature, which presumes a static relationship between the presence or absence of trust and people’s behaviors, our mixed-methods research focuses on situated trust, or trust that is shaped by people’s information-seeking and assessment practices through emerging information platforms (e.g., social media, crowdsourced systems, COVID data platforms). Our findings characterize the shifts in trustee (what/who people trust) from information on social media to the social media platform(s), how distrust manifests skepticism in issues of data discrepancy, the insufficient presentation of uncertainty, and how this trust and distrust shift over time. We highlight the deep challenges in existing information infrastructures that influence trust and distrust formation.

In response to COVID-19, a vast number of visualizations have been created to communicate information to the public. Information exposure in a public health crisis can impact people’s attitudes towards and responses to the crisis and risks, and ultimately the trajectory of a pandemic. As such, there is a need for work that documents, organizes, and investigates what COVID-19 visualizations have been presented to the public. We address this gap through an analysis of 668 COVID-19 visualizations. We present our findings through a conceptual framework derived from our analysis, that examines who, (uses) what data, (to communicate) what messages, in what form, under what circumstances in the context of COVID-19 crisis visualizations. We provide a set of factors to be considered within each component of the framework. We conclude with directions for future crisis visualization research.

Abstract Graphics Processing Units (GPUs) have been widely used to accelerate artificial intelligence, physics simulation, medical imaging, and information visualization applications. To improve GPU performance, GPU hardware designers need to identify performance issues by inspecting a huge amount of simulator‐generated traces. Visualizing the execution traces can reduce the cognitive burden of users and facilitate making sense of behaviors of GPU hardware components. In this paper, we first formalize the process of GPU performance analysis and characterize the design requirements of visualizing execution traces based on a survey study and interviews with GPU hardware designers. We contribute data and task abstraction for GPU performance analysis. Based on our task analysis, we propose Daisen, a framework that supports data collection from GPU simulators and provides visualization of the simulator‐generated GPU execution traces. Daisen features a data abstraction and trace format that can record simulator‐generated GPU execution traces. Daisen also includes a web‐based visualization tool that helps GPU hardware designers examine GPU execution traces, identify performance bottlenecks, and verify performance improvement. Our qualitative evaluation with GPU hardware designers demonstrates that the design of Daisen reflects the typical workflow of GPU hardware designers. Using Daisen, participants were able to effectively identify potential performance bottlenecks and opportunities for performance improvement. The open‐sourced implementation of Daisen can be found at gitlab.com/akita/vis . Supplemental materials including a demo video, survey questions, evaluation study guide, and post‐study evaluation survey are available at osf.io/j5ghq .

Temporal event sequence alignment has been used in many domains to visualize nuanced changes and interactions over time. Existing approaches align one or two sentinel events. Overview tasks require examining all alignments of interest using interaction and time or juxtaposition of many visualizations. Furthermore, any event attribute overviews are not closely tied to sequence visualizations. We present Sequence Braiding, a novel overview visualization for temporal event sequences and attributes using a layered directed acyclic network. Sequence Braiding visually aligns many temporal events and attribute groups simultaneously and supports arbitrary ordering, absence, and duplication of events. In a controlled experiment we compare Sequence Braiding and IDMVis on user task completion time, correctness, error, and confidence. Our results provide good evidence that users of Sequence Braiding can understand high-level patterns and trends faster and with similar error. A full version of this paper with all appendices; the evaluation stimuli, data, and analysis code; and source code are available at [Formula: see text].

Tools and interfaces are increasingly expected to be synchronous and distributed to accommodate remote collaboration. Yet, adoption of these techniques for data visualization is low partly because development is difficult: existing collaboration software systems either do not support simultaneous interaction or require expensive redevelopment of existing visualizations. We contribute VisConnect: a web-based synchronous distributed collaborative visualization system that supports most web-based SVG data visualizations, balances system safety with responsiveness, and supports simultaneous interaction from many collaborators. VisConnect works with existing visualization implementations with little-to-no code changes by synchronizing low-level JavaScript events across clients such that visualization updates proceed transparently across clients. This is accomplished via a peer-to-peer system that establishes consensus among clients on the per-element sequence of events, and uses a lock service to grant access over elements to clients. We contribute collaborative extensions of traditional visualization interaction techniques, such as drag, brush, and lasso, and discuss different strategies for collaborative visualization interactions. To demonstrate the utility of VisConnect, we present novel examples of collaborative visualizations in the healthcare domain, remote collaboration with annotation, and show in an education case study for e-learning with 22 participants that students found the ability to remotely collaborate on class activities helpful and enjoyable for understanding concepts. A free copy of this paper and source code are available on OSF at osf.io/ut7e6 and at visconnect.us.

Mass emergencies increasingly pose significant threats to human life, with a disproportionate burden being incurred by older adults. Research has explored how mobile technology can mitigate the effects of mass emergencies. However, less work has examined how mobile technologies support older adults during emergencies, considering their unique needs. To address this research gap, we interviewed 16 older adults who had recent experience with an emergency evacuation to understand the perceived value of using mobile technology during emergencies. We found that there was a lack of awareness and engagement with existing crisis apps. Our findings characterize the ways in which our participants did and did not feel crisis informatics tools address human values, including basic needs and esteem needs. We contribute an understanding of how older adults used mobile technology during emergencies and their perspectives on how well such tools address human values.

The number of informal caregivers for family members with Alzheimer's Disease (AD) is rising dramatically in the United States. AD caregivers disproportionately experience numerous health problems and are often isolated with little support. An active lifestyle can help prevent and mitigate physical and psychological health concerns amongst AD caregivers. Research has demonstrated how pervasive exergames can encourage physical activity (PA) in the general population, yet little work has explored how these tools can address the significant PA barriers that AD caregivers face. To identify opportunities for design, we conducted semi-structured interviews and participatory design sessions with 14 informal caregivers of family members with AD. Our findings characterize how becoming an AD caregiver profoundly impacts one's ability to be active, perspectives on being active, and the ways that exergames might best support this population.We discuss implications for design and howour findings challenge existing technological approaches to PA promotion.

Type 1 diabetes is a chronic, incurable autoimmune disease affecting millions of Americans in which the body stops producing insulin and blood glucose levels rise. The goal of intensive diabetes management is to lower average blood glucose through frequent adjustments to insulin protocol, diet, and behavior. Manual logs and medical device data are collected by patients, but these multiple sources are presented in disparate visualization designs to the clinician-making temporal inference difficult. We conducted a design study over 18 months with clinicians performing intensive diabetes management. We present a data abstraction and novel hierarchical task abstraction for this domain. We also contribute IDMVis: a visualization tool for temporal event sequences with multidimensional, interrelated data. IDMVis includes a novel technique for folding and aligning records by dual sentinel events and scaling the intermediate timeline. We validate our design decisions based on our domain abstractions, best practices, and through a qualitative evaluation with six clinicians. The results of this study indicate that IDMVis accurately reflects the workflow of clinicians. Using IDMVis, clinicians are able to identify issues of data quality such as missing or conflicting data, reconstruct patient records when data is missing, differentiate between days with different patterns, and promote educational interventions after identifying discrepancies.