The digital divide refers to disparities in access to and use of digital tooling across social and economic groups. This divide can reinforce marginalization both at the individual level and at the level of places, because persistent economic advantages accrue to places where new technologies are adopted early. To what extent are emerging generative artificial intelligence (AI) tools subject to these social and spatial divides? We leverage a large-scale search query database to characterize U.S. residents' knowledge of a novel generative AI tool, ChatGPT, during its first six months of release. We identify hotspots of higher-than-expected search volumes for ChatGPT in coastal metropolitan areas, while coldspots are evident in the American South, Appalachia, and the Midwest. Nationwide, counties with the highest rates of search have proportionally more educated and more economically advantaged populations, as well as proportionally more technology and finance-sector jobs in comparison with other counties or with the national average. Observed associations with race/ethnicity and urbanicity are attenuated in fully adjusted hierarchical models, but education emerges as the strongest positive predictor of generative AI awareness. In the absence of intervention, early differences in uptake show a potential to reinforce existing spatial and socioeconomic divides.

Understanding user intents in information access scenarios can help us provide more relevant and personalized search results and recommendations. However, analyzing user intents is not easy, especially for emerging forms of Web search such as Artificial Intelligence (AI)-driven chat. To understand user intents from retrospective log data, we need a way to label them with meaningful categories that capture their diversity and dynamics. Existing methods rely on manual or Machine-Learned (ML) labeling, which is either expensive or inflexible for large and dynamic datasets. Large Language Models (LLMs) could generate rich and relevant concepts, descriptions, and examples for user intents using log data of user interactions. However, using LLMs to generate a user intent taxonomy and applying it for a given Information Retrieval (IR) application can be problematic for two main reasons: (1) such a taxonomy is not externally validated; and (2) there may be an undesirable feedback loop if an LLM does both these tasks without external validation. To address this, we propose a new methodology with human experts and assessors to verify the quality of the LLM-generated taxonomy. We also present an end-to-end pipeline that uses an LLM with Human-in-the-Loop (HITL) to produce, refine, and apply labels for user intent analysis in log data. We demonstrate its effectiveness by uncovering new insights into user intents from search and chat logs from the Microsoft Bing Web search engine. The novelty in this research stems from the method for generating purpose-driven user intent taxonomies with strong validation. Our approach not only helps remove methodological and practical bottlenecks from intent-focused research, but also provides a new framework for generating, validating, and applying other kinds of taxonomies in a scalable and adaptable way, with reasonable human effort.

Ying-Chun Lin, Jennifer Neville, Jack Stokes, Longqi Yang, Tara Safavi, Mengting Wan, Scott Counts, Siddharth Suri, Reid Andersen, Xiaofeng Xu, Deepak Gupta, Sujay Kumar Jauhar, Xia Song, Georg Buscher, Saurabh Tiwary, Brent Hecht, Jaime Teevan. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Transforming unstructured text into structured and meaningful forms, organized by useful category labels, is a fundamental step in text mining for downstream analysis and application. However, most existing methods for producing label taxonomies and building text-based label classifiers still rely heavily on domain expertise and manual curation, making the process expensive and time-consuming. This is particularly challenging when the label space is under-specified and large-scale data annotations are unavailable. In this paper, we address these challenges with Large Language Models (LLMs), whose prompt-based interface facilitates the induction and use of large-scale pseudo labels. We propose TnT-LLM, a two-phase framework that employs LLMs to automate the process of end-to-end label generation and assignment with minimal human effort for any given use-case. In the first phase, we introduce a zero-shot, multi-stage reasoning approach which enables LLMs to produce and refine a label taxonomy iteratively. In the second phase, LLMs are used as data labelers that yield training samples so that lightweight supervised classifiers can be reliably built, deployed, and served at scale. We apply TnT-LLM to the analysis of user intent and conversational domain for Bing Copilot (formerly Bing Chat), an open-domain chat-based search engine. Extensive experiments using both human and automatic evaluation metrics demonstrate that TnT-LLM generates more accurate and relevant label taxonomies when compared against state-of-the-art baselines, and achieves a favorable balance between accuracy and efficiency for classification at scale.

Urban environmental monitoring campaigns depend on expertise from city agencies, residents, and researchers. Deployment efforts rarely include all three stakeholders, typically leading to initiatives that struggle to produce credible, actionable data. We describe the implementation of a large-scale, long-term air quality sensing network in Chicago Illinois; detail stakeholder interviews and meetings; and present three interfaces—–a website accessible via in-situ QR codes, APIs, and a mobile, mixed-media experience. We show how a collaborative approach created a more equitable sensor distribution compared to crowdsourced or regulatory designs. We highlight shared goals of education, engagement, and empowerment despite the diversity of tool and analytics needs across stakeholder groups. Reflecting on our work, we develop a “three-legged stool” framework representing the criticality of balanced participation from three key stakeholder groups—city, community, and research—in deploying novel urban technologies. This approach can help HCI researchers facilitate more democratic technology deployments in urban spaces.

This paper presents Eclipse, a platform for low-cost urban environmental sensing using solar-powered and cellular-connected devices. Dense sensor networks promise to monitor pollution at fine spatial and temporal resolutions, yet few cities have actually implemented such networks due to high costs and limited accuracy. We address these barriers by developing an end-to-end framework for urban air quality sensing with minimal infrastructure requirements. We designed an unobtrusive device that collects data on fine particulate matter (PM2.5), temperature, relative humidity, and barometric pres-sure. A modular design further includes four low-cost gas sensors - Ozone (03), Nitrogen Dioxide (NO2), Sulfur Dioxide (SO2), and Carbon Monoxide (CO) - selected based on local priorities. We deployed 115 devices across Chicago, reliably collecting data for over 90% of expected sensor-hours from July 2 - September 30, 2021. We further developed a calibration strategy that reduced errors by 41.2 – 98.8%, improving accuracy to levels recommended for hotspot detection (PM2.5and 03) or education (NO2and SO2). Through this work, we offer insights on the real-world deployment of a replicable, large-scale, end-to-end platform for hyperlocal urban environmental sensing.

We study how the ratings people receive on online labor platforms are influenced by their performance, gender, their rater's gender, and displayed ratings from other raters. We conducted a deception study in which participants collaborated on a task with a pair of simulated workers, who varied in gender and performance level, and then rated their performance. When the performance of paired workers was similar, low-performing females were rated lower than their male counterparts. Where there was a clear performance difference between paired workers, low-performing females were preferred over a similarly-performing male peer. Furthermore, displaying an average rating from other raters made ratings more extreme, resulting in high performing workers receiving significantly higher ratings and low performers lower ratings compared to when average ratings were absent. This work contributes an empirical understanding of when biases in ratings manifest, and offers recommendations for how online work platforms can counter these biases.

Roughly one in ten Americans move every year, bringing significant social and economic impact to both the places they move from and places they move to. We show that migration intent mined from internet search queries can forecast domestic migration and provide new insights beyond government data. We extract from a major search engine (Bing.com) 120 million raw queries with migration intent from 2014 to 2016, including origin and destination geographies, and the specific intent for migration such as whether the potential migration is housing or employment related. Using these queries, we map U.S. state level migration flows, validate them against government data, and demonstrate that adding search query-based metrics explains variance in migration prediction above robust baseline models. In addition, we show that the specific migration intent extracted from these queries unpack the differential demands of migrants with different demographic backgrounds and geographic interests. Examples include interactions between age, education, and income, and migration attributes such as buying versus renting housing and employment in technology versus manual labor job sectors. We discuss how local government, policy makers, and computational social scientists can benefit from this information.

We are in a transitional economic period emphasizing automation of physical jobs and the shift towards intellectual labor. How can we measure and understand human behaviors of job search, and how communities are adapting to these changes? We use internet search data to estimate employment demand in the United States. Starting with 225 million raw job search queries in 2015 and 2016 from a popular search engine, we classify queries into one of 15 fields of employment with accuracy and F-1 of 97%, and use the resulting query volumes to estimate per-sector employment demand in U.S. counties. We validate against Bureau of Labor Statistics measures, and then demonstrate benefits for communities, showing significant differences in the types of jobs searched for across socio-economic dimensions like poverty and education level. We discuss implications for macroeconomic measurement, as well as how community leaders, policy makers, and the field of HCI can benefit.

In the past few years an unprecedented wave of anti-abortion policies were introduced and enacted in state governments in the U.S., affecting millions of constituents. We study this rapid spread of policy change as a function of the underlying ideology of constituents. We examine over 200,000 public messages posted on Twitter surrounding abortion in the year 2013, a year that saw 82 new anti-abortion policies enacted. From these posts, we characterize people's expressions of opinion on abortion and show how these expressions align with policy change on these issues. We detail a number of ideological differences between constituents in states enacting anti versus pro-abortion policies, such as a tension between the moral values of purity versus fairness, and a differing emphasis on the fetus versus the pregnant woman. We also find significant differences in how males versus females discuss the issue of abortion, including greater emphasis on health and religion by males. Using these measures to characterize states, we can construct models to explain the spread of abortion policy from state to state and project which types of abortion policies a state will introduce. Models defining state similarity using our Twitter-based measures improved policy projection accuracy by 7.32% and 12.02% on average over geographic and poll-based ideological similarity, respectively. Additionally, models constructed from the expressions of male-only constituents perform better than models from the expressions of female-only constituents, suggesting that the ideology of men is more aligned with the recent spread of anti-abortion legislation than that of women.

Social media has emerged as a prominent platform where people can express their feelings about social and political issues of our time. We study the many voices discussing an issue within a constituency and how they reflect ideology and may signal the outcome of important policy decisions. Focusing on the issue of same-sex marriage legalization, we examine almost 2 million public Twitter posts related to same-sex marriage in the U.S. states over the course of 4 years starting from 2011. Among other findings, we find evidence of moral culture wars between ideologies and show that constituencies that express higher levels of emotion and have fewer actively engaged participants often precede legalization efforts that fail. From our measures, we build statistical models to predict the outcome of potential policy changes, with our best model achieving 87% accuracy. We also achieve accuracies of 70%, comparable to public opinion surveys, many months before a policy decision. We discuss how these analyses can augment traditional political science techniques as well as assist activists and policy analysts in understanding discussions on important issues at a population scale.

Analysis of smokers' posts and behaviors on Twitter reveals factors impacting abstinence and relapse during cessation attempts. Combining automatic and crowdsourced techniques, we detect users trying to quit smoking and analyze tweet and network data from a sample of 653 individuals over a two-year window of quitting. Guided by theory and practice, we derive behavioral, social, and emotional measures to compare users who abstain and relapse. We also examine the cessation process, demonstrating that Twitter can help chronicle how some people go about quitting. Among other results, we show that those who fail in their smoking cessation are far heavier posters and use relatively less positive language, while those who succeed are more social in both network ties and in directed communication. We conclude with insights on how intelligent intervention systems can harness these signals to provide tailored behavior change support.

The birth of a child is a major milestone in the life of parents. We leverage Facebook data shared voluntarily by 165 new mothers as streams of evidence for characterizing their postnatal experiences. We consider multiple measures including activity, social capital, emotion, and linguistic style in participants' Facebook data in pre- and postnatal periods. Our study includes detecting and predicting onset of post-partum depression (PPD). The work complements recent work on detecting and predicting significant postpartum changes in behavior, language, and affect from Twitter data. In contrast to prior studies, we gain access to ground truth on postpartum experiences via self-reports and a common psychometric instrument used to evaluate PPD. We develop a series of statistical models to predict, from data available before childbirth, a mother's likelihood of PPD. We corroborate our quantitative findings through interviews with mothers experiencing PPD. We find that increased social isolation and lowered availability of social capital on Facebook, are the best predictors of PPD in mothers.

We consider social media as a promising tool for public health, focusing on the use of Twitter posts to build predictive models about the forthcoming influence of childbirth on the behavior and mood of new mothers. Using Twitter posts, we quantify postpartum changes in 376 mothers along dimensions of social engagement, emotion, social network, and linguistic style. We then construct statistical models from a training set of observations of these measures before and after the reported childbirth, to forecast significant postpartum changes in mothers. The predictive models can classify mothers who will change significantly following childbirth with an accuracy of 71%, using observations about their prenatal behavior, and as accurately as 80-83% when additionally leveraging the initial 2-3 weeks of postnatal data. The study is motivated by the opportunity to use social media to identify mothers at risk of postpartum depression, an underreported health concern among large populations, and to inform the design of low-cost, privacy-sensitive early-warning systems and intervention programs aimed at promoting wellness postpartum.

We investigate the landscape of affective expression of employees at a large Fortune 500 software corporation via an internal microblogging tool. We present three analyses of emotional expression among employees, based on literature in organizational behavior: its relationship to (1) exogenous/lifestyle and endogenous workplace factors, (2) geography and (3) organizational structure. We find that employees tend to make significant accommodations in affect expression when interacting with others over the organizational hierarchy. We also find that positive affect is expressed through interpersonal communications that connect disparate geographic regions. Our findings have implications for enabling emotional reflection of employees and for management in that they can help uncover emotional patterns associated with episodes of high and low productivity, allowing organizations to improve employee engagement and promote positive attitudes.

We explore the harnessing of social media as a window on changes around major life events in individuals and larger populations. We specifically examine patterns of activity, emotional, and linguistic correlates for childbirth and postnatal course. After identifying childbirth events on Twitter, we analyze daily posting patterns and language usage before and after birth by new mothers, and make inferences about the status and dynamics of changes in emotions expressed following childbirth. We find that childbirth is associated with some changes for most new mothers, but approximately 15% of new mothers show significant changes in their online activity and emotional expression postpartum. We observe that these mothers can be distinguished by linguistic changes captured by shifts in a relatively small number of words in their social media posts. We introduce a greedy differencing procedure to identify the type of language that characterizes significant changes in these mothers during postpartum. We conclude with a discussion about how such characterizations might be applied to recognizing and understanding health and well-being in women following childbirth.

In this paper we examine the information sharing practices of people living in cities amid armed conflict. We describe the volume and frequency of microblogging activity on Twitter from four cities afflicted by the Mexican Drug War, showing how citizens use social media to alert one another and to comment on the violence that plagues their communities. We then investigate the emergence of civic media "curators," individuals who act as "war correspondents" by aggregating and disseminating information to large numbers of people on social media. We conclude by outlining the implications of our observations for the design of civic media systems in wartime.

Microblogs have become an increasingly important source of information, both in the U.S. (Twitter) and in China (Weibo). However, the brevity of microblog updates, combined with increasing access of microblog content through search rather than through direct network connections, makes it challenging to assess the credibility of news relayed in this manner [34]. This paper reports on experimental and survey data that compare the impact of several features of microblog updates (author's gender, name style, profile image, location, and degree of network overlap with the reader) on credibility perceptions among U.S. and Chinese audiences. We reveal the complex mechanism of credibility perceptions, identify several key differences in how users from each country critically consume microblog content, and discuss how to incorporate these findings into the design of improved user interfaces for accessing microblogs in different cultural settings.

We examine the possibility of distributed sensemaking: improving a user's sensemaking by leveraging previous users' work without those users directly collaborating or even knowing one another. We asked users to engage in sensemaking by organizing and annotating web search results into "knowledge maps," either with or without previous users' maps to work from. We also recorded gaze patterns as users examined others' knowledge maps. Our findings show the conditions under which distributed sensemaking can improve sensemaking quality; that a user's sensemaking process is readily apparent to a subsequent user via a knowledge map; and that the organization of content was more useful to subsequent users than the content itself, especially when those users had differing goals. We discuss the role distributed sensemaking can play in schema induction by helping users make a mental model of an information space and make recommendations for new tool and system development.

Twitter is now used to distribute substantive content such as breaking news, increasing the importance of assessing the credibility of tweets. As users increasingly access tweets through search, they have less information on which to base credibility judgments as compared to consuming content from direct social network connections. We present survey results regarding users' perceptions of tweet credibility. We find a disparity between features users consider relevant to credibility assessment and those currently revealed by search engines. We then conducted two experiments in which we systematically manipulated several features of tweets to assess their impact on credibility ratings. We show that users are poor judges of truthfulness based on content alone, and instead are influenced by heuristics such as user name when making credibility assessments. Based on these findings, we discuss strategies tweet authors can use to enhance their credibility with readers (and strategies astute readers should be aware of!). We propose design improvements for displaying social search results so as to better convey credibility.

Content in microblogging systems such as Twitter is produced by tens to hundreds of millions of users. This diversity is a notable strength, but also presents the challenge of finding the most interesting and authoritative authors for any given topic. To address this, we first propose a set of features for characterizing social media authors, including both nodal and topical metrics. We then show how probabilistic clustering over this feature space, followed by a within-cluster ranking procedure, can yield a final list of top authors for a given topic. We present results across several topics, along with results from a user study confirming that our method finds authors who are significantly more interesting and authoritative than those resulting from several baseline conditions. Additionally our algorithm is computationally feasible in near real-time scenarios making it an attractive alternative for capturing the rapidly changing dynamics of microblogs.

As users of social networking websites expand their network of friends, they are often flooded with newsfeed posts and status updates, most of which they consider to be "unimportant" and not newsworthy. In order to better understand how people judge the importance of their newsfeed, we conducted a study in which Facebook users were asked to rate the importance of their newsfeed posts as well as their friends. We learned classifiers of newsfeed and friend importance to identify predictive sets of features related to social media properties, the message text, and shared background information. For classifying friend importance, the best performing model achieved 85% accuracy and 25% error reduction. By leveraging this model for classifying newsfeed posts, the best newsfeed classifier achieved 64% accuracy and 27% error reduction.

Community-based question and answer (Q&A) systems facilitate information exchange and enable the creation of reusable knowledge repositories. While these systems are growing in usage and are changing how people find and share information, current designs are inefficient, wasting the time and attention of their users. Furthermore, existing systems do not support signaling and screening of joking and non-serious questions. Coupling Q&A services with instant and text messaging for faster questions and answers may exacerbate these issues, causing Q&A services to incur high interruption costs on their users.

For over a century, citizen scientists have volunteered to collect huge quantities of data for professional scientists to analyze. We designed Pathfinder, an online environment that challenges this traditional division of labor by providing tools for citizen scientists to collaboratively discuss and analyze the data they collect. We evaluated Pathfinder in a sustainability and commuting context using a mixed methods approach in both naturalistic and experimental settings. Our results showed that citizen scientists preferred Pathfinder to a standard wiki and were able to go beyond data collection and engage in deeper discussion and analyses. We also found that citizen scientists require special types of technological support because they generate original research. This paper offers an early example of the mutually beneficial relationship between HCI and citizen science.

We consider trails to be a document type of growing importance, authored in abundance as locative technologies become embedded in mobile devices carried by billions of humans. As these trail documents become annotated by communities of users, the resulting data sets can provide support for a host of services. In this paper we describe our sociotechnical exploration of the devices, scenarios, and end-user interactions that will come into play as these tools become widespread. We couch this work in a discussion of the sociological impact of a shift from hyperlinks to "hyperties" -- links that bridge the gap between computational media and physical world interactions. We describe a prototype hardware device for location and other sensor data capture. This device links to a complementary website for querying, sharing, and distributing the resulting route datasets. The web application allows users to find related community members via shared attributes of their contributed or annotated routes. These attributes may be generated in part by route analysis performed by systems for activity identification and classification.

Media sharing over mobile devices is quickly becoming a common practice, used to support a variety of social processes. Most existing systems employ a model of sharing that treats each shared item as a distinct message. We argue that there are compelling reasons to utilize a more flexible, cohesive approach to media sharing. To test this idea, we developed Sandboxes, a prototype application for mobile phones that goes beyond basic media sharing to offer a form of collaborative multimedia composition. The design of Sandboxes is based upon three primary themes: interactivity, flexibility and cohesiveness.

In an empirical study we investigated how matchmaking for online gaming platforms could benefit from additional implicit information conveyed in profiles that include photos or voice recordings. We used 150 real online gamer profiles (50 text-only, 50 text & photo, 50 text & voice) to elicit gaming partner preferences from 267 online gamers. We found profiles with photos to lead to lower overall preference, indicating that people used them to reject potential partners. Voice recordings did not reduce overall preference but gave participants relevant information for gaming partner choice. We close with recommendations for the design of profile-based matchmaking systems.

We conducted two studies of communication: an ethnographic study of communication primarily in homes, cars, and public places, and a survey of communication in a large corporation. A clear pattern emerged. To a greater degree than expected in the ethnographic study, people were familiar with a broad range of communication tools. Awareness and a lack of anxiety was the norm even for tools that a person rarely or had not yet used. As a result, people frequently shifted to the tool that was most appropriate for a task at hand. The resulting behaviors conflict with popular press images and have implications for the designers of communication tools.

Lightweight photo sharing, particularly via mobile devices, is fast becoming a common communication medium used for maintaining a presence in the lives of friends and family. How should such systems be designed to maximize this social presence while maintaining simplicity? An experimental photo sharing system was developed and tested that, compared to current systems, offers highly simplified, group-centric sharing, automatic and persistent people-centric organization, and tightly integrated desktop and mobile sharing and viewing. In an experimental field study, the photo sharing behaviors of groups of family or friends were studied using their normal photo sharing methods and with the prototype sharing system. Results showed that users found photo sharing easier and more fun, shared more photos, and had an enhanced sense of social presence when sharing with the experimental system. Results are discussed in the context of design principles for the rapidly increasing number of lightweight photo sharing systems.