From ice cream flavors to climate change, people exhibit a wide array of opinions on various topics, and understanding the rationale for these opinions can promote healthy discussion and consensus among them. As such, it can be valuable for a large language model (LLM), particularly as an AI assistant, to be able to empathize with or even explain these various standpoints. In this work, we hypothesize that different topic stances often manifest correlations that can be used to extrapolate to topics with unknown opinions. We explore various prompting and fine-tuning methods to improve an LLM’s ability to (a) extrapolate from opinions on known topics to unknown ones and (b) support their extrapolation with reasoning. Our findings suggest that LLMs possess inherent knowledge from training data about these opinion correlations, and with minimal data, the similarities between human opinions and model-extrapolated opinions can be improved by more than 50%. Furthermore, LLM can generate the reasoning process behind their extrapolation of opinions.

Exposure to misinformation poses significant challenges to democratic processes and public health, particularly during critical events like elections. This study adopts a user-centric approach to analyze the linguistic features of misinformation actually consumed by individuals during web browsing. Using data from a nationally representative panel of 1,240 American adults and their web-browsing data (21M URL visits) during the 2020 U.S. Presidential Election, we examine linguistic and topical differences in the content of 91K unique misinformation and hard news webpages by utilizing natural language processing techniques and Large Language Models. We find that misinformation consumed by users is generally easier to read, exhibits higher negative sentiment, and employs more moral language than hard news. We also find significant linguistic variations across topics--misinformation can be diverse and vary in linguistic features depending on the subject matter. We also identify heterogeneity across key user characteristics: older adults consume more misinformation about COVID-19 and health, with content showing more negative sentiment and fewer moral terms than expected. Republicans engage with misinformation characterized by more negative sentiment and higher moral language, focusing less on health topics and more on social and political issues. These results highlight the importance of a user-centric approach and suggest that interventions to combat misinformation should be tailored to specific topics and user characteristics for greater effectiveness.

Interpersonal conflicts occur frequently in both offline and online groups, with conditions for conflict especially ripe online. This research attempts to understand the consequences of online group conflict and reporting it to group administrators, both for the protagonists in the conflict and observers. If group conflict is aversive, then group members should reduce their group participation after observing conflict. Theories of imitation and behavioral mimicry suggest that even onlookers will exhibit more conflict and negative language after observing conflict conversations in their group. In contrast, theories of deterrence suggest that both the instigator of the conflict and onlookers will reduce their conflict and onlookers might even increase their engagement if conflicts are reported to group administrators. The current study uses de-identified and aggregated data from Facebook group conversations and Mahalanobis distance matching to test these ideas. Results are consistent with the hypothesis that conflict in group conversations reduces engagement within the group and increases the amount of conflict and the negativity of language users express in the group. However, inconsistent with deterrence theories, conflict and language negativity increase and group engagement decreases when conflict is reported to group administrators.

Growing concerns have been raised about the detrimental effects of uncivil comments on the web towards democracy. However, there is still a lack of understanding about online incivility's nuanced and complicated nature and its impact on conversation development and user behaviors. This work aims to fill that research gap by modeling incivility and its relationship to political discussions. We develop a comprehensive and fine-grained taxonomy that characterizes incivility with vulgarity, name-calling (inter-personal and third-party attacks), aspersion, and stereotypes, and then apply the framework to quantify the level of each incivility category in over 40 million comments from Reddit. Using large-scale quantitative analysis, we investigate the types of interactions and contexts in which incivility is more likely to occur, model how incivility shapes subsequent conversations, and examine user engagement patterns and behavioral changes after exposure to incivility. Our findings show that conversations that start out uncivil tend to become more uncivil in responses, and exposure to different incivility categories has differing effects on community members' engagement. We conclude with the implications of our research in assisting the design and moderation of online political communities.

Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, Yuning Mao. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

What people see on social media influences their affective state. Predictions of the affective reaction of an audience to a post could help posters creating content and viewers searching for it. This paper examines the value of both real comments and artificially generated ones in predicting the affective responses of an audience. We built an affect prediction model based on Facebook anonymized public posts to predict affective responses (anger, amusement, and sadness affect) as indicated by three Facebook reaction clicks (Angry, Haha, and Sad). Using the content of the original post can predict reactions well (.71 to.87 F1-scores). Adding the text of real post comments improves F1-score by up to 11%. Surprisingly, generated comments improve predictions as much as real comments. These artificial comments were produced using a pre-trained sequence-to-sequence, BART natural language generation model given a post as input. Using artificial comments means that one can predict affect reactions early in the history of a discussion, before anyone has actually commented on a post.

Tiezheng Yu, Hanchao Yu, Davis Liang, Yuning Mao, Shaoliang Nie, Po-Yao Huang, Madian Khabsa, Pascale Fung, Yi-Chia Wang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

We present NORMBANK, a knowledge bank of 155k situational norms.This resource is designed to ground flexible normative reasoning for interactive, assistive, and collaborative AI systems.Unlike prior commonsense resources, NORMBANK grounds each inference within a multivalent sociocultural frame, which includes the setting (e.g., restaurant), the agents' contingent roles (waiter, customer), their attributes (age, gender), and other physical, social, and cultural constraints (e.g., the temperature or the country of operation).In total, NORMBANK contains 63k unique constraints from a taxonomy that we introduce and iteratively refine here.Constraints then apply in different combinations to frame social norms.Under these manipulations, norms are non-monotonic -one can cancel an inference by updating its frame even slightly.Still, we find evidence that neural models can help reliably extend the scope and coverage of NORMBANK.We further demonstrate the utility of this resource with a series of transfer experiments.For data and code, see

Extensive research has been published on the conversational factors of effective volunteer peer counseling on online mental health platforms (OMHPs). However, studies differ in how they define and measure success outcomes, with most prior work examining only a single success metric. In this work, we model the relationship between previously reported linguistic predictors of effective counseling with four outcomes following a peer-to-peer session on a single OMHP: retention in the community, following up on a previous session with a counselor, users’ evaluation of a counselor, and changes in users’ mood. Results show that predictors correlate negatively with community retention but positively with users following up with and giving higher evaluations to individual counselors. We suggest actionable insights for therapy platform design and outcome measurement based on findings that the relationship between predictors and outcomes of successful conversations depends on differences in measurement construct and operationalization.

Nan Wang, Qifan Wang, Yi-Chia Wang, Maziar Sanjabi, Jingzhou Liu, Hamed Firooz, Hongning Wang, Shaoliang Nie. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

Content Warning: some examples in this paper may be offensive or upsetting.Conversational agents have come increasingly closer to human competence in open-domain dialogue settings; however, such models can reflect insensitive, hurtful, or entirely incoherent viewpoints that erode a user's trust in the moral integrity of the system.Moral deviations are difficult to mitigate because moral judgments are not universal, and there may be multiple competing judgments that apply to a situation simultaneously.In this work, we introduce a new resource, not to authoritatively resolve moral ambiguities, but instead to facilitate systematic understanding of the intuitions, values and moral judgments reflected in the utterances of dialogue systems.The MORAL INTEGRITY CORPUS, MIC , is such a resource, which captures the moral assumptions of 38k prompt-reply pairs, using 99k distinct Rules of Thumb (RoTs).Each RoT reflects a particular moral conviction that can explain why a chatbot's reply may appear acceptable or problematic.We further organize RoTs with a set of 9 moral and social attributes and benchmark performance for attribute classification.Most importantly, we show that current neural language models can automatically generate new RoTs that reasonably describe previously unseen interactions, but they still struggle with certain scenarios.Our findings suggest that MIC will be a useful resource for understanding and language models' implicit moral assumptions and flexibly benchmarking the integrity of conversational agents.

People come to social media to satisfy a variety of needs, such as being informed, entertained and inspired, or connected to their friends and community. Hence, to design a ranking function that gives useful and personalized post recommendations, it would be helpful to be able to predict the affective response a user may have to a post (e.g., entertained, informed, angered). This paper describes the challenges and solutions we developed to apply Affective Computing to social media recommendation systems.

With the rise of social media, users from across the world are able to connect and converse with each other online. While these connections have facilitated a growth in knowledge, online discussions can also end in acrimonious conflict. Previous computational studies have focused on creating online conflict detection models from inferred labels, primarily examine disagreement but not acrimony, and do not examine the conflict’s emergence. Social science studies have investigated offline conflict, which can differ from its online form, and rarely examines its emergence. The current research aims to understand how online conflicts arise in online personal conversations. Our ground truth is a Facebook tool that allows group members to report conflict to administrators. We contrast discussions ending with a conflict report with paired non-conflict discussions from the same post. We study both user characteristics (e.g., historical user-to-user interactions) and conversation dynamics (e.g., changes in emotional intensity over the course of the conversation). We use logistic regression to identify the features that predict conflict. User characteristics such as the commenter’s gender and previous involvement in negative online activity are strong indicators of conflict. Conversational dynamics, such as an increase in person-oriented discussion, are also important signals of conflict. These results help us understand how conflicts emerge and suggest better detection models and ways to alert group administrators and members early on to mediate the conversation.

Social network platforms have enabled large-scale measurement of user-to-user networks such as friendships. Less studied is user sentiment about their networks, such as a user’s satisfaction with their number of friends. We surveyed over 85,000 Facebook users about how satisfied they were with their number of friends on Facebook, connecting these responses to their on-platform activity. As suggested in prior work, we’d expect users who are not satisfied with their friend count to have a higher probability of experiencing the friendship paradox: “your friends have more friends than you”. However in our sample, among users with more than 3,500 friends, no user experiences the friendship paradox. Instead, we still observe that those users with more friends would prefer to have even more friends. The friendship paradox also contributes to local perception bias, defined as the difference between the average number of friends among a user’s friends and the average friend count in the population. Users with a positive perception bias – their friends have more friends than others – are less satisfied with their friend count. We then introduce a weighted perception bias metric that considers the fact that different friends have different effects on an individual’s perception. We find this new weighted perception bias better distinguishes friend count satisfaction outcomes for users with high friend count when compared to the original perception bias metric. We conclude with modeling the behavior interactions via a machine learning model, demonstrating the heterogeneity in the interactions across users with different perception biases. Altogether, these findings offer more insights on users’ friend count satisfaction, which may provide guidelines to improve the user experience and promote healthy interactions.

Millions of people participate in online peer-to-peer support sessions, yet there has been little prior research on systematic psychology-based evaluations of fine-grained peer-counselor behavior in relation to client satisfaction. This paper seeks to bridge this gap by mapping peer-counselor chat-messages to motivational interviewing (MI) techniques. We annotate 14,797 utterances from 734 chat conversations using 17 MI techniques and introduce four new interviewing codes such as ''chit-chat'' and ''inappropriate'' to account for the unique conversational patterns observed on online platforms. We automate the process of labeling peer-counselor responses to MI techniques by fine-tuning large domain-specific language models and then use these automated measures to investigate the behavior of the peer counselors via correlational studies. Specifically, we study the impact of MI techniques on the conversation ratings to investigate the techniques that predict clients' satisfaction with their counseling sessions. When counselors use techniques such as reflection and affirmation, clients are more satisfied. Examining volunteer counselors' change in usage of techniques suggest that counselors learn to use more introduction and open questions as they gain experience. This work provides a deeper understanding of the use of motivational interviewing techniques on peer-to-peer counselor platforms and sheds light on how to build better training programs for volunteer counselors on online platforms.

We present the first complete attempt at concurrently training conversational agents that communicate only via self-generated language.Using DSTC2 as seed data, we trained natural language understanding (NLU) and generation (NLG) networks for each agent and let the agents interact online.We model the interaction as a stochastic collaborative game where each agent (player) has a role ("assistant", "tourist", "eater", etc.) and their own objectives, and can only interact via natural language they generate.Each agent, therefore, needs to learn to operate optimally in an environment with multiple sources of uncertainty (its own NLU and NLG, the other agent's NLU, Policy, and NLG).In our evaluation, we show that the stochastic-game agents outperform deep learning based supervised baselines.

For a company looking to provide delightful user experiences, it is of paramount importance to take care of any customer issues. This paper proposes COTA, a system to improve speed and reliability of customer support for end users through automated ticket classification and answers selection for support representatives. Two machine learning and natural language processing techniques are demonstrated: one relying on feature engineering (COTA v1) and the other exploiting raw signals through deep learning architectures (COTA v2). COTA v1 employs a new approach that converts the multi-classification task into a ranking problem, demonstrating significantly better performance in the case of thousands of classes. For COTA v2, we propose an Encoder-Combiner-Decoder, a novel deep learning architecture that allows for heterogeneous input and output feature types and injection of prior knowledge through network architecture choices. This paper compares these models and their variants on the task of ticket classification and answer selection, showing model COTA v2 outperforms COTA v1, and analyzes their inner workings and shortcomings. Finally, an A/B test is conducted in a production setting validating the real-world impact of COTA in reducing issue resolution time by 10 percent without reducing customer satisfaction.

People often try to impress their friends online, but we don't know how well they do it or what they talk about to try to make themselves look good. In the face of known egocentric biases, which cause communicators to overestimate the extent that audiences will understand the intent of their messages, and self-enhancement biases, that cause people to overvalue their own behavior, it is likely that many self-presentation attempts will often fail. However, we don't know which topics cause such failure. In an empirical study, 1300 Facebook users evaluated their most recent status update in terms of how good it make them look. In addition external judges also evaluated the same update. Posters and outsiders agreed only modestly about how good an update made the poster appear (r=.36, p<.001). Posters generally thought that their posts make them look better than did the outsider judges. They also disagreed on which topics made them look good. Posters were especially likely to overestimate their self-presentation when they wrote about the mundane details of their daily life (e.g., Clothing, Sleep, or Religious imagery), but underestimated it when they wrote about family and relationships (e.g., Birthday, Father's Day, Love).

Social networking sites (SNSs) offer users a platform to build and maintain social connections. Understanding when people feel comfortable sharing information about themselves on SNSs is critical to a good user experience, because self-disclosure helps maintain friendships and increase relationship closeness. This observational research develops a machine learning model to measure self-disclosure in SNSs and uses it to understand the contexts where it is higher or lower. Features include emotional valence, social distance between the poster and people mentioned in the post, the language similarity between the post and the community and post topic. To validate the model and advance our understanding about online self-disclosure, we applied it to de-identified, aggregated status updates from Facebook users. Results show that women self-disclose more than men. People with a stronger desire to manage impressions self-disclose less. Network size is negatively associated with self-disclosure, while tie strength and network density are positively associated.

Research suggests that online health support benefits chronically ill users. Their satisfaction might be an indicator that they perceive group interactions as beneficial and a precursor to group commitment. We examined whether receiving emotional and informational support is satisfying in its own right, or whether satisfaction depends on matches between what users sought and what they received. Two studies collected judgments in a breast cancer support community of support users sought, support they received, and their expressed satisfaction. While receiving emotional or informational support in general positively predicted satisfaction, users expressed less satisfaction when they sought informational support but received emotional support. There was also a tendency for users to express more satisfaction when they sought and received informational support. On the other hand, users were equally satisfied with emotional and informational support after seeking emotional support. Implications for membership commitment and interventions in online support groups are discussed.

Although both men and women communicate frequently on Facebook, we know little about what they talk about, whether their topics differ and how their network responds. Using Latent Dirichlet Allocation (LDA), we identify topics from more than half a million Facebook status updates and determine which topics are more likely to receive feedback, such as likes and comments. Women tend to share more personal topics (e.g., family matters), while men discuss more public ones (e.g., politics and sports). Generally, women receive more feedback than men, but "male" topics (those more often posted by men) receive more feedback, especially when posted by women.

Although economists have long recognized the importance of a critical mass in growing a community, we know little about how it is achieved. This paper examines how initial topical focus influences communities' ability to attract a critical mass. When starting an online community, organizers need to define its initial scope. Topically narrow communities will probably attract a homogeneous group of interested in its content and compatible with each other. However, they are likely to attract fewer members than a diverse one because they offer only a subset of the topics. This paper reports an empirical analysis of longitudinal data collected from Twitter, where each new Twitter poster is considered the seed of a potential social collection. Users who focus the topics of their early tweets more narrowly ultimately attract more followers with more ties among them. Our results shed light on the development of online social networking structures.

Today many people with serious diseases use online support groups to seek social support. For these groups to be sustained and effective, member retention and commitment is important. Our study examined how different types and amounts of social support in an online cancer support group are associated with participants' length of membership. We first built machine learning models to automatically identify the extent to which messages contained emotional and informational support. Agreement with human judges was high (r > 0.76). We then used these models to measure the support exchanged in 1.5 million messages. Finally, we applied quantitative event history analysis to assess how exposure to emotional and informational support predicted group members' length of subsequent participation. The results demonstrated that the more emotional support members were exposed to, the lower the risk of dropout. In contrast, informational support did not have the same strong effects on commitment. We speculate that emotional support enhanced members' relationships with one another or the group as a whole, whereas informational support satisfied members' short-term information needs.

In this paper, we introduce a method to measure shared leadership in Wikipedia as a step in developing a new model of online leadership. We show that editors with varying degrees of engagement and from peripheral as well as central roles all act like leaders, but that core and peripheral editors show different profiles of leadership behavior. Specifically, we developed machine learning models to automatically identify four types of leadership behaviors from 4 million messages sent between Wikipedia editors. We found strong evidence of shared leadership in Wikipedia, with editors in peripheral roles producing a large proportion of leadership behaviors.

We investigate how the affordances provided by alternative interfaces for on-line discussion forums affect the structure of the discourse that unfolds. In order to investigate this impact, we compare the predictive power of time related and text similarity related features for identifying parent-child links between messages. The results from this work using this methodology suggest that interfaces that make parent-child relationships between messages explicit and do not constrain the choice of previous messages that users can reply to allow patterns of conversational behavior that violate the assumptions of traditional, tree-structured models of discourse where time related and similarity related features are highly predictive. An implication for future work is that because there is evidence that interface affordances affect the form of conversational contributions, techniques that process on-line communication data may need to be adapted for different communication interfaces.