Warning: This paper contains content that may be offensive or upsetting.Improvements in model construction, including fortified safety guardrails, allow Large language models (LLMs) to increasingly pass standard safety checks.However, LLMs sometimes slip into revealing harmful behavior, such as expressing racist viewpoints, during conversations.To analyze this systematically, we introduce CoBia, a suite of lightweight adversarial attacks that allow us to refine the scope of conditions under which LLMs depart from normative or ethical behavior in conversations.CoBia creates a constructed conversation where the model utters a biased claim about a social group.We then evaluate whether the model can recover from the fabricated bias claim and reject biased follow-up questions.We evaluate 11 open-source as well as proprietary LLMs for their outputs related to six socio-demographic categories that are relevant to individual safety and fair treatment, i.e., gender, race, religion, nationality, sex orientation, and others.Our evaluation is based on established LLM-based bias metrics, and we compare the results against human judgments to scope out the LLMs' reliability and alignment.The results suggest that purposefully constructed conversations reliably reveal bias amplification and that LLMs often fail to reject biased follow-up questions during dialogue.This form of stress-testing highlights deeply embedded biases that can be surfaced through interaction.Code and artifacts are available at github.com/nafisenik/CoBia.

When students reflect on their learning from a textbook via think-aloud processes, network representations can be used to capture the concepts and relations from these data.What can we learn from the resulting network representations about students' learning processes, knowledge acquisition, and learning outcomes?This study brings methods from entity and relation extraction using classic and LLM-based methods to the application domain of educational psychology.We built a ground-truth baseline of relational data that represents relevant (to educational science), textbook-based information as a semantic network.Among the tested models, SPN4RE and LUKE achieved the best performance in extracting concepts and relations from students' verbal data.Network representations of students' verbalizations varied in structure, reflecting different learning processes.Correlating the students' semantic networks with learning outcomes revealed that denser and more interconnected semantic networks were associated with more elaborated knowledge acquisition.Structural features such as the number of edges and surface overlap with textbook networks significantly correlated with students' posttest performance.

Examining the alignment of large language models (LLMs) has become increasingly important, e.g., when LLMs fail to operate as intended. This study examines the alignment of LLMs with human values for the domain of politics. Prior research has shown that LLM-generated outputs can include political leanings and mimic the stances of political parties on various issues. However, the extent and conditions under which LLMs deviate from empirical positions are insufficiently examined. To address this gap, we analyze the factors that contribute to LLMs' deviations from empirical positions on political issues, aiming to quantify these deviations and identify the conditions that cause them. Drawing on findings from cognitive science about representativeness heuristics, i.e., situations where humans lean on representative attributes of a target group in a way that leads to exaggerated beliefs, we scrutinize LLM responses through this heuristics' lens. We conduct experiments to determine how LLMs inflate predictions about political parties, which results in stereotyping. We find that while LLMs can mimic certain political parties' positions, they often exaggerate these positions more than human survey respondents do. Also, LLMs tend to overemphasize representativeness more than humans. This study highlights the susceptibility of LLMs to representativeness heuristics, suggesting a potential vulnerability of LLMs that facilitates political stereotyping. We also test prompt-based mitigation strategies, finding that strategies that can mitigate representative heuristics in humans are also effective in reducing the influence of representativeness on LLM-generated responses.

Impact assessment is an evolving area of research that aims at measuring and predicting the potential effects of projects or programs. Measuring the impact of scientific research is a vibrant subdomain, closely intertwined with impact assessment. A recurring obstacle pertains to the absence of an efficient framework which can facilitate the analysis of lengthy reports and text labeling. To address this issue, we propose a framework for automatically assessing the impact of scientific research projects by identifying pertinent sections in project reports that indicate the potential impacts. We leverage a mixed-method approach, combining manual annotations with supervised machine learning, to extract these passages from project reports. We experiment with different machine learning algorithms, including traditional statistical models as well as pre-trained transformer language models. Our experiments show that our proposed method achieves accuracy scores up to 0.81, and that our method is generalizable to scientific research from different domains and different languages.

Prompt-based fine-tuning has become an essential method for eliciting information encoded in pre-trained language models for a variety of tasks, including text classification.For multi-class classification tasks, prompt-based fine-tuning under low-resource scenarios has resulted in performance levels comparable to those of fully fine-tuning methods.Previous studies have used crafted prompt templates and verbalizers, mapping from the label terms space to the class space, to solve the classification problem as a masked language modeling task.However, cross-domain and finegrained prompt-based fine-tuning with an automatically enriched verbalizer remains unexplored, mainly due to the difficulty and costs of manually selecting domain label terms for the verbalizer, which requires humans with domain expertise.To address this challenge, we introduce SCIPROMPT, a framework designed to automatically retrieve scientific topic-related terms for low-resource text classification tasks.To this end, we select semantically correlated and domain-specific label terms within the context of scientific literature for verbalizer augmentation.Furthermore, we propose a new verbalization strategy that uses correlation scores as additional weights to enhance the prediction performance of the language model during model tuning.Our method outperforms stateof-the-art, prompt-based fine-tuning methods on scientific text classification tasks under few and zero-shot settings, especially in classifying fine-grained and emerging scientific topics 1 .

The Ukraine-Russia conflict has brought sizable detrimental impact to the global energy, food, finance, and manufacturing industries, as well to many affected people. In this paper, we use Twitter (now X) to automatically identify who needs what from text data and how the types of needs that we categorized and standardize evolved throughout this conflict. Our findings suggest that the Ukraine expresses a need for weapons, Russia for land, Europe for gas, and America for leadership. The majority of needs expressed on Twitter during this conflict are related to the categories transportation, military, health & medical, financial and money, energy, and essential items (food, water, shelter, non-food items). Stated needs changed as the conflict escalated or fell into stalemate. Needs also varied depending on the tweet's location, with tweets from Ukraine's neighboring countries being related to food and medicine, while tweets from non-neighboring countries stated needs for clothing and tents. Tweets written in Ukrainian and Russian shared similar need terms, such as medicines and kits, compared to English tweets, which expressed needs such as ammunition and humanitarian aid. Our comparison of needs across four different disaster events, namely this conflict, an earthquake, a major hurricane, and the COVID-19 pandemic, showed how needs differ depending on the nature of the crisis and how domain-adjustment of needs categories is necessary. We contribute to the crisis informatics literature by (1) validating a methodology for using tweets to study the demand and supply of things that different stakeholders need during crisis events and (2) testing, comparing, and improving the fit of widely used need classification schemas for studying crisis from different domains.

Traditional extractive summarization treats the task as sentence-level classification and requires a fixed number of sentences for extraction.However, this rigid constraint on the number of sentences to extract may hinder model generalization due to varied summary lengths across datasets.In this work, we leverage the interrelation between information extraction (IE) and text summarization, and introduce a fine-grained autoregressive method for extractive summarization through semantic tuple extraction.Specifically, we represent each sentence as a set of semantic tuples, where tuples are predicate-argument structures derived from conducting IE.Then we adopt a Transformerbased autoregressive model to extract the tuples corresponding to the target summary given a source document.In inference, a greedy approach is proposed to select source sentences to cover extracted tuples, eliminating the need for a fixed number.Our experiments on CNN/DM and NYT demonstrate the method's superiority over strong baselines.Through the zero-shot setting for testing the generalization of models to diverse summary lengths across datasets, we further show our method outperforms baselines, including ChatGPT.

Large Language Models (LLMs) have been observed to encode and perpetuate harmful associations present in the training data.We propose a theoretically grounded framework called STEREOMAP to gain insights into their perceptions of how demographic groups have been viewed by society.The framework is grounded in the Stereotype Content Model (SCM); a well-established theory from psychology.According to SCM, stereotypes are not all alike.Instead, the dimensions of Warmth and Competence serve as the factors that delineate the nature of stereotypes.Based on the SCM theory, STEREOMAP maps LLMs' perceptions of social groups (defined by sociodemographic features) using the dimensions of Warmth and Competence.Furthermore, the framework enables the investigation of keywords and verbalizations of reasoning of LLMs' judgments to uncover underlying factors influencing their perceptions.Our results show that LLMs exhibit a diverse range of perceptions towards these groups, characterized by mixed evaluations along the dimensions of Warmth and Competence.Furthermore, analyzing the reasonings of LLMs, our findings indicate that LLMs demonstrate an awareness of social disparities, often stating statistical data and research findings to support their reasoning.This study contributes to the understanding of how LLMs perceive and represent social groups, shedding light on their potential biases and the perpetuation of harmful associations.

Abstract Automated text categorization methods are of broad relevance for domain experts since they free researchers and practitioners from manual labeling, save their resources (e.g., time, labor), and enrich the data with information helpful to study substantive questions. Despite a variety of newly developed categorization methods that require substantial amounts of annotated data, little is known about how to build models when (a) labeling texts with categories requires substantial domain expertise and/or in‐depth reading, (b) only a few annotated documents are available for model training, and (c) no relevant computational resources, such as pretrained models, are available. In a collaboration with environmental scientists who study the socio‐ecological impact of funded biodiversity conservation projects, we develop a method that integrates deep domain expertise with computational models to automatically categorize project reports based on a small sample of 93 annotated documents. Our results suggest that domain expertise can improve automated categorization and that the magnitude of these improvements is influenced by the experts' understanding of categories and their confidence in their annotation, as well as data sparsity and additional category characteristics such as the portion of exclusive keywords that can identify a category.

Information extraction (IE) is a common sub-area of natural language processing that focuses on identifying structured data from unstructured data. One application domain of IE is Information Retrieval (IR), which relies on accurate and high-performance IE to retrieve high quality results from massive datasets. Another example of IE is to identify named entities in a text. For example, in the the sentence "Katy Perry lives in the USA", Katy Perry and USA are named entities of types of PERSON and LOCATION, respectively. Also, identify the sentiment expressed in a text is another instance of IE: in the sentence, "This movie was awesome", the expressed sentiment is positive. Finally, IE is concerned with identifying various linguistic aspects of text data, e.g., part of speech of words, noun phrases, dependency parses, etc., which can serve as features for additional IE tasks. This tutorial introduces participants to a) the usage of Python based, open-source tools that support IE from social media data (mainly Twitter), and b) best practices for ensuring the responsible use of IE and research data. Participants will learn and practice various lexical, semantic, and syntactic IE techniques that are commonly used for analyzing tweets. Participants will also be familiarized with the landscape of publicly available social media data (including popular NLP and IE benchmarks) and methods for collecting and preparing them for analysis. Furthermore, participants will be trained to use a suite of open source tools (SAIL for active learning, TwitterNER for named entity recognition, TweetNLP for transformer based NLP, and SocialMediaIE for multi task learning), which utilize advanced machine learning techniques (e.g., deep learning, active learning with human-in-the-loop, multi-lingual, and multi-task learning) to perform IE on their own or existing datasets. Participants will also learn how social contexts of text production and usage of results can be integrated into IE systems to improve these systems and to consider the role of time in improving social media IE quality. Finally, participants will learn about the governance of social media data for research purposes. The tools introduced in the tutorial will focus on the three main stages of IE, namely, collection of data (including annotation), data processing and analytics, and visualization of the extracted information. More details can be found at: https://socialmediaie.github.io/tutorials/

Yubin Ge, Ly Dinh, Xiaofeng Liu, Jinsong Su, Ziyao Lu, Ante Wang, Jana Diesner. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

This paper proposes, implements and evaluates a novel, corpus-based approach for identifying categories indicative of the impact of research via a deductive (top-down, from theory to data) and an inductive (bottom-up, from data to theory) approach. The resulting categorization schemes differ in substance. Research outcomes are typically assessed by using bibliometric methods, such as citation counts and patterns, or alternative metrics, such as references to research in the media. Shortcomings with these methods are their inability to identify impact of research beyond academia (bibliometrics) and considering text-based impact indicators beyond those that capture attention (altmetrics). We address these limitations by leveraging a mixed-methods approach for eliciting impact categories from experts, project personnel (deductive) and texts (inductive). Using these categories, we label a corpus of project reports per category schema, and apply supervised machine learning to infer these categories from project reports. The classification results show that we can predict deductively and inductively derived impact categories with 76.39% and 78.81% accuracy (F1-score), respectively. Our approach can complement solutions from bibliometrics and scientometrics for assessing the impact of research and studying the scope and types of advancements transferred from academia to society.

Observed social networks are often considered as proxies for underlying social networks. The analysis of observed networks oftentimes involves the identification of influential nodes via various centrality metrics. Our work is motivated by recent research on the investigation and design of adversarial attacks on machine learning systems. We apply the concept of adversarial attacks to social networks by studying strategies by which an adversary can minimally perturb the observed network structure to achieve their target function of modifying the ranking of nodes according to centrality measures. This can represent the attempts of an adversary to boost or demote the degree to which others perceive them as influential or powerful. It also allows us to study the impact of adversarial attacks on targets and victims, and to design metrics and security measures that help to identify and mitigate adversarial network attacks. We conduct a series of experiments on synthetic network data to identify attacks that allow the adversarial node to achieve their objective with a single move. We test this approach on different common network topologies and for common centrality metrics. We find that there is a small set of moves that result in the adversary achieving their objective, and this set is smaller for decreasing centrality metrics than for increasing them. These results can help with assessing the robustness of centrality measures. The notion of changing social network data to yield adversarial outcomes has practical implications, e.g., for information diffusion on social media, influence and power dynamics in social systems, and improving network security.

Ming Jiang, Junjie Hu, Qiuyuan Huang, Lei Zhang, Jana Diesner, Jianfeng Gao. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Ming Jiang, Qiuyuan Huang, Lei Zhang, Xin Wang, Pengchuan Zhang, Zhe Gan, Jana Diesner, Jianfeng Gao. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

An upcoming frontier for distributed computing might literally save lives in future military operations. In civilian scenarios, significant efficiencies were gained from interconnecting devices into networked services and applications that automate much of everyday life from smart homes to intelligent transportation. The ecosystem of such applications and services is collectively called the Internet of Things (IoT). Can similar benefits be gained in a military context by developing an IoT for the battlefield? This paper describes unique challenges in such a context as well as potential risks, mitigation strategies, and benefits.

We present novel research at the intersection of review mining and impact assessment of issue-focused information products, namely documentary films. We develop and evaluate a theoretically grounded classification schema, related codebook, corpus annotation, and prediction model for detecting multiple types of impact that documentaries can have on individuals, such as change versus reaffirmation of behavior, cognition, and emotions, based on user-generated content, i.e., reviews. This work broadens the scope of review mining tasks, which typically comprise the prediction of ratings, helpfulness, and opinions. Our results suggest that documentaries can change or reinforce peoples' conception of an issue. We perform supervised learning to predict impact on the sentence level by using data driven as well as predefined linguistic, lexical, and psychological features; achieving an accuracy rate of 81% (F1) when using a Random Forest classifier, and 73% with a Support Vector Machine.

We extend classic review mining work by building a binary classifier that predicts whether a review of a documentary film was written by an expert or a layman with 90.70% accuracy (F1 score), and compare the characteristics of the predicted classes. A variety of standard lexical and syntactic features was used for this supervised learning task. Our results suggest that experts write comparatively lengthier and more detailed reviews that feature more complex grammar and a higher diversity in their vocabulary. Layman reviews are more subjective and contextualized in peoples’ everyday lives. Our error analysis shows that laymen are about twice as likely to be mistaken as experts than vice versa. We argue that the type of author might be a useful new feature for improving the accuracy of predicting the rating, helpfulness and authenticity of reviews. Finally, the outcomes of this work might help researchers and practitioners in the field of impact assessment to gain a more fine-grained understanding of the perception of different types of media consumers and reviewers of a topic, genre or information product.

Scholars have often relied on name initials to resolve name ambiguities in large‐scale coauthorship network research. This approach bears the risk of incorrectly merging or splitting author identities. The use of initial‐based disambiguation has been justified by the assumption that such errors would not affect research findings too much. This paper tests that assumption by analyzing coauthorship networks from five academic fields—biology, computer science, nanoscience, neuroscience, and physics—and an interdisciplinary journal, PNAS. Name instances in data sets of this study were disambiguated based on heuristics gained from previous algorithmic disambiguation solutions. We use disambiguated data as a proxy of ground‐truth to test the performance of three types of initial‐based disambiguation. Our results show that initial‐based disambiguation can misrepresent statistical properties of coauthorship networks: It deflates the number of unique authors, number of components, average shortest paths, clustering coefficient, and assortativity, while it inflates average productivity, density, average coauthor number per author, and largest component size. Also, on average, more than half of top 10 productive or collaborative authors drop off the lists. Asian names were found to account for the majority of misidentification by initial‐based disambiguation due to their common surname and given name initials.

We present and test a scalable approach for assigning valence to links in unsigned graphs with the ultimate goal of enabling triadic balanced assessment in communication networks. We do this by applying domain-adjusted sentiment analysis to the content of communication data and translating aggregated sentiment scores for information exchanged between network members into link signs. This approach facilitates fast, informed and systematic balance testing (we generate link signs for 166,670 triads in our data); allowing for empirical hypothesis testing and theory building based on current or archival communication data. The proposed technique eliminates the need for manually labeling text data, and overcomes limitations with inferring valence from self-reported or user-generated (meta-) data in situations where historical context and ground truth valence data might be unavailable or limited. We test this approach on corporate email data to complement the large amount of prior work based on social media data and the limited knowledge on sentiment in professional settings. Our results suggest that sentiment is overall slightly positive and emotionality is low, which reflects conventions of language use in a corporate environment. We observe that people draw from (the top of) a smaller pool of positive terms more frequently than from a larger set of negative terms. The ratio of balanced triads (on average about 88%) to unbalanced triads (12%) remains relatively stable despite changes in corporate performance. The labor-intense adjustment of a given lexical resource to some dataset and domain pays off as it generates more empirical evidence with lower variance.

In many scientific fields, the order of coauthors on a paper conveys information about each individual's contribution to a piece of joint work. We argue that in prior network analyses of coauthorship networks, the information on ordering has been insufficiently considered because ties between authors are typically symmetrized. This is basically the same as assuming that each coauthor has contributed equally to a paper. We introduce a solution to this problem by adopting a coauthorship credit allocation model proposed by Kim and Diesner (2014), which in its core conceptualizes coauthoring as a directed, weighted, and self‐looped network. We test and validate our application of the adopted framework based on a sample data of 861 authors who have published in the journal Psychometrika. The results suggest that this novel sociometric approach can complement traditional measures based on undirected networks and expand insights into coauthoring patterns such as the hierarchy of collaboration among scholars. As another form of validation, we also show how our approach accurately detects prominent scholars in the Psychometric Society affiliated with the journal.

This paper illustrates how data pre-processing choices about author name disambiguation can affect research findings about scholarly networks and hypotheses about underlying social mechanisms. We have analyzed three big scholarly datasets that were disambiguated algorithmically and via two common initial-based disambiguation methods; namely first-initial and all-initials disambiguation. The comparison of resulting bibliometric and network properties revealed that initial-disambiguation bears the prevalent risks of incorrectly merging author identities, underestimating the number of unique authors and inflating the average productivity and number of collaborators per author. The gaps between outcomes of name ambiguity resolution methods range from −4.23% to −87.36% per dataset for the number of unique authors, from 3.75% to 691.20% for average productivity, and from 5.06% to 285.28% for degree centrality for initial based methods compared to algorithmic disambiguation. This calls for special attention to data pre-processing choices in scholarly big data research.

Definition acquisition is a necessary step in building an artificial cognitive assistant that helps military personnel to gain fast and precise understanding of the various terms and procedures defined in applicable legal documents. We approach the task of identifying definitional sentences from operations law documents by formalizing this task as a sentence-classification task and solving it by using machine-learning methods. This paper reports on a series of empirical experiments in that we evaluate and compare the performance of learning algorithms in terms of label-prediction accuracy. Using supervised techniques results in an F1 score of 95.93% and a 96.72% recall rate. However, for real-world applications, it would be too costly and unrealistic to ask personnel involved in military operations to label substantial amounts of data in order to build a new classifier for different types or genres of text data. Therefore, we propose and implement a semisupervised (SS) solution that trades off prediction accuracy to label efficiency. Our SS approach achieves a 90.47% F1 score and 93.44% recall rate by using only eight sentences labeled by a human expert.

Anaphora resolution (AR) identifies the entities that pronouns refer to. Coreference resolution (CR) associates the various instances of an entity with each other. Given our data, our findings suggest that deduplicating and normalizing text data by using AR and CR impacts the literal mention, frequency, identity, and existence of about 75% of the entities in texts. Results are more moderate on the relation level: 13% of the links are modified and 8% are removed. Performing social network analysis on the relations extracted from texts leads to findings contrary to the results from corpus statistics: AR and CR cause different directions in the change of network analytical measures, AR alters these measures more strongly than CR does, and each technique identifies a different set of most crucial nodes. Bringing the results from corpus statistics and social network analysis together suggests that CR is more effective in normalizing entities, while AR is a more powerful technique for splitting up generic nodes into named entities with adjusted weights. Data changes due to AR and CR are qualitatively and quantitatively meaningful: the statistical properties of entities and relations change along with their identities. Consequently, the relational data represent the underlying social structure more truthfully. Our results can support analysts in eliminating some misinterpretations of graphs distilled from texts and in selected those nodes from social networks on which reference resolution should be performed.