We introduce Taxonomy-Free Character Role Labeling (TF-CRL); a novel task that assigns open-ended narrative role labels to characters in news stories based on their functional role in the narrative.Unlike fixed taxonomies, TF-CRL enables more nuanced and comparative analysis by generating compositional labels (e.g., Resilient Leader, Scapegoated Visionary).We evaluate several large language models (LLMs) on this task using human preference rankings and ratings across four criteria: faithfulness, relevance, informativeness, and generalizability.LLMs almost uniformly outperform human annotators across all dimensions.We further show how TF-CRL supports rich narrative analysis by revealing novel latent taxonomies and enabling cross-domain narrative comparisons.Our approach offers new tools for studying media portrayals, character framing, and the socio-political impacts of narrative roles at-scale.1

Stories are not only designed to entertain but to encode lessons reflecting their authors' beliefs about the world.In this paper, we propose a new task of narrative schema labelling based on the concept of "story morals" to identify the values and lessons conveyed in stories.Using large language models (LLMs) such as GPT-4, we develop methods to automatically extract and validate story morals across a diverse set of narrative genres, including folktales, novels, movies and TV, personal stories from social media, and the news.Our approach involves a multi-step prompting sequence to derive morals and validate them through both automated metrics and human assessments.The findings suggest that LLMs can effectively approximate human story moral interpretations and offer a new avenue for computational narrative understanding.By clustering the extracted morals on a sample dataset of folktales from around the world, we highlight the commonalities and distinctiveness of narrative values, providing preliminary insights into the distribution of values across cultures.This work opens up new possibilities for studying narrative schemas and their role in shaping human beliefs and behaviors.1

Abusive language in online discourse negatively affects a large number of social media users.Many computational methods have been proposed to address this issue of online abuse.The existing work, however, tends to focus on detecting the more explicit forms of abuse leaving the subtler forms of abuse largely untouched.Our work addresses this gap by making three core contributions.First, inspired by the theory of impoliteness, we propose a novel task of detecting a subtler form of abuse, namely unpalatable questions.Second, we publish a context-aware dataset for the task using data from a diverse set of Reddit communities.Third, we implement a wide array of learning models and also investigate the benefits of incorporating conversational context into computational models.Our results show that modeling subtle abuse is feasible but difficult due to the language involved being highly nuanced and context-sensitive.We hope that future research in the field will address such subtle forms of abuse since their harm currently passes unnoticed through existing detection systems.

Deep neural networks have been displaying superior performance over traditional supervised classifiers in text classification.They learn to extract useful features automatically when sufficient amount of data is presented.However, along with the growth in the number of documents comes the increase in the number of categories, which often results in poor performance of the multiclass classifiers.In this work, we use external knowledge in the form of topic category taxonomies to aide the classification by introducing a deep hierarchical neural attention-based classifier.Our model performs better than or comparable to state-of-the-art hierarchical models at significantly lower computational cost while maintaining high interpretability.

Sentiment analysis is used as a proxy to measure human emotion, where the objective is to categorize text according to some predefined notion of sentiment.Sentiment analysis datasets are typically constructed with gold-standard sentiment labels, assigned based on the results of manual annotations.When working with such annotations, it is common for dataset constructors to discard "noisy" or "controversial" data where there is significant disagreement on the proper label.In datasets constructed for the purpose of Twitter sentiment analysis (TSA), these controversial examples can compose over 30% of the originally annotated data.We argue that the removal of such data is a problematic trend because, when performing real-time sentiment classification of short-text, an automated system cannot know a priori which samples would fall into this category of disputed sentiment.We therefore propose the notion of a "complicated" class of sentiment to categorize such text, and argue that its inclusion in the short-text sentiment analysis framework will improve the quality of automated sentiment analysis systems as they are implemented in real-world settings.We motivate this argument by building and analyzing a new publicly available TSA dataset of over 7,000 tweets annotated with 5x coverage, named MTSA.Our analysis of classifier performance over our dataset offers insights into sentiment analysis dataset and model design, how current techniques would perform in the real world, and how researchers should handle difficult data.

When reporting the news, journalists rely on the statements of stakeholders, experts, and officials. The attribution of such a statement is verifiable if its fidelity to the source can be confirmed or denied. In this paper, we develop a new NLP task: determining the verifiability of an attribution based on linguistic cues. We operationalize the notion of verifiability as a score between 0 and 1 using human judgments in a comparison-based approach. Using crowdsourcing, we create a dataset of verifiability-scored attributions, and demonstrate a model that achieves an RMSE of 0.057 and Spearman’s rank correlation of 0.95 to human-generated scores. We discuss the application of this technique to the analysis of mass media.

Anaphor resolution is an important task in NLP with many applications.Despite much research effort, it remains an open problem.The difficulty of the problem varies substantially across different sub-problems.One sub-problem, in particular, has been largely untouched by prior work despite occurring frequently throughout corpora: the anaphor that has multiple antecedents, which here we call multi-antecedent anaphors or manaphors.Current coreference resolvers restrict anaphors to at most a single antecedent.As we show in this paper, relaxing this constraint poses serious problems in coreference chain-building, where each chain is intended to refer to a single entity.This work provides a formalization of the new task with preliminary insights into multi-antecedent noun-phrase anaphors, and offers a method for resolving such cases that outperforms a number of baseline methods by a significant margin.Our system uses local agglomerative clustering on candidate antecedents and an existing coreference system to score clusters to determine which cluster of mentions is antecedent for a given anaphor.When we augment an existing coreference system with our proposed method, we observe a substantial increase in performance (0.6 absolute CoNLL F1) on an annotated corpus.

Microtask platforms are becoming commonplace tools for performing human research, producing gold-standard data, and annotating large datasets. These platforms connect requesters (researchers or companies) with large populations (crowds) of workers, who perform small tasks, typically taking less than five minutes each. A topic of ongoing research concerns the design of tasks that elicit high quality annotations. Here we identify a seemingly banal feature of nearly all crowdsourcing workflows that profoundly impacts workers' responses. Microtask assignments typically consist of a sequence of tasks sharing a common format (e.g., circle galaxies in an image). Using image-labeling, a canonical microtask format, we show that earlier tasks can have a strong influence on responses to later tasks, shifting the distribution of future responses by 30-50% (total variational distance). Specifically, prior tasks influence the content that workers focus on, as well as the richness and specialization of responses. We call this phenomenon intertask effects. We compare intertask effects to framing, effected by stating the requester's research interest, and find that intertask effects are on par or stronger. If uncontrolled, intertask effects could be a source of systematic bias, but our results suggest that, with appropriate task design, they might be leveraged to hone worker focus and acuity, helping to elicit reproducible, expert-level judgments. Intertask effects are a crucial aspect of human computation that should be considered in the design of any crowdsourced study.

Characters are fundamental to literary analysis.Current approaches are heavily reliant on NER to identify characters, causing many to be overlooked.We propose a novel technique for character detection, achieving significant improvements over state of the art on multiple datasets.

While much work has considered the problem of latent attribute inference for users of social media such as Twitter, little has been done on non-English-based content and users.Here, we conduct the first assessment of latent attribute inference in languages beyond English, focusing on gender inference.We find that the gender inference problem in quite diverse languages can be addressed using existing machinery.Further, accuracy gains can be made by taking language-specific features into account.We identify languages with complex orthography, such as Japanese, as difficult for existing methods, suggesting a valuable direction for future research.

BACKGROUND: Because biological networks exhibit a high-degree of robustness, a systemic understanding of their architecture and function requires an appraisal of the network design principles that confer robustness. In this project, we conduct a computational study of the contribution of three degree-based topological properties (transcription factor-target ratio, degree distribution, cross-talk suppression) and their combinations on the robustness of transcriptional regulatory networks. We seek to quantify the relative degree of robustness conferred by each property (and combination) and also to determine the extent to which these properties alone can explain the robustness observed in transcriptional networks. RESULTS: To study individual properties and their combinations, we generated synthetic, random networks that retained one or more of the three properties with values derived from either the yeast or E. coli gene regulatory networks. Robustness of these networks were estimated through simulation. Our results indicate that the combination of the three properties we considered explains the majority of the structural robustness observed in the real transcriptional networks. Surprisingly, scale-free degree distribution is, overall, a minor contributor to robustness. Instead, most robustness is gained through topological features that limit the complexity of the overall network and increase the transcription factor subnetwork sparsity. CONCLUSIONS: Our work demonstrates that (i) different types of robustness are implemented by different topological aspects of the network and (ii) size and sparsity of the transcription factor subnetwork play an important role for robustness induction. Our results are conserved across yeast and E Coli, which suggests that the design principles examined are present within an array of living systems.