Social media platforms are popular tools for disseminating targeted information during major public events like elections or pandemics. Systematic analysis of the message traffic can provide valuable insights into prevailing opinions and social dynamics among different segments of the population. We are specifically interested in influence spread, and in particular whether more deliberate influence operations can be detected. However, filtering out the essential messages with telltale influence indicators from the extensive and often chaotic social media traffic is a major challenge.In this paper we present a novel approach to extract influence indicators from messages circulating among groups of users discussing particular topics. We build upon the the concept of a convo to identify influential authors who are actively promoting some particular agenda around that topic within the group. We focus on two influence indicators: the (control of) agenda and the use of emotional language.

The behavior and decision making of groups or communities can be dramatically influenced by individuals pushing particular agendas, e.g., to promote or disparage a person or an activity, to call for action, etc.. In the examination of online influence campaigns, particularly those related to important political and social events, scholars often concentrate on identifying the sources responsible for setting and controlling the agenda (e.g., public media). In this article we present a methodology for detecting specific instances of agenda control through social media where annotated data is limited or non-existent. By using a modest corpus of Twitter messages centered on the 2022 French Presidential Elections, we carry out a comprehensive evaluation of various approaches and techniques that can be applied to this problem. Our findings demonstrate that by treating the task as a textual entailment problem, it is possible to overcome the requirement for a large annotated training dataset.

We present the BeSt corpus, which records cognitive state: who believes what (i.e., factuality), and who has what sentiment towards what. This corpus is inspired by similar source-and-target corpora, specifically MPQA and FactBank. The corpus comprises two genres, newswire and discussion forums, in three languages, Chinese (Mandarin), English, and Spanish. The corpus is distributed through the LDC.

Abraham Sanders, Tomek Strzalkowski, Mei Si, Albert Chang, Deepanshu Dey, Jonas Braasch, Dakuo Wang. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Social engineers attempt to manipulate users into undertaking actions such as downloading malware by clicking links or providing access to money or sensitive information. Natural language processing, computational sociolinguistics, and media-specific structural clues provide a means for detecting both the ask (e.g., buy gift card) and the risk/reward implied by the ask, which we call framing (e.g., lose your job, get a raise). We apply linguistic resources such as Lexical Conceptual Structure to tackle ask detection and also leverage structural clues such as links and their proximity to identified asks to improve confidence in our results. Our experiments indicate that the performance of ask detection, framing detection, and identification of the top ask is improved by linguistically motivated classes coupled with structural clues such as links. Our approach is implemented in a system that informs users about social engineering risk situations.

Abstract In this paper, we describe a novel approach to computational modeling and understanding of social and cultural phenomena in multi-party dialogues. We developed a two-tier approach in which we first detect and classify certain sociolinguistic behaviors, including topic control, disagreement, and involvement, that serve as first-order models from which presence the higher level social roles, such as leadership, may be inferred.

A social robot is a robotic platform that supports natural interaction with people in a human-scale environment. Such a platform allows interesting opportunities for both traditional Computer Science students and students from other disciplines, such as psychology, philosophy, design and communications. In this paper, we describe a new social robotic platform for educational uses that is equipped with a social face, arms for gesturing, advanced sensory, mobile base, and ROS integration. By using off-the-shelf and rapidly prototyped components, together with open source software, this platform is low-cost, easy to use, and easy to reproduce.

Abstract We describe an interactive question answering system, HITIQA, which helps users find answers to complex analytical problems. Such problems often necessitate the user to submit not one but an entire series of questions, both simple and complex, and then to negotiate the final content and form of the answer. HITIQA advances research in human–computer dialogue by enabling topical, mixed initiative interaction over unstructured data. HITIQA uses the process of text framing to bring a level of semantic representation to open-domain data in order to facilitate meaningful dialogue with the user. In this paper we give an overview of HITIQA's design and explain the workings of its main components with particular attention given to its dialogue capabilities. We also present results of end-to-end system evaluations that demonstrate the effectiveness of the system as a whole, as well as contributions of the individual components and specifically the benefits of our dialogue-based approach. While our research continues, a number of HITIQA prototypes have recently been deployed at various government agencies where they are being tested under real operational conditions.

Abstract The purpose of this work is to identify potential evaluation criteria for interactive, analytical question‐answering (QA) systems by analyzing evaluative comments made by users of such a system. Qualitative data collected from intelligence analysts during interviews and focus groups were analyzed to identify common themes related to performance, use, and usability. These data were collected as part of an intensive, three‐day evaluation workshop of the High‐Quality Interactive Question Answering (HITIQA) system. Inductive coding and memoing were used to identify and categorize these data. Results suggest potential evaluation criteria for interactive, analytical QA systems, which can be used to guide the development and design of future systems and evaluations. This work contributes to studies of QA systems, information seeking and use behaviors, and interactive searching.

Abstract We describe a procedure for quantitative evaluation of interactive question‐answering systems and illustrate it with application to the High‐Quality Interactive Question‐Answering (HITIQA) system. Our objectives were (a) to design a method to realistically and reliably assess interactive question‐answering systems by comparing the quality of reports produced using different systems, (b) to conduct a pilot test of this method, and (c) to perform a formative evaluation of the HITIQA system. Far more important than the specific information gathered from this pilot evaluation is the development of (a) a protocol for evaluating an emerging technology, (b) reusable assessment instruments, and (c) the knowledge gained in conducting the evaluation. We conclude that this method, which uses a surprisingly small number of subjects and does not rely on predetermined relevance judgments, measures the impact of system change on work produced by users. Therefore this method can be used to compare the product of interactive systems that use different underlying technologies.

In this paper, we discuss how to utilize the co-occurrence of answers in building and automatic question answering system that answers a series of questions on a specific topic in a batch mode. Experiments show that the answers to the many of the questions in the series usually have a high degree of co-occurrence in relevant document passages. This feature sometimes can't be easily utilized in an automatic QA system which processes questions independently. However it can be utilized in a QA system that processes questions in a batch mode. We have used our pervious TREC QA system as baseline and augmented it with new answer clustering and co-occurrence maximization components to build the batch QA system. The experiment results show that the QA system running under the batch mode get significant performance improvement over our baseline TREC QA system.

Abstract The authors report on a series of experiments to automate the assessment of document qualities such as depth and objectivity. The primary purpose is to develop a quality‐sensitive functionality, orthogonal to relevance, to select documents for an interactive question‐answering system. The study consisted of two stages. In the classifier construction stage, nine document qualities deemed important by information professionals were identified and classifiers were developed to predict their values. In the confirmative evaluation stage, the performance of the developed methods was checked using a different document collection. The quality prediction methods worked well in the second stage. The results strongly suggest that the best way to predict document qualities automatically is to construct classifiers on a person‐by‐person basis.

We present a prototype natural-language problem-solving application for a financial services call center, developed as part of the Amitiés multilingual human-computer dialogue project. Our automated dialogue system, based on empirical evidence from real call-center conversations, features a data-driven approach that allows for mixed system/customer initiative and spontaneous conversation. Preliminary evaluation results indicate efficient dialogues and high user satisfaction, with performance comparable to or better than that of current conversational travel information systems.

One central goal of the AMITIÉS multilingual human-computer dialogue project is to create a dialogue management system capable of engaging the user in human-like conversation in a specific domain. To that end, we have developed new methods for the manual annotation of spoken dialogue transcriptions from European financial call centers. We have modified the DAMSL dialogic schema to create a dialogue act taxonomy appropriate for customer services. To capture the semantics, we use a domain-independent framework populated with domain-specific lists. We have designed a new flexible, platform-independent annotation tool, XDMLTool, and annotated several hundred dialogues in French and English. Inter-annotator agreement was moderate. We are using these data to design our dialogue system, and we hope that they will help us to derive appropriate dialogue strategies for novel situations. 1.

In this paper we describe a Cross Document Summarizer XDoX designed specifically to summarize large document sets (50-500 documents and more). Such sets of documents are typically obtained from routing or filtering systems run against a continuous stream of data, such as a newswire. XDoX works by identifying the most salient themes within the set (at the granularity level that is regulated by the user) and composing an extraction summary, which reflects these main themes. In the current version, XDoX is not optimized to produce a summary based on a few unrelated documents; indeed, such summaries are best obtained simply by concatenating summaries of individual documents. We show examples of summaries obtained in our tests as well as from our participation in the first Document Understanding Conference (DUC).

This paper describes a system for the summarization of multiple text‐only news‐like documents. We address two main issues: clustering of documents in order to find the main topics that should be mentioned in the multidocument summary and organization of the information in order to create a summary that presents the information in a logical way and is easy to read. The system is based on a single‐document summarizer that uses text‐extraction.

In this paper we compare two grammar-based generation algorithms: the Semantic-Head-Driven Generation Algorithm (SHDGA), and the Essential Arguments Algorithm (EAA). Both algorithms have successfully addressed several outstanding problems in grammar-based generation, including dealing with non-monotonic compositionality of representation, left-recursion, deadlock-prone rules, and nondeterminism. We concentrate here on the comparison of selected properties: generality, efficiency, and determinism. We show that EAA's traversals of the analysis tree for a given language construct, include also the one taken on by SHDGA. We also demonstrate specific and common situations in which SHDGA will invariably run into serious inefficiency and nondeterminism, and which EAA will handle in an efficient and deterministic manner. We also point out that only EAA allows to treat the underlying grammar in a truly multi-directional manner.

We developed a prototype information retrieval system which uses advanced natural language processing techniques to enhance the effectiveness of traditional key-word based document retrieval. The backbone of our system is a statistical retrieval engine which performs automated indexing of documents, then search and ranking in response to user queries. This core architecture is augmented with advanced natural language processing tools which are both robust and efficient. In early experiments, the augmented system has displayed capabilities that appear to make it superior to the purely statistical base.

We describe a system of reversible grammar in which, given a logic-grammar specification of a natural language, two efficient PROLOG programs are derived by an off-line compilation process: a parser and a generator for this language. The centerpiece of the system is the inversion algorithm designed to compute the generator code from the parser's PROLOG code, using the collection of minimal sets of essential arguments (MSEA) for predicates. The system has been implemented to work with Definite Clause Grammars (DCG) and is a part of an English-Japanese machine translation project currently under development at NYU's Courant Institute.

The use of a single grammar in natural language parsing and generation is most desirable for a variety of reasons, including efficiency, perspicuity, integrity, robustness, and a certain amount of elegance. These characteristics have been noted before by several researchers, but it was only recently that more serious attention started to be paid to the problem of creating a bidirectional system for natural language processing. In this paper we discuss a somewhat more radical version of the problem: given a parser for a language, can we reverse it so that it becomes an efficient generator for the same language? Furthermore, since both the parser and the generator are based upon the same grammar, are there any normalization conditions upon the form of the grammar that must be met in order to assure the maximum efficiency of the reversed program? Can other grammars be transformed into the normal form? We describe the results of an experiment with PROLOG‐based logic grammar which has been derived from a substantial‐coverage string grammar for English. We present an alogorithm for automated inversion of a unification parser into an efficient unification generator, using the collections of minimal sets of essential arguments for predicates. We discuss the scope of the present version of the algorithm and then point out several possible avenues for extension. We also outline a preliminary solution to the question of grammar's “normal form” and suggest a handful of normalizing transformations that can be used to enhance the efficiency of the generator. This research interacts closely with a Japanese‐English machine translation project at New York University, for which the first implementation of the inversion algorithm has been prepared.

We are concerned with developing a computational method for selecting possible antecedents of referring expressions over sentence boundaries. Our stratified model which uses a Λ‐categorial language for meaning representation incorporates valuable features of Fregean‐type semantics (a la Lewis, Montague, Partee, and others) along with features of situation semantics developed by Barwise and Perry. We consider a series of selected two‐sentence stories which we use to illustrate referential interdependencies between sentences. We explain the conditions under which such dependencies arise, explain the conditions under which various translations can be performed, and formalize a set of rules which specify how to compute the reference. We restrict our discussion to two‐sentence stories to avoid most of the problems inherent in where to look for the reference, that is, how to determine the proper antecedent. We restrict our considerations in this paper to situations where a reference, if it can be computed at all, has a unique antecedent. Thus we consider examples such as John wants to catch a fish. He (John) wants to eat it. and John interviewed a man. The man killed him (John). We then summarize the transformation which encompasses these rules and relate it to the stratified model. We discuss three aspects of this transformation that merit special attention from the computational viewpoint and summarize the contributions we have made. We also discuss the computational characteristics of the stratified model in general and present our ideas for a computer realization; there is no implementation of the t“ratified model at this time.