Abstract Information graphics (bar charts, line graphs, grouped bar charts, etc) often appear in popular media such as newspapers and magazines. In most cases, the information graphic is intended to convey a high‐level message. This message plays a role in facilitating the discourse purpose of the document but is seldom repeated in the document's text, headlines, or captions. We present a methodology and an implemented system for recognizing the intended message of a grouped bar chart. The recognition system relies on the following components: (1) a linguistic classifier that processes text in the graphic and predicts the most linguistically salient entity from those that are mentioned in text, (2) a cognitive model that estimates the relative perceptual effort required for an individual to recognize some high‐level message in a graph, and (3) a Bayesian network that captures the probabilistic relationship between the high‐level intended message of a graphic and its communicative signals. This research contributes to three applications: accessibility of information graphics for sight‐impaired individuals, retrieval of information graphics from a digital library, and summarization of multimodal documents.

Currently, there is a lack of text complexity awareness in NLG systems.Much attention has been given to text simplification.However, based upon results of an experiment, we unveiled that sophisticated readers in fact would rather read more sophisticated text, instead of the simplest text they could get.Therefore, we propose a technique that considers different readability levels during the micro planning phase of an NLG system.Our technique considers grammatical and syntactic choices, as well as lexical items, when generating text.The application uses the domain of descriptive summaries of line graphs as its use case.The technique proposed uses learning for identifying features of text complexity; a graph search algorithm for efficient aggregation given a target reading level, and a combination of language modeling and word vectors for the creation of a domain-aware synset which allows the creation of disambiguated lexicon that is appropriate to different reading levels.We found that generating text at different target reading levels is indeed preferred by readers with varying reading abilities.To the best of our knowledge, this is the first time readability awareness is considered in the micro planning phase of NLG systems.

User modeling and adaptation had its inception as a field at a workshop in Maria Laach, Germany in 1986. Most of the work at that time focused on applications in natural language processing, such as adapting explanations to the user's level of expertise. Since then, the field has grown tremendously and new applications are arising each year. As appropriate for the 30th anniversary of the first workshop, this talk will discuss how the field has evolved, novel work that we are pursuing on applying user modeling and adaptation to information retrieval, insights into where the field is headed and the hottest topics for exploration, and some thoughts on the conflict between the benefits of user modeling and its intrusion on people's lives.

This paper presents the methodology for generating textual summaries of line graphs in the SIGHT (Summarizing Information GrapHics Textually) system and the evaluation of line graph summaries produced by SIGHT. The system is designed to deliver the high-level knowledge conveyed by informational graphics present in online popular media articles (newspaper and magazines) to individuals who do not have visual access to the image. It works by producing and delivering a concise summary of the graph's content including the most important visual features present in the graphic. The system is briefly described; the evaluation compares the utility of the generated textual summaries to visually viewing the graphic in order to answer important questions about the line graph.

Deciding on the complexity of a generated text in NLG systems is a contentious task.Some systems propose the generation of simple text for low-skilled readers; some choose what they anticipate to be a "good measure" of complexity by balancing sentence length and number of sentences (using scales such as the D-level sentence complexity) for the text; while others target high-skilled readers.In this work, we discuss an approach that aims to leverage the experience of the reader when reading generated text by matching the syntactic complexity of the generated text to the reading level of the surrounding text.We propose an approach for sentence aggregation and lexical choice that allows generated summaries of line graphs in multimodal articles available online to match the reading level of the text of the article in which the graphs appear.The technique is developed in the context of the SIGHT (Summarizing Information Graphics Textually) system.This paper tackles the micro planning phase of sentence generation discussing additionally the steps of lexical choice, and pronominalization.

This demo presents a Natural Language Gener-ation (NLG) system that generates summaries of informational graphics, specifically simple line graphs, present in popular media. The sys-tem is intended to capture the high-level knowledge conveyed by the graphic and its out-standing visual features. It comprises a content selection phase that extracts the most important content of the graphic, an organization phase, which orders the propositions in a coherent manner, and a realization phase that uses the text surrounding the article to make decisions on the choice of lexical items and amount of ag-gregation applied to the propositions to gener-ate the summary of the graphic. 1

Information retrieval research has made significant progress in the retrieval of text documents and images. However, relatively little attention has been given to the retrieval of information graphics (non-pictorial images such as bar charts and line graphs) despite their proliferation in popular media such as newspapers and magazines. Our goal is to build a system for retrieving bar charts and line graphs that reasons about the content of the graphic itself in deciding its relevance to the user query. This paper presents the first steps toward such a system, with a focus on identifying the category of intended message of potentially relevant bar charts and line graphs. Our learned model achieves accuracy higher than 80\% on a corpus of collected user queries.

Information graphics (such as bar charts and line graphs) play a vital role in many multimodal documents. The majority of information graphics that appear in popular media are intended to convey a message and the graphic designer uses deliberate communicative signals, such as highlighting certain aspects of the graphic, in order to bring that message out. The graphic, whose communicative goal (intended message) is often not captured by the document's accompanying text, contributes to the overall purpose of the document and cannot be ignored. This article presents our approach to providing the high-level content of a non-scientific information graphic via a brief textual summary which includes the intended message and the salient features of the graphic. This work brings together insights obtained from empirical studies in order to determine what should be contained in the summaries of this form of non-linguistic input data, and how the information required for realizing the selected content can be extracted from the visual image and the textual components of the graphic. This work also presents a novel bottom–up generation approach to simultaneously construct the discourse and sentence structures of textual summaries by leveraging different discourse related considerations such as the syntactic complexity of realized sentences and clause embeddings. The effectiveness of our work was validated by different evaluation studies.

Although intelligent interactive systems have been the focus of many research efforts, very few have addressed systems for individuals with disabilities. This article presents our methodology for an intelligent interactive system that provides individuals with sight impairments with access to the content of information graphics (such as bar charts and line graphs) in popular media. The article describes the methodology underlying the system's intelligent behavior, its interface for interacting with users, examples processed by the implemented system, and evaluation studies both of the methodology and the effectiveness of the overall system. This research advances universal access to electronic documents.

Interactive SIGHT is a system that is intended to provide people with visual impairments access to the kind of information graphics found in popular media (i.e., electronic newspapers or magazines). The majority of such graphics are intended to convey a message; the graphic designer chose the graphic and its design in order to make a point. Interactive SIGHT, which is implemented as a browser extension that works on simple bar charts, provides a brief high-level summary of the graphic through natural language text that is conveyed to the user as speech. The user may request further information about the graphic through a follow-up question facility which allows many follow-up responses to be generated. The demo will illustrate the system's methodology on several bar charts that have been preloaded along with some accompanying text.

Information graphics are non-pictorial graphics such as bar charts and line graphs that depict attributes of entities and relations among entities. Most information graphics appearing in popular media have a communicative goal or intended message; consequently, information graphics constitute a form of language. This paper argues that information graphics are a valuable knowledge resource that should be retrievable from a digital library and that such graphics should be taken into account when summarizing a multimodal document for subsequent indexing and retrieval. But to accomplish this, the information graphic must be understood and its message recognized. The paper presents our Bayesian system for recognizing the primary message of one kind of information graphic (simple bar charts) and discusses the potential role of an information graphic's message in indexing graphics and summarizing multimodal documents.

This paper presents a corpus study that explores the extent to which captions contribute to recognizing the intended message of an information graphic. It then presents an implemented graphic interpretation system that takes into account a variety of communicative signals, and an evaluation study showing that evidence obtained from shallow processing of the graphic's caption has a significant impact on the system's success. This work is part of a larger project whose goal is to provide sight-impaired users with effective access to information graphics.

Transformation-Based Learning (TBL) is a relatively new machine learning method that has achieved notable success on language problems. This paper presents a variant of TBL, called Randomized TBL, that overcomes the training time problems of standard TBL without sacrificing accuracy. It includes a set of experiments on part-of-speech tagging in which the size of the corpus and template set are varied. The results show that Randomized TBL can address problems that are intractable in terms of training time for standard TBL. In addition, for language problems such as dialogue act tagging where the most effective features have not been identified through linguistic studies, Randomized TBL allows the researcher to experiment with a large set of templates capturing many potentially useful features and feature interactions.

A natural language collaborative consultation system must take user preferences into account. A model of user preferences allows a system to appropriately evaluate alternatives using criteria of importance to the user. Additionally, decision research suggests both that an accurate model of user preferences could enable the system to improve a user's decision‐making by ensuring that all important alternatives are considered, and that such a model of user preferences must be built dynamically by observing the user's actions during the decision‐making process. This paper presents two strategies: one for dynamically recognizing user preferences during the course of a collaborative planning dialogue and the other for exploiting the model of user preferences to detect suboptimal solutions and suggest better alternatives. Our recognition strategy utilizes not only the utterances themselves but also characteristics of the dialogue in developing a model of user preferences. Our generation strategy takes into account both the strength of a preference and the closeness of a potential match in evaluating actions in the user's plan and suggesting better alternatives. By modeling and utilizing user preferences, our system is able to fulfill its role as a collaborative agent.

In collaborative planning activities, since the agents are autonomous and heterogeneous, it is inevitable that conflicts arise in their beliefs during the planning process.In cases where such conflicts are relevant to the t~t~k at hand, the agents should engage in collaborative negotiation as an attempt to square away the discrepancies in their beliefs.This paper presents a computational strategy for detecting conflicts regarding proposed beliefs and for engaging in collaborative negotiation to resolve the conflicts that warrant resolution.Our model is capable of selecting the most effective aspect to address in its pursuit of conflict resolution in cases where multiple conflicts arise, and of selecting appropriate evidence to justify the need for such modification.Furthermore, by capturing the negotiation process in a recursive Propose-Evaluate.Modify cycle of actions, our model can successfully handle embedded negotiation subdialogues.

This paper presents our implemented computational model for interpreting and generating indirect answers to Yes-No questions. Its main features are 1) a discourse-plan-based approach to implicature, 2) a reversible architecture for generation and interpretation, 3) a hybrid reasoning model that employs both plan inference and logical inference, and 4) use of stimulus conditions to model a speaker's motivation for providing appropriate, unrequested information. The model handles a wider range of types of indirect answers than previous computational models and has several significant advantages.

Americanae nace como un proyecto conjunto que surge dentro de la Red Europea de Información y Documentación sobre América Latina (REDIAL), y que ha afrontado la Biblioteca de la Agencia Española de Cooperación Internacional para el Desarrollo (AECID). Esta nueva biblioteca virtual hace más accesibles los libros digitales de tema americanista a los investigadores y usuarios interesados de cualquier parte del mundo.

This paper presents a plan-based model that handles negotiation subdialogues by inferring both the communicative actions that people pursue when speaking and the beliefs underlying these actions. We contend that recognizing the complex discourse actions pursued in negotiation subdialogues (e.g., expressing doubt) requires both a multistrength belief model and a process model that combines different knowledge sources in a unified framework. We show how our model identifies the structure of negotiation subdialogues, including recognizing expressions of doubt, implicit acceptance of communicated propositions, and negotiation subdialogues embedded within other negotiation subdialogues.

This paper presents a tripartite model of dialogue in which three different kinds of actions are modeled: domain actions, problem-solving actions, and discourse or communicative actions. We contend that our process model provides a more finely differentiated representation of user intentions than previous models; enables the incremental recognition of communicative actions that cannot be recognized from a single utterance alone; and accounts for implicit acceptance of a communicated proposition.

Intersentential elliptical utterances occur frequently in information-seeking dialogues. This paper presents a pragmatics-based framework for interpreting such utterances, including identification of the speaker's discourse goal in employing the fragment. We claim that the advantage of this approach is its reliance upon pragmatic information, including discourse content and conversational goals, rather than upon precise representations of the preceding utterance alone.

An utterance may be syntactically and semantically well-formed yet violate the pragmatic rules of the world model. This paper presents a context-based strategy for constructing a cooperative but limited response to pragmatically ill-formed queries. Suggestion heuristics use a context model of the speaker's task inferred from the preceding dialogue to propose revisions to the speaker's ill-formed query. Selection heuristics then evaluate these suggestions based upon semantic and relevance criteria.