James F. Allen

dblp:a/JFAllen · DBLP profile ↗
← Back
87ranked-venue papers
25as first author
0since 2021 · last 2020
0000-0003-4543-5457ORCID · corroborated

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 75 · 21 first-authorGraphics, computer vision, multimedia, augmented reality and games · 33 · 7 first-authorTheory of computation · 5 · 2 first-authorApplied, interdisciplinary, general and emerging computing · 5 · 2 first-authorHuman-computer interaction and ubiquitous computing · 4 · 1 first-authorDatabases, data management, data science and information retrieval · 1 · 1 first-author

Expertise — from the expertise taxonomy: the topics of the expert's papers under the CCF categories. A weight counts papers with recency: 1 for a paper about the topic, 0.3 when the topic is its context, halved every five years.

Artificial intelligence
33 papers
Information extraction and text analysis · 48% Knowledge representation and reasoning · 22% Question answering and dialogue systems · 11%
Human-computer interaction and pervasive computing
6 papers
Human-AI interaction · 57% Collaborative and social computing · 37% Ubiquitous computing and smart environments · 5%
Software engineering, system software, and programming languages
1 paper
Program synthesis and code generation · 100%

Topics — the 30 heaviest of 56, each with the papers that count most for it

TopicWeightPapersLastEvidence papers
Natural language and speech › Information extraction and text analysis
semantic parsing
0.822020
Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract) · AAAI 2020
Effective Broad-Coverage Deep Parsing · AAAI 2018
Human-AI interaction › large language model interaction › language-based interaction
natural language interface
0.522017
Natural Language Dialogue for Building and Learning Models and Structures · AAAI 2017
Integrating Programming by Example and Natural Language Programming · AAAI 2013
Natural language and speech › Information extraction and text analysis
word sense disambiguation
0.412020
Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract) · AAAI 2020
Collaborative and social computing
crowdsourcing
0.322013
Chorus: a crowd-powered conversational assistant · UIST 2013
Real-Time Collaborative Planning with the Crowd · AAAI 2012
Natural language and speech › Information extraction and text analysis › entity typing
entity understanding
0.312017
Apples to Apples: Learning Semantics of Common Entities Through a Novel Comprehension Task · ACL (1) 2017
Knowledge, reasoning and agents › Knowledge representation and reasoning
knowledge acquisition
0.312017
Natural Language Dialogue for Building and Learning Models and Structures · AAAI 2017
Natural language and speech › Question answering and dialogue systems
machine reading comprehension
0.312017
Apples to Apples: Learning Semantics of Common Entities Through a Novel Comprehension Task · ACL (1) 2017
Knowledge, reasoning and agents › Knowledge representation and reasoning
semantic representation
0.212015
Semantic Framework for Comparison Structures in Natural Language · EMNLP 2015
Computer vision › Vision and language
grounded language learning
0.212013
SALL-E: Situated Agent for Language Learning · AAAI 2013
Program synthesis and code generation
natural language programming
0.212013
Integrating Programming by Example and Natural Language Programming · AAAI 2013
Program synthesis and code generation
programming by example
0.212013
Integrating Programming by Example and Natural Language Programming · AAAI 2013
Knowledge, reasoning and agents › Planning, search and constraint satisfaction › multi-agent planning
collaborative planning
0.112012
Real-Time Collaborative Planning with the Crowd · AAAI 2012
Knowledge, reasoning and agents › Knowledge representation and reasoning › semantic representation
logical form
0.112020
Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract) · AAAI 2020
Natural language and speech › Information extraction and text analysis › natural language semantics › computational semantics
semantic role assignment
0.112020
Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract) · AAAI 2020
Knowledge, reasoning and agents › Planning, search and constraint satisfaction › plan recognition
goal recognition
0.122006
Fast Hierarchical Goal Schema Recognition · AAAI 2006
Corpus-based, Statistical Goal Recognition · IJCAI 2003
Bioinformatics and computational biology
biomedical text mining
0.112018
Effective Broad-Coverage Deep Parsing · AAAI 2018
Knowledge, reasoning and agents › Planning, search and constraint satisfaction
task planning
0.112017
Natural Language Dialogue for Building and Learning Models and Structures · AAAI 2017
Natural language and speech › Question answering and dialogue systems
spoken dialogue systems
0.122005
Two Diverse Systems Built using Generic Components for Spoken Dialogue (Recent Progress on TRIPS) · ACL 2005
A Robust System for Natural Spoken Dialogue · ACL 1996
Natural language and speech › Information extraction and text analysis
semantic role labeling
0.112015
Semantic Framework for Comparison Structures in Natural Language · EMNLP 2015
Natural language and speech › Question answering and dialogue systems › natural language interface
conversational interfaces
0.012013
Chorus: a crowd-powered conversational assistant · UIST 2013
Ubiquitous computing and smart environments › context recognition
activity recognition
0.012012
Learning Names for RFID-Tagged Objects in Activity Videos · AAAI 2012
Natural language and speech › Speech recognition and synthesis › spontaneous speech processing
speech repair detection
0.021997
Intonational Boundaries, Speech Repairs and Discourse Markers: Modeling Spoken Dialog · ACL 1997
Deyecting and Correcting Speech Repairs · ACL 1994
Natural language and speech › Speech recognition and synthesis › spoken language understanding
spoken language processing
0.021997
Intonational Boundaries, Speech Repairs and Discourse Markers: Modeling Spoken Dialog · ACL 1997
Deyecting and Correcting Speech Repairs · ACL 1994
Knowledge, reasoning and agents › Knowledge representation and reasoning
temporal reasoning
0.071991
Planning as Temporal Reasoning · KR 1991
Short Time Periods · IJCAI 1987
A Model for Concurrent Actions Having Temporal Extent · AAAI 1987
Knowledge, reasoning and agents › Planning, search and constraint satisfaction
plan recognition
0.022006
Fast Hierarchical Goal Schema Recognition · AAAI 2006
Generalized Plan Recognition · AAAI 1986
Natural language and speech › Information extraction and text analysis
discourse analysis
0.011997
Intonational Boundaries, Speech Repairs and Discourse Markers: Modeling Spoken Dialog · ACL 1997
Natural language and speech › Question answering and dialogue systems
dialogue modeling
0.011994
Discourse Obligations in Dialogue Processing · ACL 1994
Logic in computer science
temporal logic
0.021986
A formal logic of plans in temporally rich domains · Proc. IEEE 1986
Towards a General Theory of Action and Time · Artif. Intell. 1984
Knowledge, reasoning and agents › Knowledge representation and reasoning › reasoning about action and change
action representation
0.021987
A Model for Concurrent Actions Having Temporal Extent · AAAI 1987
What's Necessary to Hide?: Modeling Action Verbs · ACL 1981
Knowledge, reasoning and agents › Knowledge representation and reasoning › reasoning about action and change
plan inference
0.011989
Two Constraints on Speech Act Ambiguity · ACL 1989

Methods — techniques the papers use, named apart from their topics

ablation experiments · 0.7semantic parsing · 0.6concept composition · 0.6statistical word sense disambiguation · 0.4regular expressions · 0.3game-theoretic incentive mechanism · 0.3semantic-driven model · 0.3neural model · 0.3memory-based classifier · 0.3worker voting · 0.2shared memory · 0.2mahalanobis distance · 0.2k-nearest neighbors · 0.2real-time crowdsourcing · 0.1ontological categorization · 0.1automated planning · 0.1anaphoric reference resolution · 0.1modal logic · 0.0
YearPublicationVenuePosition
2020 Improving Semantic Parsing Using Statistical Word Sense Disambiguation (Student Abstract)
abstract
A Semantic Parser generates a logical form graph from an utterance where the edges are semantic roles and nodes are word senses in an ontology that supports reasoning. The generated representation attempts to capture the full meaning of the utterance. While the process of parsing works to resolve lexical ambiguity, a number of errors in the logical forms arise from incorrectly assigned word sense determinations. This is especially true in logical and rule-based semantic parsers. Although the performance of statistical word sense disambiguation methods is superior to the word sense output of semantic parser, these systems do not produce the rich role structure or a detailed semantic representation of the sentence content. In this work, we use decisions from a statistical WSD system to inform a logical semantic parser and greatly improve semantic type assignments in the resulting logical forms.
Ritwik Bose, Siddharth Vashishtha, James F. Allen
AAAI3
2020 A Broad-Coverage Deep Semantic Lexicon for Verbs
abstract
Progress on deep language understanding is inhibited by the lack of a broad coverage lexicon that connects linguistic behavior to ontological concepts and axioms. We have developed COLLIE-V, a deep lexical resource for verbs, with the coverage of WordNet and syntactic and semantic details that meet or exceed existing resources. Bootstrapping from a hand-built lexicon and ontology, new ontological concepts and lexical entries, together with semantic role preferences and entailment axioms, are automatically derived by combining multiple constraints from parsing dictionary definitions and examples. We evaluated the accuracy of the technique along a number of different dimensions and were able to obtain high accuracy in deriving new concepts and lexical entries. COLLIE-V is publicly available.
James F. Allen, Hannah An, Ritwik Bose, William de Beaumont, Choh Man Teng 0001
LREC1
2018 Effective Broad-Coverage Deep Parsing
abstract
Current semantic parsers either compute shallow representations over a wide range of input, or deeper representations in very limited domains. We describe a system that provides broad-coverage, deep semantic parsing designed to work in any domain using a core domain-general lexicon, ontology and grammar. This paper discusses how this core system can be customized for a particularly challenging domain, namely reading research papers in biology. We evaluate these customizations with some ablation experiments
James F. Allen, Omid Bahkshandeh, William de Beaumont, Lucian Galescu, Choh Man Teng 0001
AAAI1
2018 Cogent: A Generic Dialogue System Shell Based on a Collaborative Problem Solving Model
abstract
The bulk of current research in dialogue systems is focused on fairly simple task models, primarily state-based.Progress on developing dialogue systems for more complex tasks has been limited by the lack generic toolkits to build from.In this paper we report on our development from the ground up of a new dialogue model based on collaborative problem solving.We implemented the model in a dialogue system shell (Cogent) that allows developers to plug in problem-solving agents to create dialogue systems in new domains.The Cogent shell has now been used by several independent teams of researchers to develop dialogue systems in different domains, with varied lexicons and interaction style, each with their own problem-solving backend.We believe this to be the first practical demonstration of the feasibility of a CPSbased dialogue system shell.
Lucian Galescu, Choh Man Teng 0001, James F. Allen, Ian Perera
SIGDIAL Conference3
2018 A Situated Dialogue System for Learning Structural Concepts in Blocks World
abstract
We present a modular, end-to-end dialogue system for a situated agent to address a multimodal, natural language dialogue task in which the agent learns complex representations of block structure classes through assertions, demonstrations, and questioning.The concept to learn is provided to the user through a set of positive and negative visual examples, from which the user determines the underlying constraints to be provided to the system in natural language.The system in turn asks questions about demonstrated examples and simulates new examples to check its knowledge and verify the user's description is complete.We find that this task is non-trivial for users and generates natural language that is varied yet understandable by our deep language understanding architecture.
Ian Perera, James F. Allen, Choh Man Teng 0001, Lucian Galescu
SIGDIAL Conference2
2018 A Notion of Semantic Coherence for Underspecified Semantic Representation
abstract
The general problem of finding satisfying solutions to constraint-based underspecified representations of quantifier scope is NP-complete. Existing frameworks, including Dominance Graphs, Minimal Recursion Semantics, and Hole Semantics, have struggled to balance expressivity and tractability in order to cover real natural language sentences with efficient algorithms. We address this trade-off with a general principle of coherence, which requires that every variable introduced in the domain of discourse must contribute to the overall semantics of the sentence. We show that every underspecified representation meeting this criterion can be efficiently processed, and that our set of representations subsumes all previously identified tractable sets.
Mehdi Manshadi, Daniel Gildea, James F. Allen
Comput. Linguistics3
2017 Natural Language Dialogue for Building and Learning Models and Structures
abstract
We demonstrate an integrated system for building and learning models and structures in both a real and virtual environment. The system combines natural language understanding, planning, and methods for composition of basic concepts into more complicated concepts. The user and the system interact via natural language to jointly plan and execute tasks involving building structures, with clarifications and demonstrations to teach the system along the way. We use the same architecture for building and simulating models of biology, demonstrating the general-purpose nature of the system where domain-specific knowledge is concentrated in sub-modules with the basic interaction remaining domain-independent. These capabilities are supported by our work on semantic parsing, which generates knowledge structures to be grounded in a physical representation, and composed with existing knowledge to create a dynamic plan for completing goals. Prior work on learning from natural language demonstrations enables learning of models from very few demonstrations, and features are extracted from definitions in natural language. We believe this architecture for interaction opens up a wide possibility of human-computer interaction and knowledge transfer through natural language.
Ian Perera, James F. Allen, Lucian Galescu, Choh Man Teng 0001, Mark H. Burstein, Scott Friedman 0001, David D. McDonald 0002, Jeffrey M. Rye
AAAI2
2017 Apples to Apples: Learning Semantics of Common Entities Through a Novel Comprehension Task
abstract
Understanding common entities and their attributes is a primary requirement for any system that comprehends natural language.In order to enable learning about common entities, we introduce a novel machine comprehension task, GuessTwo: given a short paragraph comparing different aspects of two realworld semantically-similar entities, a system should guess what those entities are.Accomplishing this task requires deep language understanding which enables inference, connecting each comparison paragraph to different levels of knowledge about world entities and their attributes.So far we have crowdsourced a dataset of more than 14K comparison paragraphs comparing entities from a variety of categories such as fruits and animals.We have designed two schemes for evaluation: open-ended, and binary-choice prediction.For benchmarking further progress in the task, we have collected a set of paragraphs as the test set on which human can accomplish the task with an accuracy of 94.2% on open-ended prediction.We have implemented various models for tackling the task, ranging from semantic-driven to neural models.The semantic-driven approach outperforms the neural models, however, the results indicate that the task is very challenging across the models.
Omid Bakhshandeh, James F. Allen
ACL (1)2
2016 Learning to Jointly Predict Ellipsis and Comparison Structures
abstract
Domain-independent meaning representation of text has received a renewed interest in the NLP community.Comparison plays a crucial role in shaping objective and subjective opinion and measurement in natural language, and is often expressed in complex constructions including ellipsis.In this paper, we introduce a novel framework for jointly capturing the semantic structure of comparison and ellipsis constructions.Our framework models ellipsis and comparison as interconnected predicate-argument structures, which enables automatic ellipsis resolution.We show that a structured prediction model trained on our dataset of 2,800 gold annotated review sentences yields promising results.Together with this paper we release the dataset and an annotation tool which enables two-stage expert annotation on top of tree structures.
Omid Bakhshandeh, Alexis Wellwood, James F. Allen
CoNLL3
2016 A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories
abstract
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, James Allen. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He 0001, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, James F. Allen
HLT-NAACL8
2015 Learning Semantically Rich Event Inference Rules Using Definition of Verbs
Nasrin Mostafazadeh, James F. Allen
CICLing (1)2
2015 Quantity, Contrast, and Convention in Cross-Situated Language Comprehension
abstract
Typically, visually-grounded language learning systems only accept feature data about objects in the environment that are explicitly mentioned, whether through annotation labels or direct reference through natural language.We show that when objects are described ambiguously using natural language, a system can use a combination of the pragmatic principles of Contrast and Conventionality, and multiple-instance learning to learn from ambiguous examples in an online fashion.Applying child language learning strategies to visual learning enables more effective learning in real-time environments, which can lead to enhanced teaching interactions with robots or grounded systems in multi-object environments.
Ian Perera, James F. Allen
CoNLL2
2015 Semantic Framework for Comparison Structures in Natural Language
abstract
Comparison is one of the most important phenomena in language for expressing objective and subjective facts about various entities.Systems that can understand and reason over comparative structure can play a major role in the applications which require deeper understanding of language.In this paper we present a novel semantic framework for representing the meaning of comparative structures in natural language, which models comparisons as predicate-argument pairs interconnected with semantic roles.Our framework supports not only adjectival, but also adverbial, nominal, and verbal comparatives.With this paper, we provide a novel dataset of gold-standard comparison structures annotated according to our semantic framework.
Omid Bakhshandeh, James F. Allen
EMNLP2
2014 What is the Ground? Continuous Maps for Symbol Grounding
Ian Perera, James F. Allen
CogSci2
2013 Integrating Programming by Example and Natural Language Programming
abstract
We motivate the integration of programming by example and natural language programming by developing a system for specifying programs for simple text editing operations based on regular expressions. The programs are described with unconstrained natural language instructions, and providing one or more examples of input/output. We show that natural language allows the system to deduce the correct program much more often and much faster than is possible with the input/output example(s) alone, showing that natural language programming and programming by example can be combined in a way that overcomes the ambiguities that both methods suffer from individually, while providing a more natural interface to the user.
Mehdi Manshadi, Daniel Gildea, James F. Allen
AAAI3
2013 SALL-E: Situated Agent for Language Learning
abstract
We describe ongoing research towards building a cognitively plausible system for near one-shot learning of the meanings of attribute words and object names, by grounding them in a sensory model. The system learns incrementally from human demonstrations recorded with the Microsoft Kinect, in which the demonstrator can use unrestricted natural language descriptions. We achieve near-one shot learning of simple objects and attributes by focusing solely on examples where the learning agent is confident, ignoring the rest of the data. We evaluate the system's learning ability by having it generate descriptions of presented objects, including objects it has never seen before, and comparing the system response against collected human descriptions of the same objects. We propose that our method of retrieving object examples with a k-nearest neighbor classifier using Mahalanobis distance corresponds to a cognitively plausible representation of objects. Our initial results show promise for achieving rapid, near one-shot, incremental learning of word meanings.
Ian Perera, James F. Allen
AAAI2
2013 Plurality, Negation, and Quantification: Towards Comprehensive Quantifier Scope Disambiguation
Mehdi Manshadi, Daniel Gildea, James F. Allen
ACL (1)3
2013 A Markov logic framework for recognizing complex events from multimodal data
abstract
We present a general framework for complex event recognition that is well-suited for integrating information that varies widely in detail and granularity. Consider the scenario of an agent in an instrumented space performing a complex task while describing what he is doing in a natural manner. The system takes in a variety of information, including objects and gestures recognized by RGB-D and descriptions of events extracted from recognized and parsed speech. The system outputs a complete reconstruction of the agent's plan, explaining actions in terms of more complex activities and filling in unobserved but necessary events. We show how to use Markov Logic (a probabilistic extension of first-order logic) to create a model in which observations can be partial, noisy, and refer to future or temporally ambiguous events; complex events are composed from simpler events in a manner that exposes their structure for inference and learning; and uncertainty is handled in a sound probabilistic manner. We demonstrate the effectiveness of the approach for tracking kitchen activities in the presence of noisy and incomplete observations.
Young Chol Song, Henry A. Kautz, James F. Allen, Mary D. Swift, Yuncheng Li, Jiebo Luo 0001, Ce Zhang 0001
ICMI3
2013 Rethinking Logics of Action and Time
abstract
It is over thirty years since I developed interval temporal logic and the accompanying logic of action and time. Overall, these theories have held up well and, with some extensions over the years, have remained useful in our work on AI planning/reasoning systems and natural language understanding. Recently I have become interested in systems that can learn by reading, and specifically, that can learn necessary conditions for event occurrence from reading dictionary definitions. This task adds new constraints on the form of the temporal logic we need. In this talk, I will review our earlier work on temporal logic and then look at the problems that have forced a recent generalization of the formalism in order to allow compositional construction of event definitions from natural language definitions.
James F. Allen
TIME1
2013 Chorus: a crowd-powered conversational assistant
abstract
Despite decades of research attempting to establish conversational interaction between humans and computers, the capabilities of automated conversational systems are still limited. In this paper, we introduce Chorus, a crowd-powered conversational assistant. When using Chorus, end users converse continuously with what appears to be a single conversational partner. Behind the scenes, Chorus leverages multiple crowd workers to propose and vote on responses. A shared memory space helps the dynamic crowd workforce maintain consistency, and a game-theoretic incentive mechanism helps to balance their efforts between proposing and voting. Studies with 12 end users and 100 crowd workers demonstrate that Chorus can provide accurate, topical responses, answering nearly 93% of user queries appropriately, and staying on-topic in over 95% of responses. We also observed that Chorus has advantages over pairing an end user with a single crowd worker and end users completing their own tasks in terms of speed, quality, and breadth of assistance. Chorus demonstrates a new future in which conversational assistants are made usable in the real world by combining human and machine intelligence, and may enable a useful new way of interacting with the crowds powering other systems.
Walter S. Lasecki, Rachel Wesley, Jeffrey Nichols 0001, Anand Kulkarni, James F. Allen, Jeffrey P. Bigham
UIST5
2012 Real-Time Collaborative Planning with the Crowd
abstract
Planning is vital to a wide range of domains, including robotics, military strategy, logistics, itinerary generation and more, that both humans and computers find difficult. Collaborative planning holds the promise of greatly improving performance on these tasks by leveraging the strengths of both humans and automated planners. However, this requires formalizing the problem domain and input, which must be done by hand, a priori, restricting its use in general real-world domains. We propose using a real-time crowd of workers to simultaneously solve the planning problem, formalize the domain, and train an automated system. As plans are developed, the system is able to learn the domain, and contribute larger segments of work.
Walter S. Lasecki, Jeffrey P. Bigham, James F. Allen, George Ferguson
AAAI3
2012 Learning Names for RFID-Tagged Objects in Activity Videos
abstract
We describe a method for determining the names of RFID-tagged objects in activity videos using descriptions which have been parsed to provide anaphoric reference resolution and ontological categorization.
Ian Perera, James F. Allen
AAAI2
2012 An Annotation Scheme for Quantifier Scope Disambiguation
Mehdi Manshadi, James F. Allen, Mary D. Swift
LREC2
2012 Merging Temporal Annotations
abstract
In corpus linguistics obtaining high-quality semantically-annotated corpora is a fundamental goal. Various annotations of the same text can be obtained from automated systems, human annotators, or a combination of both. Obtaining, by manual means, a merged annotation from these, which improves the correctness of each individual annotation, is costly. We present automatic algorithms specifically for merging temporal annotations. These have been evaluated merging the annotations of three state-of-the-art systems on the gold standard corpora and the correctness of the merged annotation improved over that of individual annotations and baseline merging algorithms.
Hector Llorens, Naushad UzZaman, James F. Allen
TIME3
2012 Fruit Carts: A Domain and Corpus for Research in Dialogue Systems and Psycholinguistics
abstract
We describe a novel domain, Fruit Carts, aimed at eliciting human language production for the twin purposes of (a) dialogue system research and development and (b) psycholinguistic research. Fruit Carts contains five tasks: choosing a cart, placing it on a map, painting the cart, rotating the cart, and filling the cart with fruit. Fruit Carts has been used for research in psycholinguistics and in dialogue systems. Based on these experiences, we discuss how well the Fruit Carts domain meets four desired features: unscripted, context-constrained, controllable difficulty, and separability into semi-independent subdialogues. We describe the domain in sufficient detail to allow others to replicate it; researchers interested in using the corpora themselves are encouraged to contact the authors directly.
Gregory Aist, Ellen Campana, James F. Allen, Mary D. Swift, Michael K. Tanenhaus
Comput. Linguistics3
2011 Multimodal summarization of complex sentences
abstract
In this paper, we introduce the idea of automatically illustrating complex sentences as multimodal summaries that combine pictures, structure and simplified compressed text. By including text and structure in addition to pictures, multimodal summaries provide additional clues of what happened, who did it, to whom and how, to people who may have difficulty reading or who are looking to skim quickly. We present ROC-MMS, a system for automatically creating multimodal summaries (MMS) of complex sentences by generating pictures, textual summaries and structure. We show that pictures alone are insufficient to help people understand most sentences, especially for readers who are unfamiliar with the domain. An evaluation of ROC-MMS in the Wikipedia domain illustrates both the promise and challenge of automatically creating multimodal summaries.
Naushad UzZaman, Jeffrey P. Bigham, James F. Allen
IUI3
2011 Natural discourse reference generation reduces cognitive load in spoken systems
abstract
Abstract The generation of referring expressions is a central topic in computational linguistics. Natural referring expressions – both definite references like ‘the baseball cap’ and pronouns like ‘it’ – are dependent on discourse context. We examine the practical implications of context-dependent referring expression generation for the design ofspoken systems. Currently, not all spoken systems have the goal of generating natural referring expressions. Many researchers believe that the context-dependency of natural referring expressions actually makes systemslessusable. Using the dual-task paradigm, we demonstrate that generating natural referring expressions that are dependent on discourse context reduces cognitive load. Somewhat surprisingly, we also demonstrate that practice does not improve cognitive load in systems that generate consistent (context-independent) referring expressions. We discuss practical implications for spoken systems as well as other areas of referring expression generation.
Ellen Campana, Michael K. Tanenhaus, James F. Allen, Roger W. Remington
Nat. Lang. Eng.3
2010 TRIOS-TimeBank Corpus: Extended TimeBank Corpus with Help of Deep Understanding of Text
Naushad UzZaman, James F. Allen
LREC2
2010 Towards a Personal Health Management Assistant
George Ferguson, Jill Quinn, Cecilia Horwitz, Mary D. Swift, James F. Allen, Lucian Galescu
J. Biomed. Informatics5
2008 Production in a Multimodal Corpus: how Speakers Communicate Complex Actions
Carlos Gómez Gallo, T. Florian Jaeger, James F. Allen, Mary D. Swift
LREC3
2008 Linking Semantic and Knowledge Representations in a Multi-Domain Dialogue System
abstract
We describe a two-layer architecture for supporting semantic interpretation and domain reasoning in dialogue systems. Building system that supports both semantic interpretation and domain reasoning in a transparent and well-integrated manner is an unresolved problem because of the diverging requirements of the semantic representations used in contextual interpretation versus the knowledge representations used in domain reasoning. We propose an architecture that provides both portability and efficiency in natural language interpretation by maintaining separate semantic and domain knowledge representations, and integrating them via an ontology mapping procedure. The ontology mapping is used to obtain representations of utterances in a form most suitable for domain reasoners and to automatically specialize the lexicon. The use of a linguistically motivated parser for producing semantic representations for complex natural language sentences facilitates building portable semantic interpretation components as well as connections with domain reasoners. Two evaluations demonstrate the effectiveness of our approach: we show that a small number of mapping rules are sufficient for customizing the generic semantic representation to a new domain, and that our automatic lexicon specialization technique improves parser speed and accuracy.
Myroslava O. Dzikovska, James F. Allen, Mary D. Swift
J. Log. Comput.2
2008 Utilizing Natural Language for One-Shot Task Learning
abstract
Learning tasks from a single demonstration presents a significant challenge because the observed sequence is specific to the current situation and is inherently an incomplete representation of the procedure. Observation-based machine-learning techniques are not effective without multiple examples. However, when a demonstration is accompanied by natural language explanation, the language provides a rich source of information about the relationships between the steps in the procedure and the decision-making processes that led to them. In this article, we present a one-shot task learning system built on TRIPS, a dialogue-based collaborative problem solving system, and show how natural language understanding can be used for effective one-shot task learning.
Hyuckchul Jung, James F. Allen, Lucian Galescu, Nathanael Chambers, Mary D. Swift, William Taysom
J. Log. Comput.2
2007 PLOW: A Collaborative Task Learning Agent
James F. Allen, Nathanael Chambers, George Ferguson, Lucian Galescu, Hyuckchul Jung, Mary D. Swift, William Taysom
AAAI1
2006 Fast Hierarchical Goal Schema Recognition
Nate Blaylock, James F. Allen
AAAI2
2006 Using Semantics to Identify Web Objects
Nathanael Chambers, James F. Allen, Lucian Galescu, Hyuckchul Jung, William Taysom
AAAI2
2006 Software architectures for incremental understanding of human speech
abstract
The prevalent state of the art in spoken language understanding by spoken dialog systems is both modular and whole-utterance. It is modular in that incoming utterances are processed by independent components that handle different aspects, such as acoustics, syntax, semantics, and intention / goal recognition. It is whole-utterance in that each component completes its work for an entire utterance prior to handing off the utterance to the next component. However, a growing body of evidence suggests that humans do not process language that way. Rather, people process speech by rapidly integrating constraints from multiple sources of knowledge and multiple linguistic levels incrementally, as the utterance unfolds. In this paper we describe ongoing work aimed at developing an architecture that will allow machines to understand spoken language in a similar way. This revolutionary approach is promising for two reasons: 1) it more accurately reflects contemporary models of human language understanding, and 2) it results in empirical improvements including increased parsing performance.
Gregory Aist, James F. Allen, Ellen Campana, Lucian Galescu, Carlos Gómez Gallo, Scott C. Stoness, Mary D. Swift, Michael K. Tanenhaus
INTERSPEECH2
2006 Chester: Towards a personal medication advisor
James F. Allen, George Ferguson, Nate Blaylock, Donna K. Byron, Nathanael Chambers, Myroslava O. Dzikovska, Lucian Galescu, Mary D. Swift
J. Biomed. Informatics1
2005 Two Diverse Systems Built using Generic Components for Spoken Dialogue (Recent Progress on TRIPS)
James F. Allen, George Ferguson, Amanda Stent, Scott C. Stoness, Mary D. Swift, Lucian Galescu, Nathanael Chambers, Ellen Campana, Gregory Aist
ACL1
2004 Skeletons in the parser: Using a shallow parser to improve deep parsing
Mary D. Swift, James F. Allen, Daniel Gildea
COLING2
2004 Evaluating cognitive load in spoken language interfaces using a dual-task paradigm
abstract
As speech interfaces become more prevalent, it i s becoming more crucial that they be developed in a way that minimizes cognitive load for users. One major barrier to creating systems that are more human-centered has been the lack of an accepted online methodology for directly evaluating the cognitive resource demands of different systems. The present study extends a classic tool from cognitive psychology, the dual-task paradigm, to speech interface evaluation. Participants follow simple instructions generated by a system, while simultaneously monitoring for a simple visual probe. Performance on the monitoring task is used as a measure of cognitive resource demands; whenever language understanding is more demanding, performance on the monitoring task suffers. In the present study we used this methodology to investigate patterns of reference generation and how they impact human understanding.
Ellen Campana, Michael K. Tanenhaus, James F. Allen, Roger W. Remington
INTERSPEECH3
2004 Semi-automatic Syntactic and Semantic Corpus Annotation with a Deep Parser
Mary D. Swift, Myroslava O. Dzikovska, Joel R. Tetreault, James F. Allen
LREC4
2003 Corpus-based, Statistical Goal Recognition
Nate Blaylock, James F. Allen
IJCAI2
2002 Pronunciation of proper names with a joint n-gram model for bi-directional grapheme-to-phoneme conversion
Lucian Galescu, James F. Allen
INTERSPEECH2
2002 Eye-fixation as a measure of real-time processing of synthesized words
abstract
We present experimental evidence from a study in which we monitor eye movements as people respond to pre-recorded instructions generated by a human speaker and by two text-tospeech synthesizers. We replicate findings demonstrating that people process human speech incrementally, making partial commitments as a word unfolds. Specifically, they entertain multiple lexical candidates on the fly depending on segmental overlap in the candidate set. Importantly, incremental understanding is also observed for synthesized text-to-speech instructions. These results, including some suggestive differences in responses with the two text-to-speech systems, establish the potential for using eye-tracking methodology together with synthesized speech stimuli as a powerful theoretical and experimental tool for spoken language processing research. 1. BACKGROUND Spoken utterances unfold over time, and psycholinguistic studies of natural speech have shown that the time course of spoken language results in a stream of temporary ambiguities at
Mary D. Swift, Ellen Campana, James F. Allen, Michael K. Tanenhaus
INTERSPEECH3
2001 An architecture for more realistic conversational systems
abstract
In this paper, we describe an architecture for conversational systems that enables human-like performance along several important dimensions. First, interpretation is incremental, multi-level, and involves both general and task- and domain-specific knowledge. Second, generation is also incremental, proceeds in parallel with interpretation, and accounts for phenomena such as turn-taking, grounding and interruptions. Finally, the overall behavior of the system in the task at hand is determined by the (incremental) results of interpretation, the persistent goals and obligations of the system, and exogenous events of which it becomes aware. As a practical matter, the architecture supports a separation of responsibilities that enhances portability to new tasks and domains.
James F. Allen, George Ferguson, Amanda Stent
IUI1
2000 Hierarchical statistical language models: experiments on in-domain adaptation
abstract
We introduce a hierarchical statistical language model, represented as a collection of local models plus a general sentence model. We provide an example that mixes a trigram general model and a PFSA local model for the class of decimal numbers, described in terms of sub-word units (graphemes). This model practically extends the vocabulary of the overall model to an infinite size, but still has better performance compared to a word-based model. Using in-domain language model adaptation experiments, we show that local models can encode enough linguistic information, if well trained, that they may be ported to new language models without re-estimation.
Lucian Galescu, James F. Allen
INTERSPEECH2
2000 Evaluating hierarchical hybrid statistical language models
abstract
We introduce in this paper a hierarchical hybrid statistical language model, represented as a collection of local models plus a general model that binds together the local ones. The model provides a unified framework for modelling language both above and below the word level, and we exemplify with models of both kinds for a large vocabulary task domain. To our knowledge this is the first paper to report an extensive evaluation of the improvements achieved from the use of local models within a hierarchical framework in comparison with a conventional word-based trigram model.
Lucian Galescu, James F. Allen
INTERSPEECH2
2000 An architecture for a generic dialogue shell
abstract
This paper describes our work on dialogue systems that can mimic human conversation, with the goal of providing intuitive access to a wide range of applications by expanding the user's options in the interaction. We concentrate on practical dialogue: dialogues in which the participants need to accomplish some objective or perform some task. Two hypotheses regarding practical dialogue motivate our research. First, that the conversational competence required for practical dialogues, while still complex, is significantly simpler to achieve than general human conversational competence. And second, that within the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management is independent of the task being performed. If these hypotheses are true, then it should be possible to build a generic dialogue shell for practical dialogue, by which we mean the full range of components required in a dialogue system, including speech recognition, language processing, dialogue management and response planning, built in such a way as to be readily adapted to new applications by specifying the domain and task models. This paper documents our progress and what we have learned so far based on building and adapting systems in a series of different problem solving domains.
James F. Allen, Donna K. Byron, Myroslava O. Dzikovska, George Ferguson, Lucian Galescu, Amanda Stent
Nat. Lang. Eng.1
1999 Speech Repairs, Intonational Phrases and Discourse Markers: Modeling Speakers' Utterances in Spoken Dialog
Peter A. Heeman, James F. Allen
Comput. Linguistics2
1998 Rapid language model development for new task domains
Lucian Galescu, Eric K. Ringger, James F. Allen
LREC3
1997 Intonational Boundaries, Speech Repairs and Discourse Markers: Modeling Spoken Dialog
abstract
To understand a speaker's turn of a conversation, one needs to segment it into intonational phrases, clean up any speech repairs that might have occurred, and identify discourse markers. In this paper, we argue that these problems must be resolved together, and that they must be resolved early in the processing stream. We put forward a statistical language model that resolves these problem, does POS tagging, and can be used as the language model of a speech recognizer. We find that by accounting for the interactions between these tasks that the performance on each task improves, as does POS tagging and perplexity.
Peter A. Heeman, James F. Allen
ACL2
1997 Incorporating POS tagging into language modeling
abstract
Language models for speech recognition tend to concentrate solely on recognizing the words that were spoken.In this paper, we redefine the speech recognition problem so that its goal is to find both the best sequence of words and their syntactic role (part-of-speech) in the utterance.This is a necessary first step towards tightening the interaction between speech recognition and natural language understanding.
Peter A. Heeman, James F. Allen
EUROSPEECH2
1996 A Robust System for Natural Spoken Dialogue
abstract
This paper describes a system that leads us to believe in the feasibility of constructing natural spoken dialogue systems in task-oriented domains. It specifically addresses the issue of robust interpretation of speech in the presence of recognition errors. Robustness is achieved by a combination of statistical error post-correction, syntactically- and semantically-driven robust parsing, and extensive use of the dialogue context. We present an evaluation of the system using time-to-completion and the quality of the final solution that suggests that most native speakers of English can use the system successfully with virtually no training.
James F. Allen, Bradford W. Miller, Eric K. Ringger, Teresa Sikorski
ACL1
1996 Error correction via a post-processor for continuous speech recognition
abstract
This paper presents a new technique for overcoming several types of speech recognition errors by post-processing the output of a continuous speech recognizer. The post-processor output contains fewer errors, thereby making interpretation by higher-level modules, such as a parser, in a speech understanding system more reliable. The primary advantage to the post-processing approach over existing approaches for overcoming SR errors lies in its ability to introduce options that are not available in the SR module's output. This work provides evidence for the claim that a modern continuous speech recognizer can be used successfully in "black-box" fashion for robustly interpreting spontaneous utterances in a dialogue with a human.
Eric K. Ringger, James F. Allen
ICASSP2
1996 Combining the detection and correction of speech repairs
Peter A. Heeman, Kyung-ho Loken-Kim, James F. Allen
ICSLP3
1996 A fertility channel model for post-correction of continuous speech recognition
abstract
We have implemented a post-processor called SPEECHPP to correct word-level error^ committed by an arbitrary speech recognizer.A p plying a noisychannelmodel, SPEECHPPuses a Viterbi beam-search that employs language and channel models.Previous work demonstrated that a simple word-for-word channel model was sufficient to yield substantial incieases in word accuracy.This paper demonstrates that some improvements in word accuracy result from augmenting the channel model with an account of word fertility in the channel.This work further demonstrates that a modern continuous speech recognizer can be used in "black-box" fashion for robustly recognizing speech for which the recognizer was not originally trained.This work also demonstrates that in the case when the recognizercan be tuned to the new task, environment, or spealcer, the post-processor can also contribute to performance improvements.
Eric K. Ringger, James F. Allen
ICSLP2
1995 The TRAINS project: a case study in building a conversational planning agent
abstract
The TRAINS project is an effort to build a conversationally proficient planning assistant. A key part of the project is the construction of the TRAINS system, which provides the research platform for a wide range of issues in natural language understanding, mixed-initiative planning systems, and representing and reasoning about time, actions and events. Four years have now passed since the beginning of the project. Each year a demonstration system has been produced that focused on a dialogue that illustrates particular aspects of the research. The commitment to building complete integrated systems is a significant overhead on the research, but it is considered essential to guarantee that the results constitute real progress in the field. This paper describes the goals of the project, and the experience with the effort so far.
James F. Allen, Lenhart K. Schubert, George Ferguson, Peter A. Heeman, Chung Hee Hwang, Tsuneaki Kato, Marc Light, Nathaniel G. Martin, Bradford W. Miller, Massimo Poesio, David R. Traum
J. Exp. Theor. Artif. Intell.1
1994 Deyecting and Correcting Speech Repairs
abstract
Interactive spoken dialog provides many new challenges for spoken language systems.One of the most critical is the prevalence of speech repairs.This paper presents an algorithm that detects and corrects speech repairs based on finding the repair pattern.The repair pattern is built by finding word matches and word replacements, and identifying fragments and editing terms.Rather than using a set of prebuilt templates, we build the pattern on the fly.In a fair test, our method, when combined with a statistical model to filter possible repairs, was successful at detecting and correcting 80% of the repairs, without using prosodic information or a parser.
Peter A. Heeman, James F. Allen
ACL2
1994 Discourse Obligations in Dialogue Processing
abstract
We show that in modeling social interaction, particularly dialogue, the attitude of obligation can be a useful adjunct to the popularly considered attitudes of belief, goal, and intention and their mutual and shared counterparts. In particular, we show how discourse obligations can be used to account in a natural manner for the connection between a question and its answer in dialogue and how obligations can be used along with other parts of the discourse context to extend the coverage of a dialogue system.
David R. Traum, James F. Allen
ACL2
1994 Optimal and Heuristic Task Scheduling Under Qualitative Temporal Constraints
Frank D. Anger, James F. Allen, Rita V. Rodríguez
IEA/AIE2
1994 Actions and Events in Interval Temporal Logic
abstract
We present a representation of events and action based on interval temporal logic that is significantly more expressive and more natural than most previous AI approaches. The representation is motivated by work in natural language semantics and discourse, temporal logic, and AI planning and plan recognition. The formal basis of the representation is presented in detail, from the axiomatization of time periods to the relationship between actions and events and their effects. The power of the representation is illustrated by applying it to the axiomatization and solution of several standard problems from the AI literature on action and change. An approach to the frame problem based on explanation closure is shown to be both powerful and natural when combined with our representational framework. We also discuss features of the logic that are beyond the scope of many traditional representations, and describe our approach to difficult problems such as external event! and simultaneous actions.
James F. Allen, George Ferguson
J. Log. Comput.1
1992 Prosody as a cue for discourse structure
Shinya Nakajima, James F. Allen
ICSLP2
1992 A "speech acts" approach to grounding in conversation
David R. Traum, James F. Allen
ICSLP2
1991 Planning as Temporal Reasoning
James F. Allen
KR1
1991 A Language for Planning with Statistics
Nathaniel G. Martin, James F. Allen
UAI2
1991 Time and time again: The many ways to represent time
abstract
One of the most crucial problems in any computer system that involves representing the world is the representation of time. This includes applications such as databases, simulation, expert systems, and applications of Artificial Intelligence in general. In this brief article, I will give a survey of the basic techniques available for representing time, and then talk about temporal reasoning in a general setting as needed in AI applications. Quite different representations of time are usable depending on the assumptions that can be made about the temporal information to be represented. the most crucial issue is the degree of certainty one can assume. Can one assume that a timestamp can be assigned to each event, or barring that, that the events are fully ordered? Or can we only assume that a partial ordering of events is known? Can events be simultaneous? Can they overlap in time and yet not be simultaneous? If they are not instaneous, do we know the durations of events? Different answers to each of these questions allow very different representations of time.
James F. Allen
Int. J. Intell. Syst.1
1989 Two Constraints on Speech Act Ambiguity
abstract
Existing plan-based theories of speech act interpretation do not account for the conventional aspect of speech acts. We use patterns of linguistic features (e.g. mood, verb form, sentence adverbials, thematic roles) to suggest a range of speech act interpretations for the utterance. These are filtered using plan-based conversational implicatures to eliminate inappropriate ones. Extended plan reasoning is available but not necessary for familiar forms. Taking speech act ambiguity seriously, with these two constraints, explains how "Can you pass the salt?" is a typical indirect request while "Are you able to pass the salt?" is not.
Elizabeth A. Hinkelman, James F. Allen
ACL2
1989 Moments, points in an interval-based temporal logic
abstract
Abstract This paper develops and explores a first‐order theory of time that appears useful as an underlying framework for a wide range of practical applications in artificial intelligence (AI). In particular, it presents a concise, formal axiomatization of “interval‐based” time as described by Allen and then explores the relationship between interval‐based and point‐based temporal theories in detail. This analysis should be useful to both theoretical and practical work in AI that involves the representation of time, since it shows what distinctions are actually substantive and what arise merely from formalisms being notational variants of one another.
James F. Allen, Patrick J. Hayes
Comput. Intell.1
1987 A Model for Concurrent Actions Having Temporal Extent
Richard N. Pelavin, James F. Allen
AAAI2
1987 Short Time Periods
Patrick J. Hayes, James F. Allen
IJCAI2
1986 Panel: Real-Time Performance in Problem Solving
Michael R. Fehling, Malcolm Acock, James F. Allen, Michael P. Georgeff, Victor R. Lesser, Robert C. Moore
AAAI3
1986 Generalized Plan Recognition
Henry A. Kautz, James F. Allen
AAAI2
1986 Plans, goals, and language
abstract
One of the most promising computational approaches to representing context in natural language systems has been based on work in general problem solving. In this approach, plans are used both to represent the domain of discourse as well as the communication process itself. Using a simplified framework for planning and action reasoning, we describe techniques that allow systems to handle many dialogues that are problematic for other systems, including the use of sentence fragments, indirect speech, helpful responses, the tracking of the topic of conversations both with and without interrupting subdialogues, and topic change.
James F. Allen, Diane J. Litman
Proc. IEEE1
1986 A formal logic of plans in temporally rich domains
abstract
This paper outlines a temporal logic extended with two modalities that can be used to support planning in temporally rich domains. In particular, the logic can represent planning environments that have assertions about future possibilities in addition to the present state, and plans that contain concurrent actions. The logic is particularly expressive in the ways that concurrent actions can interact with each other and allows situations where either one of the actions can be executed, but both cannot, as well as situations where neither action can be executed alone, but they can be done together. Two modalities are introduced and given a formal semantics: INEV expresses simple temporal possibility, and IFTRIED expresses counteffactual-like statements about actions.
Richard N. Pelavin, James F. Allen
Proc. IEEE2
1985 A Common-Sense Theory of Time
James F. Allen, Patrick J. Hayes
IJCAI1
1984 A Plan Recognition Model for Clarification Subdialogues
abstract
One of the promising approaches to analyzing task-oriented dialogues has involved modeling the plans of the speakers in the task domain. In general, these models work well as long as the topic follows the task structure closely, but they have difficulty in accounting for clarification subdialogues and topic change. We have developed a model based on a hierarchy of plans and metaplans that accounts for the clarification subdialogues while maintaining the advantages of the plan-based approach.
Diane J. Litman, James F. Allen
COLING2
1984 Towards a General Theory of Action and Time
James F. Allen
Artif. Intell.1
1983 Planning Using a Temporal World Model
James F. Allen, Johannes A. G. M. Koomen
IJCAI1
1982 ARGOT: The Rochester Dialogue System
James F. Allen, Alan M. Frisch, Diane J. Litman
AAAI1
1982 What's in a Semantic Network?
abstract
Ever since Wood's "What's in a Link" paper, there has been a growing concern for formalization in the study of knowledge representation. Several arguments have been made that frame representation languages and semantic-network languages are syntactic variants of the first-order predicate calculus (FOPC). The typical argument proceeds by showing how any given frame or network representation can be mapped to a logically isomorphic FOPC representation. For the past two years we have been studying the formalization of knowledge retrievers as well as the representation languages that they operate on. This paper presents a representation language in the notation of FOPC whose form facilitates the design of a semantic-network-like retriever.
James F. Allen, Alan M. Frisch
ACL1
1982 Knowledge Retrieval as Limited Inference
Alan M. Frisch, James F. Allen
CADE2
1981 What's Necessary to Hide?: Modeling Action Verbs
abstract
This paper considers what types of knowledge one must possess in order to reason about actions. Rather than concentrating on how actions are performed, as is done in the problem-solving literature, it examines the set of conditions under which an action can be said to have occurred. In other words, if one is told that action A occurred, what can be inferred about the state of the world? In particular, if the representation can define such conditions, it must have good models of time, belief, and intention. This paper discusses these issues and suggests a formalism in which general actions and events can be defined. Throughout, the action of hiding a book from someone is used as a motivating example.
James F. Allen
ACL1
1981 An Interval-Based Representation of Temporal Knowledge
James F. Allen
IJCAI1
1980 Analyzing Intention in Utterances
James F. Allen, C. Raymond Perrault
Artif. Intell.1
1980 A Plan-Based Analysis of Indirect Speech Acts
C. Raymond Perrault, James F. Allen
Am. J. Comput. Linguistics2
1979 Plans, Inference, and Indirect Speech Acts
abstract
Americanae nace como un proyecto conjunto que surge dentro de la Red Europea de Información y Documentación sobre América Latina (REDIAL), y que ha afrontado la Biblioteca de la Agencia Española de Cooperación Internacional para el Desarrollo (AECID). Esta nueva biblioteca virtual hace más accesibles los libros digitales de tema americanista a los investigadores y usuarios interesados de cualquier parte del mundo.
James F. Allen, C. Raymond Perrault
ACL1
1975 A Speech Understanding System Based Upon A Co-Routine Parser
James F. Allen
IJCAI1