Alexandru Coca, Mark Gaynor, Zhenxing Zhang, Jianpeng Cheng, Bo-Hsiang Tseng, Peter Boothroyd, Hector Martinez Alonso, Diarmuid O Seaghdha, Anders Johannsen. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Programmable task-oriented dialogue (TOD) agents enable language models to follow structured dialogue policies, but their effectiveness hinges on accurate dialogue state tracking (DST). We present PyTOD, an agent that generates executable code to track dialogue state and uses policy and execution feedback for efficient error correction. To achieve this, PyTOD employs a simple constrained decoding approach, using a language model instead of grammar rules to follow API schemata. This leads to state-of-the-art DST performance on the challenging SGD benchmark. Our experiments show that PyTOD surpasses strong baselines in both accuracy and cross-turn consistency, demonstrating the effectiveness of execution-aware state tracking.

Jianpeng Cheng, Devang Agrawal, Héctor Martínez Alonso, Shruti Bhargava, Joris Driesen, Federico Flego, Dain Kaplan, Dimitri Kartsaklis, Lin Li, Dhivya Piraviperumal, Jason D. Williams, Hong Yu, Diarmuid Ó Séaghdha, Anders Johannsen. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

Westlaw Edge, a new legal research platform from Westlaw, was launched in 2018. The three AI-enabled features that launched with Westlaw Edge are KeyCite Overruling Risk, Litigation Analytics, and WestSearch Plus. Keycite Overruling Risk uses NLP and machine learning to warn users when a point of law in a case may have been implicitly undermined based on a prior decision, when that prior citation has no direct citation relationship to the at-risk case. Litigation Analytics allows users to access valuable metadata extracted from legal dockets about the legal actions carried out by parties, lawyers, law firms, and judges presiding over cases. WestSearch Plus is a non-factoid Question Answering system that provides legally correct, jurisdictionally relevant, and conversationally responsive answers to user-entered questions in the legal domain. Added AI-enabled features are set to launch on the Westlaw Edge platform in 2019. It is available for demo at ICAIL 2019.

Multitask learning has been applied successfully to a range of tasks, mostly morphosyntactic.However, little is known on when MTL works and whether there are data characteristics that help to determine its success.In this paper we evaluate a range of semantic sequence labeling tasks in a MTL setup.We examine different auxiliary tasks, amongst which a novel setup, and correlate their impact to datadependent conditions.Our results show that MTL is not always effective, significant improvements are obtained only for 1 out of 5 tasks.When successful, auxiliary tasks with compact and more uniform label distributions are preferable.

Héctor Martínez Alonso, Željko Agić, Barbara Plank, Anders Søgaard. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017.

In this article we present an expansion of the supersense inventory.All new supersenses are extensions of members of the current inventory, which we postulate by identifying semantically coherent groups of synsets.We cover the expansion of the already-established supernsense inventory for nouns and verbs, the addition of coarse supersenses for adjectives in absence of a canonical supersense inventory, and supersenses for verbal satellites.We evaluate the viability of the new senses examining the annotation agreement, frequency and co-ocurrence patterns.

We propose a novel approach to cross-lingual part-of-speech tagging and dependency parsing for truly low-resource languages. Our annotation projection-based approach yields tagging and parsing models for over 100 languages. All that is needed are freely available parallel texts, and taggers and parsers for resource-rich languages. The empirical evaluation across 30 test languages shows that our method consistently provides top-level accuracies, close to established upper bounds, and outperforms several competitive baselines.

Knowledge bases have the potential to advance artificial intelligence, but often suffer from recall problems, i.e., lack of knowledge of new entities and relations. On the contrary, social media such as Twitter provide abundance of data, in a timely manner: information spreads at an incredible pace and is posted long before it makes it into more commonly used resources for knowledge extraction. In this paper we address the question whether we can exploit social media to extract new facts, which may at first seem like finding needles in haystacks. We collect tweets about 60 entities in Freebase and compare four methods to extract binary relation candidates, based on syntactic and semantic parsing and simple mechanism for factuality scoring. The extracted facts are manually evaluated in terms of their correctness and relevance for search. We show that moving from bottom-up syntactic or semantic dependency parsing formalisms to top-down frame-semantic processing improves the robustness of knowledge extraction, producing more intelligible fact candidates of better quality. In order to evaluate the quality of frame semantic parsing on Twitter intrinsically, we make a multiply frame-annotated dataset of tweets publicly available.

Anders Søgaard, Željko Agić, Héctor Martínez Alonso, Barbara Plank, Bernd Bohnet, Anders Johannsen. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2015.

Using automatic measures such as labeled and unlabeled attachment scores is common practice in dependency parser evaluation.In this paper, we examine whether these measures correlate with human judgments of overall parse quality.We ask linguists with experience in dependency annotation to judge system outputs.We measure the correlation between their judgments and a range of parse evaluation metrics across five languages.The humanmetric correlation is lower for dependency parsing than for other NLP tasks.Also, inter-annotator agreement is sometimes higher than the agreement between judgments and metrics, indicating that the standard metrics fail to capture certain aspects of parse quality, such as the relevance of root attachment or the relative importance of the different parts of speech.

We present a multilingual corpus of Wikipedia and Twitter texts annotated with FRAMENET 1.5 semantic frames in nine different languages, as well as a novel technique for weakly supervised cross-lingual frame-semantic parsing. Our approach only assumes the existence of linked, comparable source and target lan-guage corpora (e.g., Wikipedia) and a bilingual dictionary (e.g., Wiktionary or BABELNET). Our approach uses a truly interlingual representation, enabling us to use the same model across all nine lan-guages. We present average error reduc-tions over running a state-of-the-art parser on word-to-word translations of 46 % for target identification, 37 % for frame identi-fication, and 14 % for argument identifica-tion. 1

Héctor Martínez Alonso, Barbara Plank, Arne Skjærholt, Anders Søgaard. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015.

Dirk Hovy, Barbara Plank, Héctor Martínez Alonso, Anders Søgaard. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2015.

In NLP, we need to document that our pro-posed methods perform significantly bet-ter with respect to standard metrics than previous approaches, typically by re-porting p-values obtained by rank- or randomization-based tests. We show that significance results following current re-search standards are unreliable and, in ad-dition, very sensitive to sample size, co-variates such as sentence length, as well as to the existence of multiple metrics. We estimate that under the assumption of per-fect metrics and unbiased data, we need a significance cut-off at ⇠0.0025 to reduce the risk of false positive results to <5%. Since in practice we often have consider-able selection bias and poor metrics, this, however, will not do alone. 1

Often the bottleneck in document classification is finding good representations that zoom in on the most important aspects of the documents.Most research uses n-gram representations, but relevant features often occur discontinuously, e.g., not. . .good in sentiment analysis.In this paper we present experiments getting experts to provide regular expressions, as well as crowdsourced annotation tasks from which regular expressions can be derived.Somewhat surprisingly, it turns out that these crowdsourced feature combinations outperform automatic feature combination methods, as well as expert features, by a very large margin and reduce error by 24-41% over n-gram representations.