Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms-only within a given query or document. Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists. To address the issue of high DFs, we present a new variant of FLOPS regularization: DF-FLOPS. This new regularization technique penalizes the usage of high-DF terms, thereby shortening posting lists and reducing retrieval latency. Unlike other inference-time sparsification methods, such as stopword removal, DF-FLOPS regularization allows for the selective inclusion of high-frequency terms in cases where the terms are truly salient. We find that DF-FLOPS successfully reduces the prevalence of high-DF terms and lowers retrieval latency (around 10x faster) in a production-grade engine while maintaining effectiveness both in-domain (only a 2.2-point drop in MRR@10) and cross-domain (improved performance in 12 out of 13 tasks on which we tested). With retrieval latencies on par with BM25, this work provides an important step towards making LSR practical for deployment in production-grade search engines.

Part of speech (POS) tagging is a familiar NLP task.State of the art taggers routinely achieve token-level accuracies of over 97% on news body text, evidence that the problem is wellunderstood.However, the register of English news headlines, "headlinese", is very different from the register of long-form text, causing POS tagging models to underperform on headlines.In this work, we automatically annotate news headlines with POS tags by projecting predicted tags from corresponding sentences in news bodies.We train a multi-domain POS tagger on both long-form and headline text and show that joint training on both registers improves over training on just one or naïvely concatenating training sets.We evaluate on a newly-annotated corpus of over 5,248 English news headlines from the Google sentence compression corpus, and show that our model yields a 23% relative error reduction per token and 19% per headline.In addition, we demonstrate that better headline POS tags can improve the performance of a syntax-based open information extraction system.We make POSH, the POS-tagged Headline corpus, available to encourage research in improved NLP models for news headlines.

Tianze Shi, Adrian Benton, Igor Malioutov, Ozan İrsoy. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

Tianze Shi, Ozan İrsoy, Igor Malioutov, Lillian Lee. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.

We reduce the task of (span-based) PropBankstyle semantic role labeling (SRL) to syntactic dependency parsing.Our approach is motivated by our empirical analysis that shows three common syntactic patterns account for over 98% of the SRL annotations for both English and Chinese data.Based on this observation, we present a conversion scheme that packs SRL annotations into dependency tree representations through joint labels that permit highly accurate recovery back to the original format.This representation allows us to train statistical dependency parsers to tackle SRL and achieve competitive performance with the current state of the art.Our findings show the promise of syntactic dependency trees in encoding semantic role relations within their syntactic domain of locality, and point to potential further integration of syntactic methods into semantic role labeling in the future.

Statistically-based parsers for large corpora, in particular the Penn Tree Bank (PTB), typically have not used all the linguistic information encoded in the annotated trees on which they are trained. In particular, they have not in general used information that records the effects of derivations, such as empty categories and the representation of displaced phrases, as is the case with passive, topicalization, and wh-constructions. Here we explore ways to use this information to “unwind” derivations, yielding a regularized underlying syntactic structure that can be used as an additional source of information for more accurate parsing. In effect, we make use of two joint sets of tree structures for parsing: the surface structure and its corresponding underlying structure where arguments have been restored to their canonical positions. We present a pilot experiment on passives in the PTB indicating that through the use of these two syntactic representations we can improve overall parsing performance by exploiting transformational regularities, in this way paring down the search space of possible syntactic analyses.

In this paper we discuss our research activities in the area of spoken lecture processing. Our goal is to improve the access to on-line audio/visual recordings of academic lectures by developing tools for the processing, transcription, indexing, segmentation, summarization, retrieval and browsing of this media. In this paper, we provide an overview of the technology components and systems that have been developed as part of this project, present some experimental results, and discuss our ongoing and future research plans. Index Terms:spoken lecture processing, spoken document retrieval, audio browsing

We consider the task of unsupervised lecture segmentation. We formalize segmentation as a graph-partitioning task that optimizes the normalized cut criterion. Our approach moves beyond localized comparisons and takes into account long-range cohesion dependencies. Our results demonstrate that global analysis improves the segmentation accuracy and is robust in the presence of speech recognition errors.

Geant4 is a very large, highly accurate toolkit for Monte Carlo simulation of particle-matter interaction. It has been applied to high-energy physics, cosmic ray modeling, radiation shields, radiation therapy, mine detection, and other areas. Geant4 is being used to help design some high energy physics experiments (notably CMS and Atlas) to be run on the future large hadron collider: the largest particle collider in the world. The parallelization, ParGeant4, represents a challenge due to the unique characteristics of Geant4: (i) complex object-oriented design; (ii) intrinsic use of templates and abstract classes to be instantiated later by the end user; (iii) large program with many developers; and (iv) frequent releases. The key issue for parallelization is not just how to parallelize "correctly" but also how to parallelize "with minimum effort". In addition, the parallelization should make as few assumptions about the source code as possible, due to the frequent release schedule of Geant4. We use TOP-C (Task Oriented Parallel C/C++) for parallelization and Marshalgen for marshaling/ serialization. In some examples on a cluster of 100 nodes yielded a speedup of up to 94.4. The code’s portability, scalability and performance are also discussed.