In information retrieval (IR), candidate set pruning has been commonly used to speed up twostage relevance ranking.However, such an approach lacks accurate error control and often trades accuracy against computational efficiency in an empirical fashion, missing theoretical guarantees.In this paper, we propose the concept of certified error control of candidate set pruning for relevance ranking, which means that the test error after pruning is guaranteed to be controlled under a user-specified threshold with high probability.Both in-domain and outof-domain experiments show that our method successfully prunes the first-stage retrieved candidate sets to improve the second-stage reranking speed while satisfying the pre-specified accuracy constraints in both settings.For example, on MS MARCO Passage v1, our method reduces the average candidate set size from 1000 to 27, increasing reranking speed by about 37 times, while keeping MRR@10 greater than a pre-specified value of 0.38 with about 90% empirical coverage.In contrast, empirical baselines fail to meet such requirements.

Limited-vocabulary speech commands recognition is the task of classifying a short utterance as one of several speech commands, for which neural networks obtain state-of-the-art results. In particular, recurrent neural networks represent a common approach for streaming commands recognition systems. In this paper, we explore resource-efficient methods to short-circuit such systems in the time domain when the model is confident in its prediction. We propose applying a frame-level labeling objective to further improve the efficiency–accuracy trade-off. On two datasets in limited-vocabulary commands recognition, our best method achieves an average time savings of 45% of the utterance without reducing the absolute accuracy by more than 0.6 points. We show that the per-instance savings depend on the length of the unique prefix in the phonemes across a dataset.

Most real-world problems that machine learning algorithms are expected to solve face the situation with (1) unknown data distribution; (2) little domain-specific knowledge; and (3) datasets with limited annotation. We propose Non-Parametric learning by Compression with Latent Variables (NPC-LV), a learning framework for any dataset with abundant unlabeled data but very few labeled ones. By only training a generative model in an unsupervised way, the framework utilizes the data distribution to build a compressor. Using a compressor-based distance metric derived from Kolmogorov complexity, together with few labeled data, NPC-LV classifies without further training. We show that NPC-LV outperforms supervised methods on all three datasets on image classification in the low data regime and even outperforms semi-supervised learning methods on CIFAR-10. We demonstrate how and when negative evidence lowerbound (nELBO) can be used as an approximate compressed length for classification. By revealing the correlation between compression rate and classification accuracy, we illustrate that under NPC-LV how the improvement of generative models can enhance downstream classification accuracy.

Ji Xin, Raphael Tang, Yaoliang Yu, Jimmy Lin. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.

The slow speed of BERT has motivated much research on accelerating its inference, and the early exiting idea has been proposed to make trade-offs between model quality and efficiency.This paper aims to address two weaknesses of previous work: (1) existing fine-tuning strategies for early exiting models fail to take full advantage of BERT; (2) methods to make exiting decisions are limited to classification tasks.We propose a more advanced fine-tuning strategy and a learning-toexit module that extends early exiting to tasks other than classification.Experiments demonstrate improved early exiting for BERT, with better trade-offs obtained by the proposed finetuning strategy, successful application to regression tasks, and the possibility to combine it with other acceleration methods.

Recent work has shown that dense passage retrieval techniques achieve better ranking accuracy in open-domain question answering compared to sparse retrieval techniques such as BM25, but at the cost of large space and memory requirements.In this paper, we analyze the redundancy present in encoded dense vectors and show that the default dimension of 768 is unnecessarily large.To improve space efficiency, we propose a simple unsupervised compression pipeline that consists of principal component analysis (PCA), product quantization, and hybrid search.We further investigate other supervised baselines and find surprisingly that unsupervised PCA outperforms them in some settings.We perform extensive experiments on five question answering datasets and demonstrate that our best pipeline achieves good accuracy-space trade-offs, for example, 48× compression with less than 3% drop in top-100 retrieval accuracy on average or 96× compression with less than 4% drop.Code and data are available at http: //pyserini.io/.

Query auto completion (QAC) is the task of predicting a search engine user’s final query from their intermediate, incomplete query. In this paper, we extend QAC to the streaming voice search setting, where automatic speech recognition systems produce intermediate transcriptions as users speak. Naively applying existing methods fails because the intermediate transcriptions often don’t form prefixes or even substrings of the final transcription. To address this issue, we propose to condition QAC approaches on intermediate transcriptions to complete voice queries. We evaluate our models on a speech-enabled smart television with real-life voice search traffic, finding that this ASR-aware conditioning improves the completion quality. Our best method obtains an 18% relative improvement in mean reciprocal rank over previous methods.

In natural language processing, a recently popular line of work explores how to best report the experimental results of neural networks.One exemplar publication, titled "Show Your Work: Improved Reporting of Experimental Results" (Dodge et al., 2019), advocates for reporting the expected validation effectiveness of the best-tuned model, with respect to the computational budget.In the present work, we critically examine this paper.As far as statistical generalizability is concerned, we find unspoken pitfalls and caveats with this approach.We analytically show that their estimator is biased and uses error-prone assumptions.We find that the estimator favors negative errors and yields poor bootstrapped confidence intervals.We derive an unbiased alternative and bolster our claims with empirical evidence from statistical simulation.Our codebase is at https://github.com/ castorini/meanmax.

Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications.However, they are also notorious for being slow in inference, which makes them difficult to deploy in realtime applications.We propose a simple but effective method, DeeBERT, to accelerate BERT inference.Our approach allows samples to exit earlier without passing through the entire model.Experiments show that DeeBERT is able to save up to ∼40% inference time with minimal degradation in model quality.Further analyses show different behaviors in the BERT transformer layers and also reveal their redundancy.Our work provides new ideas to efficiently apply deep transformer-based models to downstream tasks.

Ji Xin, Jimmy Lin, Yaoliang Yu. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.

Fine-grained entity typing aims to identify the semantic type of an entity in a particular plain text. It is an important task which can be helpful for a lot of natural language processing (NLP) applications. Most existing methods typically extract features separately from the entity mention and context words for type classification. These methods inevitably fail to model complex correlations between entity mentions and context words. They also neglect rich background information about these entities in knowledge bases (KBs). To address these issues, we take information from KBs into consideration to bridge entity mentions and their context together, and thereby propose Knowledge-Attention Neural Fine-Grained Entity Typing. Experimental results and case studies on real-world datasets demonstrate that our model significantly outperforms other state-of-the-art methods, revealing the effectiveness of incorporating KB information for entity typing. Code and data for this paper can be found at https://github.com/thunlp/KNET.

Entity typing aims to classify semantic types of an entity mention in a specific context.Most existing models obtain training data using distant supervision, and inevitably suffer from the problem of noisy labels.To address this issue, we propose entity typing with language model enhancement.It utilizes a language model to measure the compatibility between context sentences and labels, and thereby automatically focuses more on context-dependent labels.Experiments on benchmark datasets demonstrate that our method is capable of enhancing the entity typing model with information from the language model, and significantly outperforms the stateof-the-art baseline.