In a spoken dialogue system, an NLU model is preceded by a speech recognition system that can deteriorate the performance of natural language understanding.This paper proposes a method for investigating the impact of speech recognition errors on the performance of natural language understanding models.The proposed method combines the back transcription procedure with a fine-grained technique for categorizing the errors that affect the performance of NLU models.The method relies on the usage of synthesized speech for NLU evaluation.We show that the use of synthesized speech in place of audio recording does not change the outcomes of the presented technique in a significant way.

This paper describes a challenge on Conversational AI correctness with the goal to develop Natural Language Understanding models that are robust against speech recognition errors.The data for the competition consist of natural language utterances along with semantic frames that represent the commands targeted at a virtual assistant.The specification of the task is given along with the data preparation procedure and the evaluation rules.The baseline models for the task are discussed and the results of the competition are reported.

This paper presents a new approach to the problem of correcting speech recognition errors by means of post-editing.It consists of using a neural sequence tagger that learns how to correct an ASR (Automatic Speech Recognition) hypothesis word by word and a corrector module that applies corrections returned by the tagger.The proposed solution is applicable to any ASR system, regardless of its architecture, and provides high-precision control over errors being corrected.This is especially crucial in production environments, where avoiding the introduction of new mistakes by the error correction model may be more important than the net gain in overall results.The results show that the performance of the proposed error correction models is comparable with previous approaches, while requiring much smaller resources to train, which makes it suitable for industrial applications, where both inference latency and training times are critical factors that limit the use of other techniques.