This paper summarizes our contributions to the document-grounded dialog tasks at the 9th and 10th Dialog System Technology Challenges (DSTC9 and DSTC10). In both iterations the task consists of three subtasks: first detect whether the current turn is knowledge seeking, second select a relevant knowledge document, and third generate a response grounded on the selected document. For DSTC9 we proposed different approaches to make the selection task more efficient. The best method, Hierarchical Selection, actually improves the results compared to the original baseline and gives a speedup of 24x. In the DSTC10 iteration of the task, the challenge was to adapt systems trained on written dialogs to perform well on noisy automatic speech recognition transcripts. Therefore, we proposed data augmentation techniques to increase the robustness of the models as well as methods to adapt the style of generated responses to fit well into the proceeding dialog. Additionally, we proposed a noisy channel model that allows for increasing the factuality of the generated responses. In addition to summarizing our previous contributions, in this work, we also report on a few small improvements and reconsider the automatic evaluation metrics for the generation task which have shown a low correlation to human judgments.

Named entity recognition is a key component in various natural language processing systems, and neural architectures provide significant improvements over conventional approaches. Regardless of different word embedding and hidden layer structures of the networks, a conditional random field layer is commonly used for the output. This work proposes to use a neural language model as an alternative to the conditional random field layer, which is more flexible for the size of the corpus. Experimental results show that the proposed system has a significant advantage in terms of training speed, with a marginal performance degradation.

Clustering techniques have been integrated at different levels into the training procedure of a continuous-density hidden Markov model (HMM) speech recognizer. These clustering techniques can be used in two ways. First acoustically similar states are tied together. It will help to reduce the number of parameters but also allow to train otherwise rarely seen states together with more robust ones (state-tying). Secondly densities are clustered across states, this reduces the number of densities while at the same time keeping the best performances of our recognizer (density-clustering). We have applied these techniques both to word-based small-vocabulary and phoneme-based large-vocabulary recognition tasks. On the WSJ task, we could achieve a reduction of the word error rate by 7%. On the TI/NIST-connected digit task, the number of parameters was reduced by a factor 2-3 while keeping the same string error rate.

We report on recent developments of the Philips large vocabulary speech recognition system and on our experiments with the Wall Street Journal (WSJ) corpus. A two-pass decoding has been devised that allows an easy integration of more complex language models. First, a word lattice is produced using a time synchronous beam search with a bigram language model. Next, a higher-order language model is applied to the lattice at the phrase level. The conditions insuring the validity of this approach are explained and practical results for trigram demonstrate its usefulness. The main system development stages on WSJ data are presented and our final recognizers are evaluated on Nov. '92 and Nov. '93 test-data for both 5 K and 20 K vocabularies.>

Hybrid systems, using neural networks (NNs) and hidden Markov models (HMMs) are designed to take advantage of both methods; the pattern classification power of NNs and the temporal modelling structure of HMMs. This paper describes the use of expert sub-network modules of the type time delay neural network (TDNN) for phone recognition of continuous speech. The originality of the hybrid system developed is in combining the probabilities of the modular TDNN architecture with those of CDHMMs during the recognition phase. On three speakers of the DARPA RM speaker-dependent task, we show that these small TDNNs trained on phone ambiguities can improve word recognition performance of state-of-the-art CDHMMs. The TDNN implementation achieved a word error rate reduction of 15%. We discuss strategies for extending this approach from the DARPA RM speaker-dependent database to the larger DARPA RM speaker-independent database.>

The paper presents a hybrid continuous-speech recognition system that leads to improved results on the speaker dependent DARPA Resource Management task. This hybrid system, called the combined system, is based on a combination of normalized neural network output scores with hidden Markov model (HMM) emission probabilities. The neural network is trained under mean square error and the HMM is trained under maximum likelihood estimation. In theory, whatever criterion may be used, the same word error rate should be reached if enough training data is available. As this is never the case, the idea of combining two different criteria, each of them extracting complementary characteristics of the feature is interesting. A state-of-the-art HMM system will be combined with a time delay neural network (TDNN) integrated in a Viterbi framework. A hierarchical TDNN structure is described that splits training into subtasks corresponding to subsets of phonemes. This structure makes training of TDNNs on large-vocabulary tasks manageable on workstations. It will be shown that the combined system, despite the low accuracy of the hierarchical TDNN, achieves a word error rate reduction of 15% with respect to our state-of-the-art HMM system. This reduction is obtained with a 10% increase only in the number of parameters.

This paper gives a status report of the Philips research system for phoneme-based, large-vocabulary, continuous-speech recognition. Like for many other systems, the recognition architecture is based on an integrated statistical approach. We describe the characteristic features of the system as opposed to other systems: 1. The Viterbi criterion is consistently applied both in training and testing. 2. Continuous mixture densities are used without tying or smoothing. 3. Time-synchronous beam search in connection with a phoneme look-ahead is applied to a tree-organized lexicon. The system has been successfully applied to the American English DARPA RM task. Here, we report experimental results for a German 13 000-word Philips internal dictation task. In addition to the scientific prototype, a PC version has been set up which is described here for the first time.

A comparison of several architectures of time-delayed neural networks (TDNN) as the preprocessing step for hidden Markov modeling (HMM) speaker-dependent continuous-speech recognition systems is presented. A modular TDNN architecture on the basis of acoustic-phonetic knowledge, where each sub-network is trained on a different subset of phonemes is defined. It allows the authors to define a hierarchical tree structure of sub-networks. This structure offers the possibility of proposing a framework to enlarge the number of outputs by defining context-dependent sub-networks. They also compare different methods for integrating TDNN in a HMM framework, a discrete and a continuous integration. For the speaker JWS-4 of the speaker-dependent DARPA RM1 database, with context independent phonemes, 21.3% word error rate are obtained without grammar, 4.6% with the DARPA word-pair grammar (perplexity of 60).>

Some results obtained when the recognition vocabulary size of a phoneme-based speaker-dependent continuous-speech recognizer was increased from 1000 to 10000 words are reported. The potential search space increased from 46000 to 516000 states without problems for the data-driven search. Increasing the recognition vocabulary by a factor of 10 (from a perplexity of 917 to 9686) increased the word error rate by a factor of two (from 21.8% to 43.1%). Phoneme models were tested with both discrete probabilities and continuous mixture densities. The mixture density models performed better; moreover, they saved about half of the search costs. A language model was found to be very important for a larger vocabulary size. With a test set perplexity of 388 (i.e. a reduction by a factor of 25 compared to the case without a bigram model) the error rate decreased by a factor of 2.4. In order to check how meaningful perplexity is for the prediction of the system's performance, a stochastic language model was constructed with a perplexity of 1000, the size of the vocabulary used in previous experiments, and about the same error rate was obtained.>