We propose two techniques to enhance the performance of recurrent neural network (RNN)-based acoustic models. The first technique addresses training efficiency. Because RNNs require sequential input, it is difficult to randomly shuffle training samples to accelerate stochastic gradient descent based training. We propose a "pseudo-shuffling" procedure that instead augments training sample unexpectedness by skipping successive samples. The second proposed technique is a novel "direct decoding" framework in which the posterior probability of the RNN is inputted into a decoder without conversion into a hidden Markov model emission probability. In our large vocabulary speech recognition experiments with English lecture recordings, the first technique significantly improved RNN training efficiency, showing a 14.3% relative word error rate (WER) improvement. The second technique further achieved an additional 3.1% relative WER improvement. Our sigmoid-type RNN achieved a 10.7% better WER than same-sized deep neural networks without using long short-term memory cells.

We developed a robot system which can play one-on-one volleyball games with a human player. It can not only hit a flying ball back to the player correctly but also have various autonomous functionalities to interact with humans necessary for playing games, i.e., it can pick up a ball of a requested color by recognizing voice instructions, can locate the players by recognizing their faces, and can shake hands. This robot performed the entire game sequence over 100 times at a company-sponsored exhibition, where the ball rally continued over 25 times with an arbitrary player. In this paper, we mainly describe the real-time visual feedback system of the robot which is essential for playing ball games.

We have proposed a new wordspotting method, combining word-based pattern matching and phoneme-based hidden Markov model (HMM). Word-based pattern matching based on the time-frequency representation of a whole word pattern is robust against pattern variations and background noise, while the phoneme-based HMM, which represents phonemic features within a word pattern, is flexible for expanding the vocabulary. Because of the difference in scope, these two have their own characteristics in terms of robustness and accuracy. To take advantage of the features of these two, we have integrated these different types of wordspotting results under a unified criterion. A syntactic and semantic parser is also utilized to prune the wordspotting results for spontaneous speech understanding. Experimental results indicate the effectiveness of the proposed method.