A recently proposed time-delay neural network (TDNN)-based age estimation system has yielded state-of-the-art performance in speech-age estimation tasks. However, the performance of this TDNN-based system can seriously degrade when the recording conditions of each utterance are different in the training and testing phases. To tackle this problem, we examine the efficiencies of a series of unsupervised domain adaptation (UDA) methods to obtain the model invariance against the difference of these conditions. In particular, we propose using local maximum mean discrepancy (LMMD) with soft-target labels to consider an ordinal relationship between age labels. In most UDA methods, the model is trained to obtain domain invariant representations by minimizing the statistical difference of the distributions between labeled source and unlabeled target data without considering their age class labels. In contrast, our LMMD-based approach locally minimizes the differences in their distributions on each age class while considering adjacent age classes using soft-target labels. We conducted speech-age estimation experiments on in-house datasets under mismatched conditions including different background noise, reverberation, and microphones. The experimental comparison demonstrated that the LMMD-based method contributed to efficiently reducing the effect of mismatches of input data, yielding significant improvements over other UDA methods, such as MMD and reverse gradients.

Estimating a speaker’s age from their speech is more challenging than age estimation from their face because of insufficiently available public corpora. To tackle this problem, we construct a new audio-visual age corpus named AgeVoxCeleb by annotating age labels to VoxCeleb2 videos. AgeVoxCeleb is the first large-scale, balanced, and multi-modal age corpus that contains both video and speech of the same speakers from a wide age range. Using AgeVox-Celeb, our paper makes the following contributions: (i) A facial age estimation model can outperform a speech age estimation model by comparing the state-of-the-art models in each task. (ii) Facial age estimation is more robust against the difference between training and test sets. (iii) We developed cross-modal transfer learning from face to speech age estimation, showing that the estimated age with a facial age estimation model can be used to train a speech age estimation model. Proposed AgeVoxCeleb will be published in https://github.com/nttcslab-sp/agevoxceleb.

This paper proposes a general post-processing method for improving speaker-attribute estimation. Estimating speaker-specific attributes such as age and gender is an important task with a wide range of applications. While the recent proposed deep neural network-based end-to-end approach achieves high performance, the model tends to over-fit to specific speakers when the amount of training data is limited or imbalanced. To solve this over-fitting problem, we propose a general framework for correcting unreliable results. The proposed algorithm first clusters the target utterances into speaker clusters by speaker similarity based on i-vectors. Then, for each of the speaker cluster, the speaker-attribute class of the cluster is determined by voting on the utterances assigned to the cluster. By then replacing the result of each utterance with the clusters' speaker-attribute class, we can correct the result of unreliable utterances. We used two tasks to evaluate the proposed algorithm including age estimation using the NIST-SRE10 and age-gender classification using an in-house read speech corpus, yielding significant improvements in mean absolute and classification errors.

This article presents a novel customer satisfaction (CS) estimation method that outputs both turn-level and call-level estimations simultaneously. Our key idea is to directly apply turn-level estimation results to call-level estimation and optimize them jointly; previous works treat both as being independent. Our proposal applies long short-term memory recurrent neural networks (LSTM-RNNs) to turn-level and call-level CS estimation to capture long-range sequential context in contact center calls. In addition, both networks are hierarchically stacked so as to use turn-level estimation results for call-level estimation directly. In order to learn the relationship between the two tasks, we also introduce joint optimization training to the stacked model. Several analyses of turn-level and call-level CS are provided on acted and real calls to support the proposed method. Experiments show that the proposed framework outperforms the conventional methods in both turn-level and call-level estimations.

This paper presents a novel emotion classification method for natural speech. One of the problems in the state-of-the-art method based on Deep Neural Network (DNN) is the paucity of the training data compared to model complexity. To solve this problem, this paper utilizes the ambiguous emotional utterances, utterances that have no dominant target emotion label. While previous work ignored ambiguous emotional utterances for training, the proposed method leverages all annotated labels via soft-target training. In addition, this paper modifies the soft-target training in order to effectively handle both clear and ambiguous emotional utterances. Experiments show that the proposed method yields performance improvements in terms of both weighted and unweighted accuracies.