Household speaker identification with few enrollment utterances is an important yet challenging problem, especially when household members share similar voice characteristics and room acoustics. A common embedding space learned from a large number of speakers is not universally applicable for the optimal identification of every speaker in a household. In this work, we first formulate household speaker identification as a few-shot open-set recognition task and then propose a novel embedding adaptation framework to adapt speaker representations from the given universal embedding space to a household-specific embedding space using a set-to-set function, yielding better household speaker identification performance. With our algorithm, Open-set Few-shot Embedding Adaptation with Transformer (openFEAT), we observe that the speaker identification equal error rate (IEER) on simulated households with 2 to 7 hard-to-discriminate speakers is reduced by 23% to 31% relative.

This paper proposes a novel formulation of prototypical loss with mixup for speaker verification. Mixup is a simple yet efficient data augmentation technique that fabricates a weighted combination of random data point and label pairs for deep neural network training. Mixup has attracted increasing attention due to its ability to improve robustness and generalization of deep neural networks. Although mixup has shown success in diverse domains, most applications have centered around closed-set classification tasks. In this work, we propose contrastive-mixup, a novel augmentation strategy that learns distinguishing representations based on a distance metric. During training, mixup operations generate convex interpolations of both inputs and virtual labels. Moreover, we have reformulated the prototypical loss function such that mixup is enabled on metric learning objectives. To demonstrate its generalization given limited training data, we conduct experiments by varying the number of available utterances from each speaker in the VoxCeleb database. Experimental results show that applying contrastive-mixup outperforms the existing baseline, reducing error rate by 16% relatively, especially when the number of training utterances per speaker is limited.

We address performance fairness for speaker verification using the adversarial reweighting (ARW) method.ARW is reformulated for speaker verification with metric learning, and shown to improve results across different subgroups of gender and nationality, without requiring annotation of subgroups in the training data.An adversarial network learns a weight for each training sample in the batch so that the main learner is forced to focus on poorly performing instances.Using a min-max optimization algorithm, this method improves overall speaker verification fairness.We present three different ARW formulations: accumulated pairwise similarity, pseudo-labeling, and pairwise weighting, and measure their performance in terms of equal error rate (EER) on the VoxCeleb corpus.Results show that the pairwise weighting method can achieve 1.08% overall EER, 1.25% for male and 0.67% for female speakers, with relative EER reductions of 7.7%, 10.1% and 3.0%, respectively.For nationality subgroups, the proposed algorithm showed 1.04% EER for US speakers, 0.76% for UK speakers, and 1.22% for all others.The absolute EER gap between gender groups was reduced from 0.70% to 0.58%, while the standard deviation over nationality groups decreased from 0.21 to 0.19.

This paper considers humming-based systems for human verification and identification. Humming of a target person is modeled as a Gaussian mixture model, and the matching score between a target model and humming is computed as the likelihood of humming given a target model. Verification is performed by comparing the matching score to the likelihood given a universal background model, and identification is performed by selecting the best-matched model. The verification and identification performances are evaluated using various acoustical features. The experimental results show that linear prediction cepstral coefficients and perceptually linear prediction coefficients are conducive to verification and identification, respectively.

In this paper, a novel multimedia identification system based on quantum hashing is considered. Many traditional systems are based on binary hash which is obtained by encoding intermediate hash extracted from multimedia content. In the system considered, the intermediate hash values extracted from a query are encoded into quantum hash values by incorporating uncertainty in the binary hash values. For this, the intermediate hash difference between the query and its true-underlying content is considered as a random process. Then, the uncertainty is represented by the probability density estimate of the intermediate hash difference. The quantum hashing system is evaluated using both audio and video databases, and with marginal increment in computational cost, the quantum hashing system is shown to be more robust against various distortions than the binary hashing system using the same intermediate hash values.

This paper proposes a novel music genre classification system based on two novel features and a weighted voting. The proposed features, modulation spectral flatness measure (MSFM) and modulation spectral crest measure (MSCM), represent the time-varying behavior of a music and indicate the beat strength. The weighted voting method determines the music genre by summarizing the classification results of consecutive time segments. Experimental results show that the proposed features give more accurate classification results when combined with traditional features than the octave-based modulation spectral contrast (OMSC) does in spite of short feature vector and that the weighted voting is more effective than statistical method and majority voting.

Motivated by the effectual use of temporal information in speech recognition, we investigate the effectiveness of the temporal dynamics of the spectral sub-band centroid (SSC) fingerprints for audio fingerprinting. The SSC, which is known to be a robust audio fingerprint against various distortions, does not involve any temporal dynamics. Here, the temporal dynamics are defined as the difference between two neighboring SSCs. The robustness of the temporal dynamics against various distortions were compared to that of SSCs. The system using temporal dynamics showed similar performance to that using SSCs in various distortions except for time-scale modification and linear-speed change. This is to be expected since these distortions change the time correlation of an audio. In our experiment, the concatenation of SSCs and the temporal dynamics outperformed each of the individual fingerprints. This suggests that the SSCs and the temporal dynamics provide information which is mutually supplementary.

This paper proposes a syllable-lattice-based speaker verification algorithm for Mandarin Chinese input. For each speech utterance, a syllable lattice is generated with a speaker-independent large-vocabulary continuous speech recognition system in free syllable decoding. The verification decision is made based upon the likelihood ratio between a target-speaker model and a speaker-independent background model, computed on the decoded syllable lattice. The likelihood function is calculated efficiently in a forward algorithm by considering all paths in the lattice. The proposed algorithm was evaluated using a Mandarin Chinese database, where 1832 true and 26 250 impostor trials were recorded by 19 target speakers and 180 impostors. The average duration of each trial is 2 s long without silence. The target-speaker model was adapted from the speaker-independent background model using enrollment data of two minutes with silence. The proposed algorithm achieved an equal-error rate of 0.857% which is better than 1.21% of the hidden Markov model-based speaker verification algorithm without using syllable lattices. The equal-error rate was further reduced to 0.617% by incorporating the Goussian mixture model-universal background model algorithm with 2048 Gaussian kernels whose equal error rate is 0.990%.

The Gaussian mixture based GMM-UBM approaches have shown good performance in speaker verification without using contextual information. In this paper, we exploit the information provided in the arcs of a decoded syllable lattice for speaker verification. The forward algorithm is used to summarize this information in the syllable lattice instead of the best decoded string. The performance is evaluated on a Mandarin Chinese database. With two minutes of target speaker's enrollment data, the proposed algorithm shows 1.03% of equal-error rate for short input utterances with an average duration of two seconds. By combining with the GMM-UBM, the system shows a 0.74% of equal-error rate.

The performance of a fingerprinting system, which is often measured in terms of reliability and robustness, is directly related to the features that the system uses. In this letter, we present a new audio-fingerprinting method based on the normalized spectral subband moments. A threshold used to reliably determine a fingerprint match is obtained by modeling the features as a stationary process. The robustness of the normalized moments was evaluated experimentally and compared with that of the spectral flatness measure. Among the considered subband features, the first-order normalized moment showed the best performance for fingerprinting.

For multimedia fingerprinting, it is crucial to extract relevant features that allow direct access to the distinguishing characteristics of a multimedia object. Features used for fingerprinting directly relate to the performance of the entire fingerprinting system. The paper proposes a novel audio fingerprinting method based on normalized spectral subband centroids. The spectral subband centroid is selected due to its resilience against equalization, compression, and noise addition. Both reliability and robustness issues in the fingerprinting system are addressed. Experimental results show that the proposed method is not only reliable, but also robust against various audio processing steps, including MP3 compression, equalization, random start, time-scale modification, and linear speed change.

In this paper, utterance verification based on hybrid scores obtained from three pairs of models is investigated. The three models considered are the on-line garbage model, the antiword function model and a model derived using Kullback-Leibler divergence. The performance of utterance verification algorithm using hypothesis testing depends on the accuracy of the estimate of the alternative hypothesis. The three models offer different perspectives in the probability estimation of the alternate hypothesis. Performance comparison between hybrid scores using different model pairs is made. In addition, performance improvement over conventional algorithm is experimentally verified.

Regardless of the distribution of the adaptation data in the testing environment, model-based adaptation methods that have so far been reported in the literature incorporate the adaptation data undiscriminately in reducing the mismatch between the training and testing environments. When the amount of data is small and the parameter tying is extensive, adaptation based on outlier data can be detrimental to the performance of the recognizer. The distribution of the adaptation data plays a critical role on the adaptation performance. In order to maximally improve the recognition rate in the testing environment using only a small amount of adaptation data, supervised weighted training is applied to the structural maximum a posterior (SMAP) algorithm. We evaluate the performance of the proposed weighted SMAP (WSMAP) and SMAP on TIDIGITS corpus. The proposed WSMAP has been found to perform better for a small amount of data. The general idea of incorporating the distribution of the adaptation data is applicable to other adaptation algorithms.

This paper presents a novel method to enhance the performance of traditional speaker adaptation algorithm using discriminative adaptation procedure based on a novel confidence measure and non-linear weighting. Regardless of the distribution of the adaptation data, traditional model adaptation methods incorporate the adaptation data undiscriminatingly. When the data size is small and the parameter tying is extensive, adaptation based on outliers can be detrimental. A way to discriminate the contribution of each data in the adaptation is to incorporate a confidence measure based on likelihood. We evaluate and compare the performances of the proposed weighted SMAP (WSMAP) which controls the contribution of each data by sigmoid weighting using a novel confidence measure. The effectiveness of the proposed algorithm is experimentally verified by adapting native speaker models to nonnative speaker environment using TIDIGIT. 1.