This study proposes an automatic bird sound identification system to help people learn to identify bird species. The system is built upon a two-stage identification framework. The first stage performs call/song classification. If an unknown sound clip is classified as a call, it is then handled by a call identifier in the second stage; otherwise, it is handled by a song identifier. Both identifiers use two acoustic features, timbre and pitch, to determine which of the bird species the sound clip belong to. In using timbre features, bird sounds are converted into MFCCs and their first derivatives and then analyzed using Gaussian mixture models. In using pitch feature, we convert bird sounds into MIDI note sequences and then use bigram models to analyze the dynamic change information of the notes. Our experiments, conducted using a database including twenty common bird species, show that the proposed system can achieve 90.4% accuracy.

This paper presents a query-by-singing/humming method that enables fast melody comparison. The basic idea is to measure the distances between note sequences in the frequency domain instead of time domain. Thanks to the merit of fast Fourier transform, we can convert different-length note sequences into equal-dimension vectors via zero padding. The equal dimensionality allows us to compare the vectors using Euclidean distance directly, which avoids performing time-consuming alignment between sequences. To take both efficiency and effectiveness into account, the proposed fast melody comparison method is combined with dynamic time warping technique into a two-stage sequence matching system. Our experiments conducted using the MIREX 2006 database demonstrate the feasibility of the proposed system.

This study aims to develop an automatic singing evaluation system for Karaoke performances. Many Karaoke systems in the market today come with a scoring function. The addition of the feature enhances the entertainment appeal of the system due to the competitive nature of humans. The automatic Karaoke scoring mechanism to date, however, is still rudimentary, often giving inconsistent results with scoring by human raters. A cause of blunder arises from the fact that often only the singing volume is used as the evaluation criteria. To improve on the singing evaluation capabilities on Karaoke machines, this study exploits various acoustic features, including pitch, volume, and rhythm to assess a singing performance. We invited a number of singers having different levels of singing capabilities to record for Karaoke solo vocal samples. The performances were rated independently by four musicians, and then used in conjunction with additional Karaoke Video Compact Disk music for the training of our proposed system. Our experiment shows that the results of automatic singing evaluation are close to the human rating, where the Pearson product-moment correlation coefficient between them is 0.82.

Currently existing singer identification (SID) methods follow the framework of speaker identification (SPID), which requires that singing data be collected beforehand to establish each singer's voice characteristics. This framework, however, is unsuitable for many SID applications, because acquiring solo a cappella from each singer is usually not as feasible as collecting spoken data in SPID applications. Since a cappella data are difficult to acquire, many studies have tried to improve SID accuracies when only accompanied singing data are available for training; but, the improvements are not always satisfactory. Recognizing that spoken data are usually available easily, this work investigates the possibility of characterizing singers' voices using the spoken data instead of their singing data. Unfortunately, our experiment found it difficult to replace singing data fully by using spoken data in singer voice characterization, due to the significant difference between singing and speech voice for most people. Thus, we propose two alternative solutions based on the use of few singing data. The first solution aims at adapting a speech-derived model to cover singing voice characteristics. The second solution attempts to establish the relationships between speech and singing using a transformation, so that an unknown test singing clip can be converted into its speech counterpart and then identified using speech-derived models; or alternatively, training data can be converted from speech into singing to generate a singer model capable of matching test singing clips. Our experiments conducted using a 20-singer database validate the proposed solutions.

Although many Karaoke systems come with a scoring feature that provides singers with a simple performance rating, the automated scoring, however, is either random or incomparable to that of human evaluation. This study exploits various acoustic features, including pitch, volume, and rhythm to assess a singing performance. We invited a number of singers having different levels of singing capabilities to record for Karaoke solo vocal samples. The performances were rated independently by four musicians, and then used in conjunction with additional Karaoke VCD music for the training of our proposed system. Our experiment shows that the results of automated singing evaluation are close to the human rating.

One major challenge of identifying singers in popular music recordings lies in how to reduce the interference of background accompaniment in trying to characterize the singer voice. Although a number of studies on automatic Singer IDentification (SID) from acoustic features have been reported, most systems to date, however, do not explicitly deal with the background accompaniment. This study proposes a background accompaniment removal approach for SID by exploiting the underlying relationships between solo singing voices and their accompanied versions in cepstrum. The relationships are characterized by a transformation estimated using a large set of accompanied singing generated by manually mixing solo singing with the accompaniments extracted from Karaoke VCDs. Such a transformation reflects the cepstrum variations of a singing voice before and after it is added with accompaniments. When an unknown accompanied voice is presented to our system, the transformation is performed to convert the cepstrum of the accompanied voice into a solo-voice-like one. Our experiments show that such a background removal approach improves the SID accuracy significantly; even when a test music recording involves sung language not covered in the data for estimating the transformation.

A prerequisite for identifying the singers in popular music recordings is to reduce the interference of background accompaniment when trying to characterize the singer voice. This study proposes a background music removal approach for singer identification (SID) by exploiting the underlying relationships between solo voices and their accompanied versions in cepstrum. The relationships are characterized by a transformation estimated using a large set of accompanied singing generated by manually mixing solo singing with the accompaniments extracted from Karaoke VCD. This transformation reflects the cepstrum variations of a singing voice before and after it is added with accompaniments. When an unknown accompanied voice is to be identified by our system, we convert its cepstrum into a solo-like one based on the pre-trained transformation. Our experiments show that such a background music removal approach improves the SID accuracy noticeably.

Speaker verification can be viewed as a task of modeling and testing two hypotheses: the null hypothesis and the alternative hypothesis. Since the alternative hypothesis involves unknown impostors, it is usually hard to characterizeapriori. In this paper, we propose improving the characterization of the alternative hypothesis by designing two decision functions based, respectively, on a weighted arithmetic combination and a weighted geometric combination of discriminative information derived from a set of pretrained background models. The parameters associated with the combinations are then optimized using two kernel discriminant analysis techniques, namely, the kernel Fisher discriminant (KFD) and support vector machine (SVM). The proposed approaches have two advantages over existing methods. The first is that they embed a trainable mechanism in the decision functions. The second is that they convert variable-length utterances into fixed-dimension characteristic vectors, which are easily processed by kernel discriminant analysis. The results of speaker-verification experiments conducted on two speech corpora show that the proposed methods outperform conventional likelihood ratio-based approaches.

This paper investigates the problem of retrieving karaoke music using query-by-singing techniques. Unlike regular CD music, where the stereo sound involves two audio channels that usually sound the same, karaoke music encompasses two distinct channels in each track: one is a mixture of the lead vocals and background accompaniment, and the other consists of accompaniment only. Although the two audio channels are distinct, the accompaniments in the two channels often resemble each other. We exploit this characteristic to: i) infer the background accompaniment for the lead vocals from the accompaniment-only channel, so that the main melody underlying the lead vocals can be extracted more effectively, and ii) detect phrase onsets based on the Bayesian information criterion (BIC) to predict the onset points of a song where a user's sung query may begin, so that the similarity between the melodies of the query and the song can be examined more efficiently. To further refine extraction of the main melody, we propose correcting potential errors in the estimated sung notes by exploiting a composition characteristic of popular songs whereby the sung notes within a verse or chorus section usually vary no more than two octaves. In addition, to facilitate an efficient and accurate search of a large music database, we employ multiple-pass dynamic time warping (DTW) combined with multiple-level data abstraction (MLDA) to compare the similarities of melodies. The results of experiments conducted on a karaoke database comprised of 1071 popular songs demonstrate the feasibility of query-by-singing retrieval for karaoke music.

Speaker verification based on the log-likelihood ratio (LLR) is essentially a task of modeling and testing two hypotheses: the null hypothesis and the alternative hypothesis. Since the alternative hypothesis involves unknown imposters, it is usually hard to characterize a priori. In this paper, we propose a framework to better characterize the alternative hypothesis with the goal of optimally separating client speakers from imposters. The proposed framework is built on either a weighted arithmetic combination or a weighted geometric combination of useful information extracted from a set of pre-trained anti-speaker models. The parameters associated with the combinations are then optimized using minimum verification error training such that both the false acceptance probability and the false rejection probability are minimized. Our experiment results show that the proposed framework outperforms conventional LLR-based approaches.

This paper presents an effective method for clustering unknown speech utterances based on their associated speakers. The proposed method jointly optimizes the generated clusters and the number of clusters by estimating and minimizing the Rand index of the clustering. The Rand index, which reflects clustering errors that utterances from the same speaker are placed in different clusters, or utterances from different speakers are placed in the same cluster, reaches its minimal value only when the number of clusters is equal to the true speaker population size. We approximate the Rand index by a function of the similarity measures between utterances and employ the genetic algorithm to determine the cluster where each utterance should be located, such that the overall clustering errors are minimized. The experimental results show that the proposed speaker-clustering method outperforms the conventional method based on hierarchical agglomerative clustering in conjunction with the Bayesian information criterion to determine the number of clusters.

It is usually difficult to characterize the alternative hypothesis precisely in a log-likelihood ratio (LLR)-based speaker verification system. In a previous work, we proposed using a weighted arithmetic combination (WAC) or a weighted geometric combination (WGC) of the likelihoods of the background models instead of heuristic combinations, such as the arithmetic mean and the geometric mean, to better characterize the alternative hypothesis. In this paper, we further propose learning the parameters associated with WAC or WGC via an evolutionary minimum verification error (MVE) training method, such that both the false acceptance probability and the false rejection probability can be minimized. Our experiment results show that the proposed methods outperform conventional LLR-based approaches. Index Terms: genetic algorithm, log-likelihood ratio, minimum verification error training, speaker verification

This paper investigates the problem of automatically grouping unknown speech utterances based on their associated speakers. In attempts to determine which utterances should be grouped together, it is necessary to measure the voice similarities between utterances. Since most existing methods measure the inter-utterance similarities based directly on the spectrum-based features, the resulting clusters may not be well-related to speakers, but to various acoustic classes instead. This study remedies this shortcoming by projecting utterances onto a reference space trained to cover the generic voice characteristics underlying the whole utterance collection. The resultant projection vectors naturally reflect the relationships of voice similarities among all the utterances, and hence are more robust against interference from nonspeaker factors. Then, a clustering method based on maximum purity estimation is proposed, with the aim of maximizing the similarities between utterances within all the clusters. This method employs a genetic algorithm to determine the cluster to which each utterance should be assigned, which overcomes the limitation of conventional hierarchical clustering that the final result can only reach the local optimum. In addition, the proposed clustering method adapts a Bayesian information criterion to determine how many clusters should be created

This paper investigates the problem of how to partition unknown speech utterances into clusters, such that the overall within-cluster homogeneity of speakers' voice characteristics can be maximized. The within-cluster homogeneity is characterized by the likelihood probability that a cluster model, trained using all the utterances within a cluster, matches each of the within-cluster utterances. Such probability is then maximized by using a genetic algorithm, which determines the best cluster where each utterance should be located. For greater computational efficiency, also proposed is an alternative solution that approximates the likelihood probability with a divergence-based model similarity. The method is further designed to estimate the optimal number of clusters automatically.

The performance of a likelihood ratio-based speaker verification system is highly dependent on modeling of the target speaker’s voice (the null hypothesis) and characterization of non-target speakers ’ voices (the alternative hypothesis). To better characterize the ill-defined alternative hypothesis, this study proposes a new likelihood ratio measure based on a composite-structure Gaussian mixture model, the so-called GMM2. Motivated by the combined use of a variety of background models to represent the alternative hypothesis, GMM2 is designed with an inner set of mixture weights connected to the significance of each individual Gaussian density, and an outer set of mixture weights connected to the significance of each individual background model. Through the use of kernel discriminant analysis namely, Kernel Fisher Discriminant (KFD) or Support Vector Machine (SVM), GMM2 is trained in such a manner that the utterances of the null hypothesis can be optimally separated from those of the alternative hypothesis. Index Terms: speaker verification, likelihood ratio, kernel Fisher discriminant, support vector machine

In this paper, we investigate the problem of automatic singer identification, detection and tracking in popular music recordings with one or multiple singers. This problem reflects an important issue in multimedia applications that require the transcription and indexing of music data to meet the increasing demand for content-based information retrieval. The major challenges for this study arise from the fact that a singer's voice tends to be arbitrarily altered from time to time and is inextricably intertwined with the signal of the background accompaniment. To determine who is singing, or whether or when a particular singer is present in a music recording, methods are presented for separating vocal from nonvocal regions, for isolating singers' vocal characteristics from background music, and for distinguishing singers from one another. Experimental evaluations conducted on a pop music database consisting of solo and duet tracks confirm the validity of the proposed methods.

The paper investigates the problem of automatically grouping unknown speech utterances based on their associated speakers. The proposed method utilizes the vector space model, which was originally developed in document-retrieval research, to characterize each utterance as a tf-idf-based vector of acoustic terms, thereby deriving a reliable measurement of similarity between utterances. To define the required acoustic terms that are most representative in terms of voice characteristics, the Eigenvoice approach is applied to the utterances to be clustered, which creates a set of eigenvector-based terms. To further improve speaker-clustering performance, the proposed method encompasses a mechanism of blind relevance feedback for refining the inter-utterance similarity measure.

This paper addresses the problem of automatically grouping unknown speech utterances that are from the same speaker. A clustering method based on maximum purity estimation is proposed, with the aim of maximizing the similarities of voice characteristics between utterances within all the clusters. This method employs a genetic algorithm to determine the cluster where each utterance should be located, which overcomes the limitation of conventional hierarchical clustering that the final result can only reach the local optimum. The proposed clustering method also incorporates a Bayesian information criterion to determine how many clusters should be created. 1.

In this paper, we investigate the problem of automatically detecting and tracking a specified person's singing portions within a music recording with multiple simultaneous or nonsimultaneous singers. This problem reflects an important issue in multimedia applications which require transcription and indexing of music data in response to the increasing demand nowadays for content-based information retrieval. The major challenges of this study arise from the fact that the singer's voices are inextricably intertwined with the signal of the background accompaniment. To determine whether or when an accompanied voice is present and from a sought singer, methods are presented for separating vocal from non-vocal regions, for extracting and modeling singers' vocal characteristics, and for distinguishing among vocal regions performed by the target singer and other simultaneous or non-simultaneous singers.

We present a framework for music document retrieval that allows users to retrieve a specified singer's music recordings from an unlabeled database by submitting a fragment of music as a query to the system. Such a framework can be of great use for those wishing to know more about a particular singer but having no idea about what the name of this singer is. In order for the searched documents to be relevant to the query, methods are proposed to compare the similarity between document and query based on the automatic extraction of a singer's voice characteristics from a music recording.

This paper presents an effective technique for clustering speech utterances based on their associated speaker. In attempts to determine which utterances are from the same speakers, a prerequisite is to measure the similarity of voice characteristics between utterances. Since the vast majority of existing methods evaluate the inter-utterance similarity by taking only the information from the spectrum-based features of utterance pairs into account, the resulting clusters may not be well relevant to speaker, but instead likely to the environmental conditions or other acoustic classes. To compensate for this shortcoming, this study proposes to project utterances from their spectrum-based feature representation onto a reference space trained to cover the generic voice characteristics inherently in all of the utterances to be clustered. The resultant projection vectors naturally reflect the relationships between all the utterances and are more robust against the interference from non-speaker factors. We exemplarily present three distinct implementations for reference space creation. 1.

In this paper, an iterative bandwidth mismatch compensation (BMC) algorithm is proposed to alleviate the need of multiple pre-trained models for recognizing different bandwidth speech. The BMC uses the concept of the bandwidth extension as similar as in the speech enhancement approaches. However, it aims at directly improving the recognition accuracy instead of speech intelligence or quality and utilizes only recognizer’s hidden Markov models (HMMs) for both bandwidth mismatch compensation and recognition. The BMC first detects the bandwidth of the input speech signal based on a divergence measurement. The HMM/Gaussian mixture model (GMM)based method is then used to iteratively segment the input speech utterance and compensates the speech features. Experiments on serious bandwidth mismatched conditions, i.e., training on 8 kHz and testing on 4 kHz or 5.5 kHz bandwidth database have verified the effectiveness of the proposed approach.

This study presents an effective technique for automatically identifying the singer of a music recording. Since the vast majority of popular music contains background accompaniment during most or all vocal passages, directly acquiring isolated solo voice data for extracting the singer’s vocal characteristics is usually infeasible. To eliminate the interference of background music for singer identification, we leverage statistical estimation of a piece’s musical background to build a reliable model for the solo voice. Validity of the proposed singer identification system is confirmed via the experimental evaluations conducted on a 23-singer pop music database. 1.