This paper investigates acoustic modeling for recognition of bird species from audio field recordings. First, the acoustic scene is decomposed into isolated segments, corresponding to detected sinusoids. Each segment is represented by a sequence of the frequency and normalized magnitude values of the sinusoid. The temporal evolution of these features is modeled using hidden Markov models (HMMs). A novel method for an unsupervised modeling of individual bird vocalization elements is proposed. The element models are initialized using HMM-based clustering and then further trained using an iterative maximum likelihood label re-assignment procedure. State duration modeling, performed in a post-recognition stage, is explored. Finally, we developed a hybrid deep neural network—hidden Markov model. The developed acoustic models are employed for bird species identification, detection of specific species, and recognition of multiple bird species vocalizing in a given recording. The detection system employs score normalization. Recognition of multiple bird species is performed based on maximizing the likelihood of a set of segments on a subset of bird species models, with penalization based on Bayesian information criterion applied. Experimental evaluations are performed on more than 37 h of sound field recordings, containing vocalizations of 48 bird species, plus more than 16 h of non-bird sound recordings. Using 3 s of the detected signal, the best system achieved: identification accuracy of 98.7%, detection with the equal error rate of 2.7%, and recognition accuracy of 97.3% and 95.4% when vocalizations of multiple bird species are present, with the number of bird species known and estimated, respectively.

Neural networks have a reputation for being "black boxes", which it has been suggested that techniques from user interface development and visualisation in particular, could help lift. In this paper, we explore 9-dimensional bottleneck features (BNFs) that have been shown in our earlier work to well represent speech in the context of speech recognition and 2-dimensional BNFs directly extracted from bottleneck neural networks. The 9-dimensional BNFs obtained from a phone classification neural network are visualised in 2-dimensional spaces using linear discriminant analysis (LDA) and t-distributed stochastic neighbour embedding (t-SNE). The 2-dimensional BNF space is analysed with regard to phonetic features. A back-propagation method is used to create "cardinal" features for each phone under a particular neural network. Both the visualisations of 9-dimensional and 2-dimensional BNFs show distinctions between most phone categories. Particularly, the 2-dimensional BNF space seems to be a union of phonetic category related subspaces that preserve local structures within each subspace where the organisations of phones appear to correspond to phone production mechanisms. By applying LDA to the features of higher dimensional non-bottleneck layers, we observe a triangular pattern which may indicate that silence, friction and voicing are the three main properties learned by the neural networks.

Although it is generally accepted that different broad phone classes (BPCs) have different production mechanisms and are better described by different types of features, most automatic speech recognition (ASR) systems use the same features and decision criteria for all phones. Motivated by this observation, this paper proposes a two-level DNN structure, referred to as a BPC-DNN, inspired by the notion of a topological manifold. In the first level, several small separate BPC-dependent DNNs are applied to different broad phonetic classes and in the second level the outputs of these DNNs are fused to obtain senone-dependent posterior probabilities, which can be used for frame level classification or integrated into Viterbi decoding for phone recognition. In a previous paper using this approach we reported improved frame classification accuracy on the TIMIT corpus compared with a conventional DNN. The contribution of the present paper is to demonstrate that this advantage extends to full phone recognition. Our most recent results show that the BPC-DNN achieves reductions in error rate relative to a conventional DNN of 16% and 8% for frame classification and phone recognition, respectively.

This paper describes the systems developed by the University of Birmingham for the 2018 CALL Shared Task (ST) challenge. The task is to perform automatic assessment of grammatical and linguistic aspects of English spoken by German-speaking Swiss teenagers. Our developed systems consist of two components, automatic speech recognition (ASR) and text processing (TP). We explore several ways of building a DNN-HMM ASR system using out-of-domain AMI speech corpus plus a limited amount of ST data. In development experiments on the initial ST data, our final ASR system achieved the word-error-rate (WER) of 12.00%, compared to 14.89% for the official ST baseline DNN-HMM system. The WER of 9.28% was achieved on the test set data. For TP component, we first post-process the ASR output to deal with hesitations and then pass this to a template-based grammar, which we expanded from the provided baseline. We also developed a TP system based on machine learning methods, which enables to better accommodate variability of spoken language. We also fused outputs from several systems using a linear logistic regression. Our best system submitted to the challenge achieved F-measure of 0.914, D of 10.764 and D_{full} score of 5.691 on the final test set.

Most state-of-the-art automatic speech recognition (ASR) systems use a single deep neural network (DNN) to map the acoustic space to the decision space. However, different phonetic classes employ different production mechanisms and are best described by different types of features. Hence it may be advantageous to replace this single DNN with several phone class dependent DNNs. The appropriate mathematical formalism for this is a manifold. This paper assesses the use of a nonlinear manifold structure with multiple DNNs for phone classification. The system has two levels. The first comprises a set of broad phone class (BPC) dependent DNN-based mappings and the second level is a fusion network. Various ways of designing and training the networks in both levels are assessed, including varying the size of hidden layers, the use of the bottleneck or softmax outputs as input to the fusion network, and the use of different broad class definitions. Phone classification experiments are performed on TIMIT. The results show that using the BPC-dependent DNNs provides small but significant improvements in phone classification accuracy relative to a single global DNN. The paper concludes with visualisations of the structures learned by the local and global DNNs and discussion of their interpretations.

This paper presents an HMM-based automatic system for recognition of bird species from audio field recordings. It includes an improved unsupervised modelling of individual bird syllables and duration modelling. The acoustic signal is decomposed into isolated segments, each segment containing a temporal evolution of a detected sinusodal component. Modelling of bird syllables is performed using Hidden Markov models (HMMs). A set of syllables of bird vocalisations is discovered in an unsupervised manner by employing dynamic time warping and agglomerative hierarchical clustering. A novel iterative maximum likelihood procedure is used to train individual HMMs for syllables of each species. Modelling of the state duration is employed in a post-recognition stage by combining the likelihood of the acoustic and duration modelling. Experiments are performed on over 33 hours of field recordings, containing 30 bird species. Evaluations demonstrate that the use of the proposed un-supervised iterative HMM training procedure and the duration modelling provides in average 45% error rate reduction. The presented system recognises bird species with accuracy of 97.8% using 3 seconds of the detected signal.

Recent advances in automatic speech recognition have used large corpora and powerful computational resources to train complex statistical models from high-dimensional features, to attempt to capture all the variability found in natural speech. Such models are difficult to interpret and may be fragile, and contradict or ignore knowledge of human speech production and perception. We report progress towards phoneme recognition using a model of speech which employs very few parameters and which is more faithful to the dynamics and model of human speech production. Using features generated from a neural network bottleneck layer, we obtain recognition accuracy on TIMIT which compares favourably with traditional models of similar power. We discuss the implications of these results for recognition using natural features such as vocal tract resonances and spectral energies.

This paper presents an extension of our recent work on recognition of multiple bird species from their vocalisations by incorporating an improved acoustic modelling. The acoustic scene is segmented into spectro-temporal isolated segments by employing a sinusoidal detection algorithm, which is able to handle multiple simultaneous bird vocalisations. Each segment is represented as a temporal sequence of frequencies of the detected sinusoid. Each bird species is represented by a set of hidden Markov models (HMMs), each HMM modelling a particular vocalisation element. A set of elements is discovered in an unsupervised manner using a partial dynamic time warping algorithm and agglomerative hierarchical clustering. Recognition of multiple bird species is performed based on maximising the likelihood of the set of detected segments on a subset of bird species models, with a penalisation applied for increasing the number of bird species. Experimental evaluations used audio field recordings containing 30 bird species. Detected segments from several bird species are joined to simulate the presence of multiple bird species. It is demonstrated that the use of improved acoustic modelling in conjunction with the maximum likelihood score combination method provides considerable improvements over previous results and the use of majority voting.

Low-dimensional ‘bottleneck’ features extracted from neural networks have been shown to give phoneme recognition accuracy similar to that obtained with higher-dimensional MFCCs, using GMM-HMM models. Such features have also been shown to preserve well the assumptions of speech trajectory dynamics made by dynamic models of speech such as Continuous-State HMMs. However, little is understood about how networks derive these features and how and whether they can be interpreted in terms of human speech perception and production. We analyse three-dimensional bottleneck features. We show that for vowels, their spatial representation is very close to the familiar F1 :F2 vowel quadrilateral. For other classes of phonemes the features can similarly be related to phonetic and acoustic spatial representations presented in the literature. This suggests that these networks derive representations specific to particular phonetic categories, with properties similar to those used by human perception. The representation of the full set of phonemes in the bottleneck space is consistent with a hypothesized comprehensive model of speech perception and also with models of speech perception such as prototype theory.

This paper presents an automatic system for recognition of bird species from audio field recordings. The acoustic signal is first segmented into isolated time-frequency segments, each corresponding to an individual detected sinusoidal component. Each segment is represented by a temporal sequence of the frequency values of the detected sinusoid, referred to as frequency track. Hidden Markov models (HMMs) are employed to model the temporal evolution of frequency track features. Individual syllables of bird vocalisations are discovered using an unsupervised method based on dynamic time warping and agglomerative hierarchical clustering. The outcome of this is then employed to create individual HMMs for syllables of each species. Experiments are performed on over 33 hours of field recordings, containing 30 bird species. Evaluations demonstrate that the use of individual syllable HMMs provides over 40% error rate reduction over the use of single HMM for each bird species of the same complexity. The syllable HMM-based system recognises bird species with accuracy over 95% using 3 seconds of detected signal.

This paper presents an analysis of a low-dimensional representation of speech for modelling speech dynamics, extracted using bottleneck neural networks. The input to the neural network is a set of spectral feature vectors. We explore the effect of various designs and training of the network, such as varying the size of context in the input layer, size of the bottleneck and other hidden layers, and using input reconstruction or phone posteriors as targets. Experiments are performed on TIMIT. The bottleneck features are employed in a conventional HMM-based phoneme recognition system, with recognition accuracy of 70.6% on the core test achieved using only 9-dimensional features. We also analyse how the bottleneck features fit the assumptions of dynamic models of speech. Specifically, we employ the continuous-state hidden Markov model (CS-HMM), which considers speech as a sequence of dwell and transition regions. We demonstrate that the bottleneck features preserve well the trajectory continuity over time and can provide a suitable representation for CS-HMM.

Alarm sounds triggered by biomedical equipment play a key role in providing healthcare in a neonatal intensive care unit (NICU). This paper presents our work on automatic detection of acoustic alarms in a noisy NICU environment, where knowledge about the particular characteristics of each alarm class is integrated at different stages of the detection system. The feature extraction is based on applying, around alarm-specific frequencies, a method for detection of sinusoidal signals, which employs the normalised short-term magnitude and phase spectrum. Also, the ratios of magnitudes at those frequencies are\ntaken as features. The system consists of a set of GMM-based detectors, each designed to deal with a specific alarm. Temporal structure of alarms, in terms of duration of signal and silence intervals in every alarm period, is incorporated by aggregating the frame-level posterior probabilities. The experimental evaluations are performed with a database recorded in a real-world hospital environment. The performance of the detection system is assessed both at the frame level and at the alarm period level.

Research into human perception of consonants has identified phoneme-specific perceptual cues. It has also been shown that the characteristics of the speech signal most useful for recognition depend on the specific speech sound. Typical ASR features and recognisers however neither vary with the type of sound nor relate directly to perceptual cues. We investigate classification and decoding of non-sonorant consonants using basic perceptually-motivated features — phoneme durations and energy in a few broad spectral bands. Our classification results using simple classifiers suggest that features optimal for human perception also perform best for machine classification. We show how characteristics of the models learned relate to knowledge of human speech perception. Recognition results using a continuous-state HMM (CSHMM) show accuracy similar to a discrete-state HMM with similar assumptions. We conclude by outlining how the CSHMM provides a mechanism to make use of other perceptually-important features by integration with similar models for recognition of voiced sounds.

Automatic system for recognition of multiple bird species in audio recordings is presented. Time-frequency segmentation of the acoustic scene is obtained by employing a sinusoidal detection algorithm, which does not require any estimate of noise and is able to handle multiple simultaneous bird vocalizations. Each segment is characterized as a sequence of frequencies over time, referred to as a frequency track. Each bird species is represented by a hidden Markov model that models the temporal evolution of frequency tracks. The decision on the number and identity of bird species in a given recording is obtained based on maximizing the overall likelihood of the set of detected segments, with a penalization applied for increasing the number of bird models used. Experimental evaluations are performed on audio field recordings containing 30 bird species. The presence of multiple bird species is simulated by joining the set of detected segments from several bird species. Results show that the proposed method can achieve recognition performance for multiple bird species not far from that obtained for single bird species, and considerably outperforms majority voting methods.

This paper presents an automatic system for recognition of bird species from field audio recordings. The proposed system employs a novel method for detection of sinusoidal components in the acoustic scene. This provides a segmentation of the signal and also feature representation of each segment in terms of frequencies over time, referred to as frequency track. We employ hidden Markov models (HMMs) to model the temporal evolution of frequency tracks. We demonstrate the effect of including local temporal dynamics of frequency tracks and HMM modelling parameters. Experiments are performed on over 33 hours of field recordings, containing 30 bird species. Evaluations demonstrate that the HMM-based temporal modelling provides considerable performance improvement over a system based on Gaussian mixture modelling. The proposed HMM-based system is capable of recognising bird species with accuracy over 85% from only 3 seconds of detected signal.

Many current speech models used in recognition involve thousands of parameters, whereas the mechanisms of speech production are conceptually very simple. We present and evaluate a new continuous state probabilistic model (CS-HMM) for recovering dwell-transition and phoneme sequences from dynamic speech production features. We show that with very few parameters, these features can be tracked, and phoneme sequences recovered, with promising accuracy.

This paper presents results on Speaker Recognition (SR) for children's speech, using the OGI Kids corpus and GMM-UBM and GMM-SVM SR systems. Regions of the spectrum containing important speaker information for children are identified by conducting SR experiments over 21 frequency bands. As for adults, the spectrum can be split into four regions, with the first (containing primary vocal tract resonance information) and third (corresponding to high frequency speech sounds) being most useful for SR. However, the frequencies at which these regions occur are from 11% to 38% higher for children. It is also noted that subband SR rates are lower for younger children. Finally results are presented of SR experiments to identify a child in a class (30 children, similar age) and school (288 children, varying ages). Class performance depends on age, with accuracy varying from 90% for young children to 99% for older children. The identification rate achieved for a child in a school is 81%.

This letter presents an experimental study investigating the effect of frequency sub-bands on regional accent identification (AID) and speaker identification (SID) performance on the ABI-1 corpus. The AID and SID systems are based on Gaussian mixture modeling. The SID experiments show up to 100% accuracy when using the full 11.025 kHz bandwidth. The best AID performance of 60.34% is obtained when using band-pass filtered (0.23-3.4 kHz) speech. The experiments using isolated narrow sub-bands show that the regions (0-0.77 kHz) and (3.40-11.02 kHz) are the most useful for SID, while those in the region (0.34-3.44 kHz) are best for AID. AID experiments are also performed with intersession variability compensation, which provides the biggest performance gain in the (2.23-5.25 kHz) region.

This paper presents a method for detection of sinusoidal signals corrupted by an additive noise in the short-time Fourier domain. The proposed method is based on probabilistic modelling of the spectral magnitude shape and phase continuity around spectral peaks and can deal with both stationary and non-stationary sinusoidal signals. Experimental results are presented for both sinusoidal signals of a constant frequency and frequency varying continuously over time. The performance is analysed in terms of the false acceptance and false rejection error rates of spectral peak detection and also compared to our previous method. Experimental results demonstrate very high detection accuracy in even very strong noisy conditions.

In this paper we investigate an incorporation of mask modelling into an HMM-based ASR system. The mask model is estimated for each HMM state and mixture by using a separate Viterbi-style training procedure and it expresses which regions of the spectrum are expected to be uncorrupted by noise for the HMM state. Experimental evaluation is performed on noisy speech data from the Aurora 2 database. Significant performance improvements are achieved when the mask modelling is incorporated within the standard model and two models that had already compensated for the effect of the noise.

In this paper, we propose a novel model for incorporating the voicing information in a speech recognition system. The voicing information employed is estimated by a novel method that can provide this information for each filter-bank channel, without requiring any information about the fundamental frequency. A Viterbi-style training procedure is employed to estimate the voicing-probability of each mixture at each HMM state. Experiments are performed on noisy speech data from the Aurora 2 database. Significant performance improvements are achieved at low SNRs when the voicing information is incorporated within the standard model and two models that had already compensated for the effect of the noise.

This paper proposes a novel MAP denoising algorithm that uses the ICA transformation and provide a derivation demonstrating the type of situations in which the use of ICA transformation is expected to achieve best results. We also propose an employment of generalized Gaussian model (GGM) for modelling the speech and noise distributions. The performance of the proposed speech enhancement algorithm is compared with the Wiener filtering and sparse code shrinkage method. The experiments are focused on speech signal corrupted by a non-Gaussian noise. The experimental results show that the proposed algorithm achieves significantly better performance than both the Wiener filtering and the sparse code shrinkage method.

This letter presents a method for estimation of the voicing-character of speech spectra. It is based on a calculation of a similarity between the shape of the signal short-term magnitude spectra and spectra of the frame-analysis window, which is weighted by the signal magnitude spectra. It is demonstrated that the proposed voicing measure is related to the local SNR of noise-corrupted voiced speech. The performance is evaluated for detection of voiced regions in the spectra of speech corrupted by various types of noise. The experimental results in terms of false-acceptance and false-rejection show errors of less than 5% for speech corrupted by white noise at the local SNR of 10 dB and in terms of recognition accuracy obtained by an ASR system using the voicing information estimated by the proposed method and by the fulla prioriknowledge about the noise show similar recognition performance.

The probabilistic union model is a way for robust combination of multiple features with partial feature corruption. However, the calculation of the union-based probability is combinatorial with the number of features. This correspondence presents a fast algorithm for calculation of the union-based probability, which significantly reduces the computational requirements.

This paper presents a novel method for estimating the voicing-character of speech spectra, demonstrates its employment in noise robust ASR and proposes a modified calculation of filter-bank energies. The proposed voicing-character estimation is based on calculation of a similarity between the shape of the signal short-term magnitude spectra around spectral peaks and spectra of the frame-analysis window. The similarity is weighted by the signal magnitude spectra to reflect the filter-bank analysis typically used in feature extraction for speech recognition. The experimental results show less than 5% false-acceptance and false-rejection errors in detection of voiced filter-bank channels in speech signal corrupted by white noise at 10 dB local SNR. The recognition results obtained by a missing-feature based ASR system using features estimated as voiced by the proposed method are very similar to using oracle voicing-label obtained by full a-priori knowledge of noise. The employment of features obtained by a modified calculation of filter-bank energies shows further improvements in the recognition accuracy

Colorectal cancer (CRC) is one of the most common fatal cancers in developed countries and represents a significant public-health issue. About 3%-5% of patients with CRC have hereditary nonpolyposis colorectal cancer (HNPCC). Cancer morbidity and mortality can be reduced if early and intensive screening is pursued. However, despite advances in screening, population-wide genetic screening for HNPCC is not currently considered feasible due to its complexity and expense. If the risk of a family having HNPCC can be identified/assessed, then only the high-risk fraction of the population would undergo intensive screening. This identification is currently performed by a genetic counselor/physician who makes the decision based on some pre-defined criteria. Here, we report on a system to identify the risk of a family having HNPCC based on its history. We compare artificial neural networks and statistical approaches for assessing the risk of a family having HNPCC and discuss the experimental results obtained by these two approaches.

This paper is concerned with the multi-stream approach in speech recognition. In a given set of feature streams, there may be some features corrupted by noise. Ideally, these features should be excluded from recognition. To achieve this, a-priori knowledge about the identity, including both the number and location, of the noisy features is required. We present a method for estimating the number of noisy feature streams. This method assumes no knowledge about the noise. It is based on calculation of the reliability of each feature stream and then evaluation of the joint maximal reliability. Since this method decreases the uncertainty about the noisy features and is statistical in nature, it can also be used to increase robustness of other classification systems. We present an application of this method to model-order selection in the union models. We performed tests on the TIDIGITS database, corrupted by noises affecting various numbers of feature streams. The experimental results show that this model achieves recognition performance similar to the one obtained with a-priori knowledge about the identity of the corrupted features.

In this paper, we propose a combination of the probabilistic union model and the missing feature method to improve noise robustness in ASR. Specifically, the missing feature method is employed to eliminate the acoustic frames in which all frequency bands are corrupted and the union model is employed to extract useful information from the frames involving only partial frequency -band corruption. This combination enhances the capability of the union model for dealing with wide-band noise corruption, and moreover, it reduces the dependence of the missing-feature method on information about the noise - only a knowledge on the bandwidth of the noise is required. We have tested the new system based on the TIDigits database, corrupted by various types of wide-band and narrow-band noise. The experimental results show significant performance improvement by the proposed system, compared to a previous model without using the combination.

This paper introduces a new statistical approach, namely the probabilistic union model, for speech recognition involving partial, unknown frequency-band corruption. Partial frequency-band corruption accounts for the effect of a family of real-world noises. Previous methods based on the missing feature theory usually require the identity of the noisy bands. This identification can be difficult for unexpected noise with unknown, time-varying band characteristics. The new model combines the local frequency-band information based on the union of random events, to reduce the dependence of the model on information about the noise. This model partially accomplishes the target: offering robustness to partial frequency-band corruption, while requiring no information about the noise. This paper introduces the theory and implementation of the union model, and is focused on several important advances. These new developments include a new algorithm for automatic order selection, a generalization of the modeling principle to accommodate partial feature stream corruption, and a combination of the union model with conventional noise reduction techniques to deal with a mixture of stationary noise and unknown, nonstationary noise. For the evaluation, we used the TIDIGITS database for speaker-independent connected digit recognition. The utterances were corrupted by various types of additive noise, stationary or time-varying, assuming no knowledge about the noise characteristics. The results indicate that the new model offers significantly improved robustness in comparison to other models.

Real-world noise may be a mixture of known or trainable noise and unknown unexpected noise. This paper investigates the combination of the conventional noise-reduction techniques with the probabilistic union model to deal with this type of mixed noise for robust speech recognition. In particular, we have developed a multi-environment system to remove the known or trainable acoustic mismatch across different environments. The novelty of this system, in contrast to other multi-environment models, is that the acoustic model for each environment is built upon the probabilistic union model, so that this system is also capable of accommodating further unknown unexpected noise within a specific environment. We have tested the new system for connected digit recognition in different environments, each involving an environment-specific noise and some unknown untrained noise. The results indicate that the new system offers significantly improved performance for the environments involving unknown additional noise, in comparison to a baseline multi-environment system.

This paper provides a summary of our recent work on robust speech recognition based on a new statistical approach - the probabilistic union model. In particular, we considered speech recognition involving partial corruption in frequency bands, in time duration, and further in feature components. In all these situations, we assumed no prior knowledge about the corrupting noise, e.g. its band location, occurring time and statistical distribution. The new model characterizes these partial, unknown corruptions based on the union of random events. For the evaluation, we have conducted isolated-word recognition tasks by using both a speaker-independent E-set database and the TiDigits database, each being corrupted by various types of additive noise with unknown, time-varying statistics. The results indicate that the probabilistic union model offers robustness to partial corruption in speech utterances, requiring little or no knowledge about the noise characteristics.

In current speech recognition systems, speech is represented by a 2-D sequence of parameters that model the temporal evolution of the spectral envelope of speech. Linear transformation or filtering along both time and frequency axes of that 2-D sequence are used to enhance the discriminative ability and robustness of speech parameters in the HMM pattern-matching formalism. In this paper, we compared two recently reported approaches which operate on the sequence of logarithmically compressed mel-scaled filter-bank energies: the first approach - TIFFING (TIme and Frequency FilterING) - applies FIR filters to that 2-D sequence along both axes, while the second one - CTM (Cepstral Time Matrix) - uses the DCT to compute a set of parameters in the 2-D transformed domain. They are compared in several ways: (1) analytically, using Fourier transformation, (2) statistically and (3) performing recognition tests with clean and noisy speech.