Transfeminine people may undergo facial feminisation surgery, a term covering a range of procedures that aim to alter the appearance of facial features, thereby potentially changing characteristics of the vocal tract. Effects of facial feminisation surgery on the voice are relatively understudied, however, so, little information on the vocal effects of these surgeries is available to people considering undergoing these procedures. In this single-case study, we present an acoustic analysis of speech and singing data collected from a transgender singer before and after facial feminisation surgery, alongside an examination of longitudinal interview data from the participant. Our quantitative results suggest facial feminisation surgery can have an impact on the voice, and our qualitative analysis suggests this may not only be as a result of the altered characteristics of the vocal tract, but also as a result of the altered social context. Several issues for future research are identified.

Despite there being clear evidence for attentional effects in biological spatial hearing, relatively few machine hearing systems exploit attention in binaural sound localisation. This paper addresses this issue by proposing a novel binaural machine hearing system with temporal attention for robust localisation of sound sources in noisy and reverberant conditions. A convolutional neural network is employed to extract noise-robust localisation features, which are similar to interaural phase difference, directly from phase spectra of the left and right ears for each frame. A temporal attention layer operates on top of these frame-level features by incorporating outputs of a temporal mask estimation module that indicate target dominance within each frame. The combined features are then exploited by fully connected layers, which map them to the corresponding source azimuth. Both the temporal mask estimation module and the sound localisation module are trained jointly in a multi-task learning manner. Our evaluation shows that the proposed system is able to accurately estimate the azimuth of a sound source in various reverberant and noisy conditions.

Obstructive sleep apnea (OSA) is a chronic and prevalent condition with well-established comorbidities. Due to limited diagnostic resources and high cost, a significant OSA population lives undiagnosed, and accurate and low-cost methods to screen for OSA are needed. We propose a novel screening method based on breathing sounds recorded with a smartphone and respiratory effort. Whole night recordings are divided into 30-s segments, each of which is classified for the presence or absence of OSA events by a multimodal deep neural network. Data fusion techniques were investigated and evaluated based on the apnea-hypopnea index estimated from whole night recordings. Real-world recordings made during home sleep apnea testing from 103 participants were used to develop and evaluate the proposed system. The late fusion system achieved the best sensitivity and specificity when screening for severe OSA, at 0.93 and 0.92, respectively. This offers the prospect of inexpensive OSA screening at home.

Obstructive sleep apnea (OSA) is a chronic and prevalent condition with well-established comorbidities. However, many severe cases remain undiagnosed due to poor access to polysomnography (PSG), the gold standard for Obstructive sleep apnea (OSA) diagnosis. Accurate home-based methods to screen for OSA are needed, which can be applied inexpensively to high-risk subjects to identify those that require PSG to fully assess their condition. A number of methods that analyse speech or breathing sounds to screen for OSA have been previously investigated. However, these methods have constraints that limit their use in home environments (e.g., they require specialised equipment, are not robust to background noise, are obtrusive or depend on tightly controlled conditions). This paper proposes a novel method to screen for OSA, which analyses sleep breathing sounds recorded with a smartphone at home. Audio recordings made over a whole night are divided into segments, each of which is classified for the presence or absence of OSA by a deep neural network. The apnea-hypopnea index estimated from the segments predicted as containing evidence of OSA is then used to screen for the condition. Audio recordings made during home sleep apnea testing from 103 participants for 1 or 2 nights were used to develop and evaluate the proposed system. When screening for moderate OSA the acoustics based system achieved a sensitivity of 0.79 and a specificity of 0.80. The sensitivity and specificity when screening for severe OSA were 0.78 and 0.93, respectively. The system is suitable for implementation on consumer smartphones.

Acoustic analysis of sleep breathing sounds using a smartphone at home provides a much less obtrusive means of screening for sleep-disordered breathing (SDB) than assessment in a sleep clinic. However, application in a home environment is confounded by the problem that a bed partner may also be present and snore. This paper proposes a novel acoustic analysis system for snorer diarisation, a concept extrapolated from speaker diarisation research, which allows screening for SDB of both the user and the bed partner using a single smartphone. The snorer diarisation system involves three steps. First, a deep neural network (DNN) is employed to estimate the number of concurrent snorers in short segments of monaural audio recordings. Second, the identified snore segments are clustered using snorer embeddings, a feature representation that allows different snorers to be discriminated. Finally, a snore transcription is automatically generated for each snorer by combining consecutive snore segments. The system is evaluated on both synthetic snore mixtures and real two-snorer recordings. The results show that it is possible to accurately screen a subject and their bed partner for SDB in the same session from recordings of a single smartphone.

Sleep-disordered breathing (SDB) is a serious and prevalent condition, and acoustic analysis via consumer devices (e.g. smartphones) offers a low-cost solution to screening for it. We present a novel approach for the acoustic identification of SDB sounds, such as snoring, using bottleneck features learned from a corpus of whole-night sound recordings. Two types of bottleneck features are described, obtained by applying a deep autoencoder to the output of an auditory model or a short-term autocorrelation analysis. We investigate two architectures for snore sound detection: a tandem system and a hybrid system. In both cases, a `language model' (LM) was incorporated to exploit information about the sequence of different SDB events. Our results show that the proposed bottleneck features give better performance than conventional mel-frequency cepstral coefficients, and that the tandem system outperforms the hybrid system given the limited amount of labelled training data available. The LM made a small improvement to the performance of both classifiers.

A novel end-to-end binaural sound localisation approach is proposed which estimates the azimuth of a sound source directly from the waveform. Instead of employing hand-crafted features commonly employed for binaural sound localisation, such as the interaural time and level difference, our end-to-end system approach uses a convolutional neural network (CNN) to extract specific features from the waveform that are suitable for localisation. Two systems are proposed which differ in the initial frequency analysis stage. The first system is auditory-inspired and makes use of a gammatone filtering layer, while the second system is fully data-driven and exploits a trainable convolutional layer to perform frequency analysis. In both systems, a set of dedicated convolutional kernels are then employed to search for specific localisation cues, which are coupled with a localisation stage using fully connected layers. Localisation experiments using binaural simulation in both anechoic and reverberant environments show that the proposed systems outperform a state-of-the-art deep neural network system. Furthermore, our investigation of the frequency analysis stage in the second system suggests that the CNN is able to exploit different frequency bands for localisation according to the characteristics of the reverberant environment.

Graphical User Interfaces (GUIs) are amongst the most common user interfaces, enabling interactions with applications through mouse movements and key presses. Tools for automated testing of programs through their GUI exist, however they usually rely on operating system or framework specific knowledge to interact with an application. Due to frequent operating system updates, which can remove required information, and a large variety of different GUI frameworks using unique underlying data structures, such tools rapidly become obsolete, Consequently, for an automated GUI test generation tool, supporting many frameworks and operating systems is impractical. We propose a technique for improving GUI testing by automatically identifying GUI widgets in screen shots using machine learning techniques. As training data, we generate randomized GUIs to automatically extract widget information. The resulting model provides guidance to GUI testing tools in environments not currently supported by deriving GUI widget information from screen shots only. In our experiments, we found that identifying GUI widgets in screen shots and using this information to guide random testing achieved a significantly higher branch coverage in 18 of 20 applications, with an average increase of 42.5% when compared to conventional random testing.

Despite there being a clear evidence for top-down (e.g., attentional) effects in biological spatial hearing, relatively few machine hearing systems exploit the top-down model-based knowledge in sound localization. This paper addresses this issue by proposing a novel framework for the binaural sound localization that combines the model-based information about the spectral characteristics of sound sources and deep neural networks (DNNs). A target source model and a background source model are first estimated during a training phase using spectral features extracted from sound signals in isolation. When the identity of the background source is not available, a universal background model can be used. During testing, the source models are used jointly to explain the mixed observations and improve the localization process by selectively weighting source azimuth posteriors output by a DNN-based localization system. To address the possible mismatch between the training and testing, a model adaptation process is further employed the on-the-fly during testing, which adapts the background model parameters directly from the noisy observations in an iterative manner. The proposed system, therefore, combines the model-based and data-driven information flow within a single computational framework. The evaluation task involved localization of a target speech source in the presence of an interfering source and room reverberation. Our experiments show that by exploiting the model-based information in this way, the sound localization performance can be improved substantially under various noisy and reverberant conditions.

This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for robust binaural localization of multiple sources in reverberant environments. DNNs are used to learn the relationship between the source azimuth and binaural cues, consisting of the complete cross-correlation function (CCF) and interaural level differences (ILDs). In contrast to many previous binaural hearing systems, the proposed approach is not restricted to localization of sound sources in the frontal hemifield. Due to the similarity of binaural cues in the frontal and rear hemifields, front-back confusions often occur. To address this, a head movement strategy is incorporated in the localization model to help reduce the front-back errors. The proposed DNN system is compared to a Gaussian-mixture-model-based system that employs interaural time differences (ITDs) and ILDs as localization features. Our experiments show that the DNN is able to exploit information in the CCF that is not available in the ITD cue, which together with head movements substantially improves localization accuracies under challenging acoustic scenarios, in which multiple talkers and room reverberation are present.

Automatic speech recognition (ASR) has become a widespread and convenient mode of human-machine interaction, but it is still not sufficiently reliable when used under highly noisy or reverberant conditions. One option for achieving far greater robustness is to include another modality that is unaffected by acoustic noise, such as video information. Currently the most successful approaches for such audiovisual ASR systems, coupled hidden Markov models (HMMs) and turbo decoding, both allow for slight asynchrony between audio and video features, and significantly improve recognition rates in this way. However, both typically still neglect residual errors in the estimation of audio features, so-called observation uncertainties. This paper compares two strategies for adding these observation uncertainties into the decoder, and shows that significant recognition rate improvements are achievable for both coupled HMMs and turbo decoding.

Speech source localization (SSL) using a microphone array \naims to estimate the direction-of-arrival (DOA) of the speech \nsource. However, its performance often degrades rapidly in reverberant \nenvironments. In this paper, a novel dual-microphone \nSSL algorithm is proposed to address this problem. First, the \ntime-frequency regions dominated by direct sound are extracted \nby tracking the envelopes of speech, reverberation and background \nnoise. The time-difference-of-arrival (TDOA) is then \nestimated by considering only these reliable regions. Second, \na bin-wise de-aliasing strategy is introduced to make better use \nof the DOA information carried at high frequencies, where the \nspatial resolution is higher and there is typically less corruption \nby diffuse noise. Our experiments show that when compared \nwith other widely-used algorithms, the proposed algorithm produces \nmore reliable performance in realistic reverberant environments.

Speech localisation in multitalker mixtures is affected by the listener’s expectations about the spatial arrangement of the sound sources. This effect was investigated via experiments with human listeners and a machine system, in which the task was to localise a female-voice target among four spatially distributed male-voice maskers. Two configurations were used: either the masker locations were fixed or the locations varied from trial-to-trial. The machine system uses deep neural networks (DNNs) to learn the relationship between binaural cues and source azimuth, and exploits top-down knowledge about the spectral characteristics of the target source. Performance was examined in both anechoic and reverberant conditions. Our experiments show that the machine system outperformed listeners in some conditions. Both the machine and listeners were able to make use of a priori knowledge about the spatial configuration of the sources, but the effect for headphone listening was smaller than that previously reported for listening in a real room.

This paper presents a novel system that exploits synchrony spectra and deep neural networks (DNNs) for automatic speech recognition (ASR) in challenging noisy environments. Synchrony spectra measure the extent to which each frequency channel in an auditory model is entrained to a particular pitch period, and they are used together with F0 estimates either in a DNN for time-frequency (T-M) mask estimation or to augment the input features for a DNN-based ASR system. The proposed approach was evaluated in the context of the CHiME 3 Challenge. Our experiments show that the synchrony spectra features work best when augmenting the input features to the DNN-based ASR system. Compared to the CHiME-3 baseline system, our best system provides a word error rate (WER) reduction of more than 14% absolute and achieved a WER of 18.56% on the evaluation test set.

This paper is concerned with machine localisation of multiple active speech sources in reverberant environments using two (binaural) microphones. Such conditions typically present a problem for `classical' binaural models. Inspired by the human ability to utilise head movements, the current study investigated the influence of different head movement strategies on binaural sound localisation. A machine-hearing system that exploits a multi-step head rotation strategy for sound localisation was found to produce the best performance in simulated reverberant acoustic space. This paper also reports the public release of a free binaural room impulse responses (BRIRs) database that allows the simulation of head rotation used in this study.

This paper addresses the problem of localising multiple competing speakers in the presence of room reverberation, where sound sources can be positioned at any azimuth on the horizontal plane. To reduce the amount of front-back confusions which can occur due to the similarity of interaural time differences (ITDs) and interaural level differences (ILDs) in the front and rear hemifield, a machine hearing system is presented which combines supervised learning of binaural cues using multi-conditional training (MCT) with a head movement strategy. A systematic evaluation showed that this approach substantially reduced the amount of front-back confusions in challenging acoustic scenarios. Moreover, the system was able to generalise to a variety of different acoustic conditions not seen during training.

Relatively few systems for machine hearing exploit top-down \ninformation in source localisation, despite there being clear \nevidence for top-down (e.g., attentional) effects in biological \nspatial hearing. This paper addresses this issue by proposing \na framework for binaural sound localisation that exploits top- \ndown knowledge about the source spectral characteristics in the \nacoustic scene. Information from source models is used to im- \nprove the localisation process by selectively weighting binaural \ncues. The system therefore combines top-down and bottom- \nup information flow within a single computational framework. \nOur experiments show that by exploiting source models in this \nway, sound localisation performance can be improved substan- \ntially under challenging conditions in which multiple sources \nand room reverberation are present.

This paper presents a novel machine-hearing system that exploits deep neural networks (DNNs) and head movements for binaural localisation of multiple speakers in reverberant conditions.DNNs are used to map binaural features, consisting of the complete crosscorrelation function (CCF) and interaural level differences (ILDs), to the source azimuth.Our approach was evaluated using a localisation task in which sources were located in a full 360-degree azimuth range.As a result, front-back confusions often occurred due to the similarity of binaural features in the front and rear hemifields.To address this, a head movement strategy was incorporated in the DNN-based model to help reduce the front-back errors.Our experiments show that, compared to a system based on a Gaussian mixture model (GMM) classifier, the proposed DNN system substantially reduces localisation errors under challenging acoustic scenarios in which multiple speakers and room reverberation are present.

Automated test generation can effectively explore programs through their programmer interfaces and traditional graphical user interfaces, but the recent advent of natural user interfaces (NUI) based on motion and gesture detection, for example the Microsoft Kinect, has outrun software testing research. This leaves a rapidly growing domain of software ranging from entertainment to medical applications without suitable test automation techniques. To address this issue, we propose a technique that automatically tests Kinect-based applications by synthesising realistic sequences of skeletal movement. The novel test cases are generated by a statistical model, which is trained on a corpus of common gestures. Evaluation on a gesture-controlled Kinect web browser application demonstrates that our approach achieves significantly higher code coverage than random test inputs.

A nonlinear auditory model is appraised in terms of its ability to encode speech formant frequencies in the fine time structure of its output. It is demonstrated that groups of model auditory nerve (AN) fibres with similar interpeak intervals accurately encode the resonances of synthetic three-formant syllables, in close agreement with physiological data. Acoustic features are derived from the interpeak intervals and used as the input to a hidden Markov model-based automatic speech recognition system. In a digits-in-noise recognition task, interval-based features gave a better performance than features based on AN firing rate at every signal-to-noise ratio tested.

Multitasking during teleconferences is becoming increasingly common: participants continue their work whilst monitoring the audio for topics of interest. Our previous work has estab-lished the benefit of spatialised audio presentation on improv-ing multitasking performance. In this study, we investigate the different spatialisation strategies employed by subjects in order to aid their multitasking performance and improve their user experience. Subjects were given the freedom to place each par-ticipant at a different location in the acoustic space both in terms of azimuth and distance. Their strategies were based upon cues regarding keywords and which participant will utter them. Our findings suggest that subjects employ consistent strategies with regard to the location of target and distracter talkers. Further-more, manipulation of the acoustic space plays an important role in multitasking performance and the user experience. Index Terms: multitasking, spatialisation, teleconference 1.

Teleconference participants often multitask: they work on a text-based ‘foreground’ task whilst listening in the ‘background’ for an item of interest to appear. Audio material should therefore be presented in a manner that has the smallest possible impact on the foreground task without affecting topic detection. Here, we ask whether dichotic or spatialised audio presentation of a meeting is less disruptive than the single-channel mixture of talkers normally used in teleconference audio. A number of talker location configurations are used, and we examine how these impact upon a text-based foreground task: finding all letter ‘e’ occurrences in a block of text. Additionally, we examine the effect of cueing the listener to direction or gender and record listener preferences for audio presentation style. Our results suggest that spatialised audio disrupts the foreground task less than single-channel audio when direction or gender is cued.

A novel extension to recurrent timing neural networks (RTNNs) is proposed which allows such networks to exploit a joint interaural time difference-fundamental fre-quency (ITD-F0) auditory cue as opposed to F0 only. This extension involves coupling a second layer of coincidence detectors to a two-dimensional RTNN. The coincidence detectors are tuned to particular ITDs and each feeds excitation to a column in the RTNN. Thus, one axis of the RTNN represents F0 and the other ITD. The resulting behaviour allows sources to be segregated on the basis of their separation in ITD-F0 space. Furthermore, all grouping and segregation activity proceeds within individual frequency channels without recourse to across channel estimates of F0 or ITD that are commonly used in auditory scene analysis ap-proaches. The system has been evaluated using a source separation task operating on spatialised speech signals. 1

A computational auditory scene analysis (CASA) system is described, in which sound separation according to spatial location is combined with the 'missing data' approach for automatic speech recognition. Time-frequency masks for the missing data recognizer are derived from the statistics of interaural time and level differences; these masks identify acoustic features that constitute reliable evidence of the target speech signal. It is demonstrated that this approach yields good performance in a challenging environment, in which a target voice is contaminated by another talker and reverberation. The ability of the system to generalize to source-receiver configurations that were not encountered during training is discussed

We describe a novel approach to feature combination within the missing data (MD) framework for automatic speech recognition, and show its application to reverberated speech. Likelihoods from a spectral MD classifier are combined with those from a full cepstral feature vector-based recogniser. Even though the performance of the cepstral recogniser is substantially below that of the MD recogniser, the combined recogniser performs better in all conditions. We also describe improvements to the generation of time-frequency masks for the MD recogniser. Our system is compared with a previous approach based on a hybrid MLP-HMM recogniser with MSG and PLP feature vectors. The proposed system has a substantial performance advantage in the most reverberated conditions

This paper describes a perceptually motivated computational auditory scene analysis (CASA) system that combines sound separation according to spatial location with the "missing data" approach for robust speech recognition in noise. Missing data time-frequency masks are created using probability distributions based on estimates of interaural time and level differences (ITD and ILD) for mixed utterances in reverberated conditions; these masks indicate which regions of the spectrum constitute reliable evidence of the target speech signal. A number of experiments compare the relative efficacy of the binaural cues when used individually and in combination. We also investigate the ability of the system to generalize to acoustic conditions not encountered during training. Performance on a continuous digit recognition task using this method is found to be good, even in a particularly challenging environment with three concurrent male talkers.

This paper describes a perceptually motivated computational auditory scene analysis (CASA) system that combines sound separation according to spatial location with 'missing data' techniques for robust speech recognition in noise. Missing data time-frequency masks are produced using cross-correlation to estimate interaural time difference (ITD) and hence spatial azimuth; this is used to determine which regions of the signal constitute reliable evidence of the target speech signal. Three experiments are performed that compare the effects of different reverberation surfaces, localisation methods and azimuth separations on recognition accuracy, together with the effects of two post-processing techniques (morphological operations and supervised learning) for improving mask estimation. Both post-processing techniques greatly improve performance; the best performance occurs using a learned mapping.

Recent psychophysical studies suggest that human listeners do not segregate concurrent sounds by grouping frequency regions that have a common interaural time difference (ITD). However, such an approach is adopted by most computational auditory scene analysis (CASA) systems that use binaural cues. Here, we propose a CASA system that separates a target speech signal from a noise interferer, but does not require the ITD of the two sources to be consistent across frequency. We compare the CASA system with human performance on the same task, in which the speech reception threshold (SRT) is measured for speech and noise stimuli which have consistent or inconsistent ITDs in different frequency bands. The CASA system is shown to be in qualitative agreement with human performance.

The ‘missing data ’ approach for robust speech recognition uses masks indicating which regions of an acoustic mixture provide reliable evidence of the target to be recognised. Binaural cues for spatial location were used to determine missing data masks for signals consisting of utterances from three concurrent male speakers in reverberant conditions, by deriving probability distributions from estimates of interaural time and level differences (ITD and ILD) for the mixed signals. In such a system, a decision must be made about whether the acoustic features used for decoding are selected from the left or right ear, or a combination of the two. Here, features were selected from the “better ear ” (as determined by a simple heuristic) within whole time frames, or within individual time-frequency elements. A combination of left and right ear features gave better recognition performance than using either ear alone, and the best results were obtained when selecting features within individual time-frequency elements. 1.

An audio-visual localisation and tracking system for meeting scenarios is presented which draws its inspiration from neurobiological processing. Meetings are recorded by a KEMAR binaural manikin and a single camera placed directly above the manikin. Source localisation from the binaural audio and face, object and motion locations from the video frames are used as input to two linked neural oscillator networks. The strength of the connections between the two networks determines the mapping between activity at a particular audio azimuth and activity at a particular visual frame column. A Hebbian learning rule is used to establish the connection strengths. The combined network segments the video and audio features and then produces audio-visual groupings on the basis of common spatial location. The audio-visual groupings are tracked through time using a mechanism based upon that of the human oculomotor system which incorporates smooth pursuit and saccadic movement.

The analysis of scenarios in which a number of microphones record the activity of speakers, such as in a round-table meeting, presents a number of computational challenges. For example, if each participant wears a microphone, speech from both the microphone's wearer (local speech) and from other participants (crosstalk) is received. The recorded audio can be broadly classified in four ways: local speech, crosstalk plus local speech, crosstalk alone and silence. We describe two experiments related to the automatic classification of audio into these four classes. The first experiment attempted to optimize a set of acoustic features for use with a Gaussian mixture model (GMM) classifier. A large set of potential acoustic features were considered, some of which have been employed in previous studies. The best-performing features were found to be kurtosis, "fundamentalness," and cross-correlation metrics. The second experiment used these features to train an ergodic hidden Markov model classifier. Tests performed on a large corpus of recorded meetings show classification accuracies of up to 96%, and automatic speech recognition performance close to that obtained using ground truth segmentation.

A system for musical instrument recognition is introduced. In contrast to most existing systems, it can identify a solo instrument even in the presence of an accompanying keyboard instrument or orchestra. To enable recognition in the presence of a highly polyphonic background, we use features based solely on the partials of the target tone. The approach is based on the assumption that it is possible to extract the most prominent fundamental frequency and the corresponding harmonic overtone series, and that these most often belong to the solo instrument. Classification is carried out using a Gaussian classifier trained on examples of monophonic music. Testing our system on accompanied sonatas and concertos we achieved a recognition rate of 86% for 5 different instruments, an accuracy comparable to that of systems limited to monophonic music only.

The human auditory system is able to separate acoustic mixtures in order to create a perceptual description of each sound source. It has been proposed that this is achieved by an auditory scene analysis (ASA) in which a mixture of sounds is parsed to give a number of perceptual streams, each of which describes a single sound source. It is widely assumed that ASA is a precursor of attentional mechanisms, which select a stream for attentional focus. However, recent studies suggest that attention plays a key role in the formation of auditory streams. Motivated by these findings, this paper presents a conceptual framework for auditory selective attention in which the formation of groups and streams is heavily influenced by conscious and subconscious attention. This framework is implemented as a computational model comprising a network of neural oscillators, which perform stream segregation on the basis of oscillatory correlation. Within the network, attentional interest is modeled as a Gaussian distribution in frequency. This determines the connection weights between oscillators and the attentional process, which is modeled as an attentional leaky integrator (ALI). Acoustic features are held to be the subject of attention if their oscillatory activity coincides temporally with a peak in the ALI activity. The output of the model is an "attentional stream," which encodes the frequency bands in the attentional focus at each epoch. The model successfully simulates a range of psychophysical phenomena.

Gaussian mixture model (GMM) classifiers have been shown to give good instrument recognition performance for monophonic music played by a single instrument. However, many applications (such as automatic music transcription) require instrument identification from polyphonic, multi-instrumental recordings. We address this problem by incorporating ideas from missing feature theory into a GMM classifier. Specifically, frequency regions that are dominated by energy from an interfering tone are marked as unreliable and excluded from the classification process. This approach has been evaluated on random two-tone chords and an excerpt from a commercially available compact disc, with promising results.

An extension to the conventional speech / nonspeech classification framework is presented for a scenario in which a number of microphones record the activity of speakers present at a meeting (one microphone per speaker). Since each microphone can receive speech from both the participant wearing the microphone (local speech) and other participants (crosstalk), the recorded audio can be broadly classified in four ways: local speech, crosstalk plus local speech, crosstalk alone and silence. We describe a classifier in which a Gaussian mixture model (GMM) is used to model each class. A large set of potential acoustic features are considered, some of which have been employed in previous speech / nonspeech classifiers. A combination of two feature selection algorithms is used to identify the optimal feature set for each class. Results from the GMM classifier using the selected features are superior to those of a previously published approach. 1.

Guy J. Brown Department of Computer Science University of Sheffield 211 Portobello Street Sheffield, S1 4DP, UK [email protected]

An effective multipitch tracking algorithm for noisy speech is critical for acoustic signal processing. However, the performance of existing algorithms is not satisfactory. We present a robust algorithm for multipitch tracking of noisy speech. Our approach integrates an improved channel and peak selection method, a new method for extracting periodicity information across different channels, and a hidden Markov model (HMM) for forming continuous pitch tracks. The resulting algorithm can reliably track single and double pitch tracks in a noisy environment. We suggest a pitch error measure for the multipitch situation. The proposed algorithm is evaluated on a database of speech utterances mixed with various types of interference. Quantitative comparisons show that our algorithm significantly outperforms existing ones.

In this study we describe an auditory processing front-end for missing data speech recognition, which is robust in the presence of reverberation. The model attempts to identify time-frequency regions that are not badly contaminated by reverberation and have strong speech energy. This is achieved by applying reverberation masking. Subsequently, reliable time-frequency regions are passed to a ‘missing data’ speech recogniser for classification. We demonstrate that the model improves recognition performance in three different virtual rooms where reverberation time T60 varies from 0.7 sec to 2.7 sec. We also discuss the advantages of our approach over RASTA and modulation filtered spectrograms.

At a cocktail party, we can selectively attend to a single voice and filter out all the other acoustical interferences. How to simulate this perceptual ability remains a great challenge. This paper describes a novel location-based approach for speech segregation. The auditory masking effect motivates the notion of an “ideal” time-frequency binary mask, which selects the target if it is stronger than the interference in a local time-frequency region. We observe that within a narrow frequency band modifications to the relative energy of the target source with respect to the interfering energy trigger systematic deviations for binaural cues. For a given spatial configuration, this interaction produces characteristic clustering in the binaural feature space. Consequently, we perform pattern classification in order to estimate ideal binary masks. A systematic evaluation shows that the resulting system produces masks very close to ideal binary ones, and large improvement over previous models.

We present a robust algorithm for multi-pitch tracking of noisy speech. Our approach integrates an improved channel and peak selection method, a new integration method for extracting periodicity information across different frequency channels, and a hidden Markov model (HMM) for forming continuous pitch tracks, and as a result, our algorithm can reliably track single and double pitch tracks in a noisy environment. The proposed algorithm is evaluated on a database of speech utterances mixed with various interferences and the results show that our algorithm outperforms existing algorithms significantly.

A model of auditory grouping is described in which auditory attention plays a key role. The model is based upon an oscillatory correlation framework, in which neural oscillators representing a single perceptual stream are synchronised, and are desynchronised from oscillators representing other streams. The model suggests a mechanism by which attention can be directed to the high or low tones in a repeating sequence of tones with alternating frequencies. In addition, it simulates the perceptual segregation of a mistuned harmonic from a complex tone.

A fundamental problem in auditory and speech processing is the segregation of speech from concurrent sounds. This problem has been a focus of study in computational auditory scene analysis (CASA), and it has also been investigated from the perspective of blind source separation. Using a standard corpus of voiced speech mixed with interfering sounds, we report a comparison between CASA and blind source separation techniques, which have been developed independently. Our comparison reveals that they perform well under very different conditions. A number of conclusions are drawn with respect to their relative strengths and weaknesses in speech segregation applications as well as in modeling auditory function.

A neural model is described which uses oscillatory correlation to segregate speech from interfering sound sources. The core of the model is a two-layer neural oscillator network. The first layer of the network identifies the connected regions of energy in the time-frequency plane (segments). In the second layer, segments that have a common fundamental frequency are grouped into streams. A stream is represented by a synchronized population of relaxation oscillators, and different streams are represented by desynchronized oscillator populations. The model has been evaluated using a corpus of voiced speech mixed with interfering sounds, and produces an improvement in signal-to-noise ratio for every mixture.

Topics in speech and hearing are well-suited to demonstrations using media other than the printed word. Currently, educators rely largely on passive formats such as the CD collections for general auditory psychophysics [6], auditory scene analysis [2] and cochlear damage [10]. Progress in programming tools and cheap, multimedia hardware now presents the potential to go much further. The interactive auditory demonstrations project aims to provide the user with an environment in which to explore the many phenomena and processes associated with speech and hearing. This promotes a much richer space of parameter manipulation than is possible via passive media. Further, the ability to initiate actions, repeat procedures and benefit from practically any kind of multimodal feedback enables a much wider range of learning possibilities. This paper focusses on the interface issues which are revealed by interactive exploration of the domain. A demonstration of linear prediction is presented to illustrate these issues.

A multistage neural model is proposed for an auditory scene analysis task--segregating speech from interfering sound sources. The core of the model is a two-layer oscillator network that performs stream segregation on the basis of oscillatory correlation. In the oscillatory correlation framework, a stream is represented by a population of synchronized relaxation oscillators, each of which corresponds to an auditory feature, and different streams are represented by desynchronized oscillator populations. Lateral connections between oscillators encode harmonicity, and proximity in frequency and time. Prior to the oscillator network are a model of the auditory periphery and a stage in which mid-level auditory representations are formed. The model has been systematically evaluated using a corpus of voiced speech mixed with interfering sounds, and produces improvements in terms of signal-to-noise ratio for every mixture. The performance of our model is compared with other studies on computational auditory scene analysis. A number of issues including biological plausibility and real-time implementation are also discussed.

This paper describes a computational model of auditory rhythm perception, and demonstrates its application to the extraction of prosodic information from spoken language. The model consists of three stages. In the first stage, the speech waveform is processed by a simulation of the auditory periphery. Secondly, the output of the auditory periphery is processed by a multiscale filtering mechanism, analogous to a short-term auditory memory. Finally, peaks in the response of the multiscale mechanism are accumulated in a long-term auditory store, and plotted to give a representation referred to as a rhythmogram. It is demonstrated that there is a close relationship between the rhythmogram of an utterance and its corresponding stress hierarchy derived by phonological analysis. I. INTRODUCTION Recently, the use of prosody has attracted attention as a means of improving the performance of systems for automatic understanding of spoken language. Prosodic information can assist in the segmenta...