Domain shift is a problem commonly encountered when developing automated histopathology pipelines. The performance of machine learning models such as convolutional neural networks within automated histopathology pipelines is often diminished when applying them to novel data domains due to factors arising from differing staining and scanning protocols. The Dual-Channel Auto-Encoder (DCAE) model was previously shown to produce feature representations that are less sensitive to appearance variation introduced by different digital slide scanners. In this work, the Multi-Channel Auto-Encoder (MCAE) model is presented as an extension to DCAE which learns from more than two domains of data. Experimental results show that the MCAE model produces feature representations that are less sensitive to inter-domain variations than the comparative StaNoSA method when tested on a novel synthetic dataset. This was apparent when applying the MCAE, DCAE, and StaNoSA models to three different classification tasks from unseen domains. The results of this experiment show the MCAE model out performs the other models. These results show that the MCAE model is able to generalise better to novel data, including data from unseen domains, than existing approaches by actively learning normalised feature representations.

Many approaches to unconstrained face identification exploit small patches which are unaffected by distortions outside their locality. A larger area usually contains more discriminative information, but may be unidentifiable due to local appearance changes across its area, given limited training data. We propose a novel block-based approach, as a complement to existing patch-based approaches, to exploit the greater discriminative information in larger areas, while maintaining robustness to limited training data. A testing block contains several neighboring patches, each of a small size. We identify the matching training block by jointly estimating all of the matching patches, as a means of reducing the uncertainty of each small matching patch with the addition of the neighboring patch information, without assuming additional training data. We further propose a multiscale extension in which we carry out block-based matching at several block sizes, to combine complementary information across scales for further robustness. We have conducted face identification experiments using three datasets, the constrained Georgia Tech dataset to validate the new approach, and two unconstrained datasets, LFW and UFI, to evaluate its potential for improving robustness. The results show that the new approach is able to significantly improve over existing patch-based face identification approaches, in the presence of uncontrolled pose, expression, and lighting variations, using small training datasets. It is also shown that the new block-based scheme can be combined with existing approaches to further improve performance.

Exemplar-based (or Corpus-based) speech enhancement algorithms have great potential but are typically slow due to needing to search through the entire corpus. The properties of speech can be exploited to improve these algorithms. Firstly, a corpus can be clustered by a phonetic ordering into a search tree which can be used to find a best matching segment. This dramatically reduces the search space, reducing the time complexity of searching a corpus of n segments from O(n) to O(log(n)). Secondly, clustering can be used to give a lossy compression of a speech corpus by replacing original segments with codewords. These techniques are shown in comparison with sequential search and non-compressed corpora using a simple speech enhancement algorithm. A combination of these techniques for a corpus of a quarter of WSJO results in a speedup of approximately 3000×.

Conventional speech enhancement methods, based on frame, multiframe, or segment estimation, require knowledge about the noise. This paper presents a new method that aims to reduce or effectively remove this requirement. It is shown that by using the zero-mean normalized correlation coefficient (ZNCC) as the comparison measure, and by extending the effective length of speech segment matching to sentence-long speech utterances, it is possible to obtain an accurate speech estimate from noise without requiring specific knowledge about the noise. The new method, thus, could be used to deal with unpredictable noise or noise without proper training data. This paper is focused on realizing and evaluating this potential. We propose a novel realization that integrates full-sentence speech correlation with clean speech recognition, formulated as a constrained maximization problem, to overcome the data sparsity problem. Then we propose an efficient implementation algorithm to solve this constrained maximization problem to produce speech sentence estimates. For evaluation, we build the new system on one training dataset and test it on two different test datasets across two databases, for a range of different noises including highly nonstationary ones. It is shown that the new approach, without any estimation of the noise, is able to significantly outperform conventional methods that use optimized noise tracking, in terms of various objective measures including automatic speech recognition.

In this paper, we introduce a novel approach to face recognition which simultaneously tackles three combined challenges: (1) uneven illumination; (2) partial occlusion; and (3) limited training data. The new approach performs lighting normalization, occlusion de-emphasis and finally face recognition, based on finding the largest matching area (LMA) at each point on the face, as opposed to traditional fixed-size local areabased approaches. Robustness is achieved with novel approaches for feature extraction, LMA-based face image comparison and unseen data modeling. On the extended YaleB and AR face databases for face identification, our method using only a single training image per person, outperforms other methods using a single training image, and matches or exceeds methods which require multiple training images. On the labeled faces in the wild face verification database, our method outperforms comparable unsupervised methods. We also show that the new method performs competitively even when the training images are corrupted.

Given the success of patch-based approaches to image denoising, this paper addresses the ill-posed problem of patch size selection. Large patch sizes improve noise robustness in the presence of good matches, but can also lead to artefacts in textured regions due to the rare patch effect; smaller patch sizes reconstruct details more accurately but risk over-fitting to the noise in uniform regions. We propose to jointly optimize each matching patch's identity and size for grayscale image denoising, and present several implementations. The new approach effectively selects the largest matching areas, subject to the constraints of the available data and noise level, to improve noise robustness. Experiments on standard test images demonstrate our approach's ability to improve on fixed-size reconstruction, particularly at high noise levels, on smoother image regions.

It is shown that under certain conditions it is possible to obtain a good speech estimate from noise without requiring noise estimation. We study an implementation of the theory, namely wide matching, for speech enhancement. The new approach performs sentence-wide joint speech segment estimation subject to maximum recognizability to gain noise robustness. Experiments have been conducted to evaluate the new approach with variable noises and SNRs from -5 dB to noise free. It is shown that the new approach, without any estimation of the noise, significantly outperformed conventional methods in the low SNR conditions while retaining comparable performance in the high SNR conditions. It is further suggested that the wide matching and deep learning approaches can be combined towards a highly robust and accurate speech estimator.

This paper presents the maximum weighted stream posterior (MWSP) model as a robust and efficient stream integration method for audio-visual speech recognition in environments, where the audio or video streams may be subjected to unknown and time-varying corruption. A significant advantage of MWSP is that it does not require any specific measurements of the signal in either stream to calculate appropriate stream weights during recognition, and as such it is modality-independent. This also means that MWSP complements and can be used alongside many of the other approaches that have been proposed in the literature for this problem. For evaluation we used the large XM2VTS database for speaker-independent audio-visual speech recognition. The extensive tests include both clean and corrupted utterances with corruption added in either/both the video and audio streams using a variety of types (e.g., MPEG-4 video compression) and levels of noise. The experiments show that this approach gives excellent performance in comparison to another well-known dynamic stream weighting approach and also compared to any fixed-weighted integration approach in both clean conditions or when noise is added to either stream. Furthermore, our experiments show that the MWSP approach dynamically selects suitable integration weights on a frame-by-frame basis according to the level of noise in the streams and also according to the naturally fluctuating relative reliability of the modalities even in clean conditions. The MWSP approach is shown to maintain robust recognition performance in all tested conditions, while requiring no prior knowledge about the type or level of noise.

This paper studies single-channel speech separation, assuming unknown, arbitrary temporal dynamics for the speech signals to be separated. A data-driven approach is described, which matches each mixed speech segment against a composite training segment to separate the underlying clean speech segments. To advance the separation accuracy, the new approach seeks and separates the longest mixed speech segments with matching composite training segments. Lengthening the mixed speech segments to match reduces the uncertainty of the constituent training segments, and hence the error of separation. For convenience, we call the new approach Composition of Longest Segments, or CLOSE. The CLOSE method includes a data-driven approach to model long-range temporal dynamics of speech signals, and a statistical approach to identify the longest mixed speech segments with matching composite training segments. Experiments are conducted on the Wall Street Journal database, for separating mixtures of two simultaneous large-vocabulary speech utterances spoken by two different speakers. The results are evaluated using various objective and subjective measures, including the challenge of large-vocabulary continuous speech recognition. It is shown that the new separation approach leads to significant improvement in all these measures.

This paper presents a novel method of audio-visual feature-level fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there are limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature representation and a modified cosine similarity are introduced to combine and compare bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal dataset created from the SPIDRE speaker recognition database and AR face recognition database with variable noise corruption of speech and occlusion in the face images. The system's speaker identification performance on the SPIDRE database, and facial identification performance on the AR database, is comparable with the literature. Combining both modalities using the new method of multimodal fusion leads to significantly improved accuracy over the unimodal systems, even when both modalities have been corrupted. The new method also shows improved identification accuracy compared with the bimodal systems based on multicondition model training or missing-feature decoding alone.

In this paper we demonstrate a simple and novel illumination model that can be used for illumination invariant facial recognition. This model requires no prior knowledge of the illumination conditions and can be used when there is only a single training image per-person. The proposed illumination model separates the effects of illumination over a small area of the face into two components; an additive component modelling the mean illumination and a multiplicative component, modelling the variance within the facial area. Illumination invariant facial recognition is performed in a piecewise manner, by splitting the face image into blocks, then normalizing the illumination within each block based on the new lighting model. The assumptions underlying this novel lighting model have been verified on the YaleB face database. We show that magnitude 2D Fourier features can be used as robust facial descriptors within the new lighting model. Using only a single training image per-person, our new method achieves high (in most cases 100%) identification accuracy on the YaleB, extended YaleB and CMU-PIE face databases.

Given single-channel recordings of simultaneous speakers, we may need to identify the individual speakers for separating their voices. In this paper, we consider the problem of identifying two simultaneous speakers based on single-channel data, i.e., speakerpair identification. We model the problem as identifying speakers using noisy speech with partial temporal corruption, which corresponds to the heavily mixed speech frames. Inclusion of these noisy frames will damage the accuracy of both speakers identification. In this paper, we propose a new approach to automatically and optimally select the single-speaker dominated speech frames for identification. The new algorithm has been evaluated using two databases: 1) the GRID speech separation database and 2) the Wall Street Journal (WSJ0) database. The new approach has shown better performance than other approaches. On the Grid database, for example, the new approach outperformed the state of the art IBM approach in 5 out of 6 test conditions.

This paper presents a novel method of audio-visual fusion for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowledge about the corruption. Furthermore, we assume there is a limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new representation and a modified cosine similarity are introduced for combining and comparing bimodal features with limited training data as well as vastly differing data rates and feature sizes. Optimal feature selection and multicondition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Experiments have been carried out on a bimodal data set created from the SPIDRE and AR databases with variable noise corruption of speech and occlusion in the face images. The new method has demonstrated improved recognition accuracy.

Temporal dynamics and speaker characteristics are two important features of speech that distinguish speech from noise. In this paper, we propose a method to maximally extract these two features of speech for speech enhancement. We demonstrate that this can reduce the requirement for prior information about the noise, which can be difficult to estimate for fast-varying noise. Given noisy speech, the new approach estimates clean speech by recognizing long segments of the clean speech as whole units. In the recognition, clean speech sentences, taken from a speech corpus, are used as examples. Matching segments are identified between the noisy sentence and the corpus sentences. The estimate is formed by using the longest matching segments found in the corpus sentences. Longer speech segments as whole units contain more distinct dynamics and richer speaker characteristics, and can be identified more accurately from noise than shorter speech segments. Therefore, estimation based on the longest recognized segments increases the noise immunity and hence the estimation accuracy. The new approach consists of a statistical model to represent up to sentence-long temporal dynamics in the corpus speech, and an algorithm to identify the longest matching segments between the noisy sentence and the corpus sentences. The algorithm is made more robust to noise uncertainty by introducing missing-feature based noise compensation into the corpus sentences. Experiments have been conducted on the TIMIT database for speech enhancement from various types of nonstationary noise including song, music, and crosstalk speech. The new approach has shown improved performance over conventional enhancement algorithms in both objective and subjective evaluations.

Most conventional techniques for noise adaptation assume a clean initial speech model which is adapted to a specific noise condition using adaptation data accumulated from the condition. In this paper, a different problem is considered, i.e. adapting a noisy speech model to a specific noise condition. For example, the initial noisy model may be a multi-condition model which is used to provide more accurate transcripts for the adaptation data than could be provided by a clean model, thereby obtaining a more accurate adaptation. We develop the formulation for this new problem by combining and extending maximum likelihood linear regression (MLLR), constrained MLLR (CMLLR) and uncertainty decoding techniques. We also present an implementation which has been tested on the Aurora 4 database, assuming an initial multi-condition model trained using white noise corrupted data. Significant word error rate (WER) reductions are achieved in comparison with other approaches.

In this paper, we present a new approach to visual speech recognition which improves contextual modelling by combining Inter-Frame Dependent and Hidden Markov Models. This approach captures contextual information in visual speech that may be lost using a Hidden Markov Model alone. We apply contextual modelling to a large speaker independent isolated digit recognition task, and compare our approach to two commonly adopted feature based techniques for incorporating speech dynamics. Results are presented from baseline feature based systems and the combined modelling technique. We illustrate that both of these techniques achieve similar levels of performance when used independently. However significant improvements in performance can be achieved through a combination of the two. In particular we report an improvement in excess of 17% relative Word Error Rate in comparison to our best baseline system.

Temporal dynamics and speaker characteristics are two important features of speech that distinguish speech from noise. In this paper, we propose a method to maximally extract these two features of speech for speech enhancement. We demonstrate that this can reduce the requirement for prior information about the noise, which can be difficult to estimate for fast-varying noise. Given noisy speech, the new approach estimates clean speech by recognizing long segments of the clean speech as whole units. In the recognition, clean speech sentences, taken from a speech corpus, are used as examples. Matching segments are identified between the noisy sentence and the corpus sentences. The estimate is formed by using the longest matching segments found in the corpus sentences. Longer speech segments as whole units contain more distinct dynamics and richer speaker characteristics, and can be identified more accurately from noise than shorter speech segments. Therefore, estimation based on the longest recognized segments increases the noise immunity and hence the estimation accuracy. The new approach consists of a statistical model to represent up to sentence-long temporal dynamics in the corpus speech, and an algorithm to identify the longest matching segments between the noisy sentence and the corpus sentences. The algorithm is made more robust to noise uncertainty by introducing missing-feature based noise compensation into the corpus sentences. Experiments have been conducted on the TIMIT database for speech enhancement from various types of nonstationary noise including song, music, and crosstalk speech. The new approach has shown improved performance over conventional enhancement algorithms in both objective and subjective evaluations.

This paper investigates the problem of face recognition with partially occluded images without assuming prior information about the distortion, and with only a single training image or a small number of training images for each class to be identified. A new approach is presented, which is an extension of our previous posterior union model. The new approach is formulated by using a similarity measure in place of the probability measure, thereby allowing the use of a single training image to represent a class. The new approach achieves improved robustness to partial occlusion by focusing the recognition mainly on the matched local regions, which are selected automatically subject to an optimality criterion to maximize the similarity of the correct class. Two databases, XM2VTS and AR, have been used to evaluate the new approach. The results indicate that the new system is able to perform as well as an oracle model for dealing with various simulated and realistic partial distortions/occlusions without requiring prior information.

This paper presents a new approach to speech modeling and recognition. The new approach consists of a statistical model to represent up to sentence-long temporal dynamics in the training data, and an algorithm to identify the matching segments with maximum continuities between the training and testing sentences. Recognition is performed by combining the longest matching segments found from the training sentences. Because of their richer and more distinct temporal dynamics, longer speech segments as whole units can be recognized with lower error rates than shorter speech segments. Therefore basing recognition on the longest matching segments optimizes the discrimination and hence recognition of speech. The new approach has been evaluated on the TIMIT database for identifying matching speech segments. The results obtained are encouraging given the very low parametric complexity of the new model.

This paper presents a new approach to face recognition where the images are subject to unknown, partial distortion/occlusion. The new approach is a probabilistic decision-based neural network (PDBNN), built on a statistical method called the posterior union model (PUM). PUM is an approach for ignoring severely mismatched local features and focusing the recognition mainly on the matched local features. It thereby improves the robustness while assuming no prior information about the corruption. We call the new approach the posterior union decision-based neural network (PUDBNN). The new PUDBNN has been evaluated on two face image databases, XM2VTS and ORL, using testing images subjected to various types of partial distortion and occlusion. The new system has demonstrated improved performance over other systems.

In the paper, a new approach for modeling and matching long-range dependencies in free-text speech data is proposed for speaker recognition. The new approach consists of a sentence model to detail up to sentence-level dependencies in the training data, and a search algorithm that is capable of locating the matches of arbitrary-length segments between the training and testing sentences. The search algorithm is optimized to increase the probability for the match of long, continuous segments as opposed to short, separated segments, assuming that long, continuous segments contain more specific information about the speaker. The new approach has been evaluated on the NIST 1998 Speaker Recognition Evaluation database, and has shown improved performance.

TEST 02 - Elsevier's Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.

This paper investigates the problem of speaker identification and verification in noisy conditions, assuming that speech signals are corrupted by environmental noise, but knowledge about the noise characteristics is not available. This research is motivated in part by the potential application of speaker recognition technologies on handheld devices or the Internet. While the technologies promise an additional biometric layer of security to protect the user, the practical implementation of such systems faces many challenges. One of these is environmental noise. Due to the mobile nature of such systems, the noise sources can be highly time-varying and potentially unknown. This raises the requirement for noise robustness in the absence of information about the noise. This paper describes a method that combines multicondition model training and missing-feature theory to model noise with unknown temporal-spectral characteristics. Multicondition training is conducted using simulated noisy data with limited noise variation, providing a ldquocoarserdquo compensation for the noise, and missing-feature theory is applied to refine the compensation by ignoring noise variation outside the given training conditions, thereby reducing the training and testing mismatch. This paper is focused on several issues relating to the implementation of the new model for real-world applications. These include the generation of multicondition training data to model noisy speech, the combination of different training data to optimize the recognition performance, and the reduction of the model's complexity. The new algorithm was tested using two databases with simulated and realistic noisy speech data. The first database is a redevelopment of the TIMIT database by rerecording the data in the presence of various noise types, used to test the model for speaker identification with a focus on the varieties of noise. The second database is a handheld-device database collected in realistic noisy conditions, used to further validate the model for real-world speaker verification. The new model is compared to baseline systems and is found to achieve lower error rates.

We study speaker verification for handheld devices assuming realistic, noisy test conditions and assuming no prior knowledge of the noise characteristics. Data were recorded in office ("quiet") and street intersection ("noisy") environments, with the use of an internal microphone and an external headset. We assume that the speaker models are trained using the office data and tested in matched and mismatched environment/microphone conditions. Two approaches were studied, both built upon a subband feature framework: 1) a posterior union model (PUM) that focuses verification on matching subbands thereby reducing the effect of the training and testing mismatch, and 2) universal compensation (UC) that combines multi-condition training and the PUM to provide robustness to noises of arbitrary temporal-spectral characteristics. Multi-condition training using simulated noise data of different characteristics provides a "coarse" compensation for the noise, and the PUM refines the compensation by ignoring noise variations outside the given training conditions. These two models were compared to baseline systems and have shown improved robustness for realistic noisy speech data.

This paper considers the separation and recognition of overlapped speech sentences assuming single-channel observation. A system based on a combination of several different techniques is proposed. The system uses a missing-feature approach for improving crosstalk/noise robustness, a Wiener filter for speech enhancement, hidden Markov models for speech reconstruction, and speaker-dependent/-independent modeling for speaker and speech recognition. We develop the system on the Speech Separation Challenge database, involving a task of separating and recognizing two mixing sentences without assuming advanced knowledge about the identity of the speakers nor about the signal-to-noise ratio. The paper is an extended version of a previous conference paper submitted for the challenge. 1 1

This paper investigates speech recognition involving additive background noise, assuming no knowledge about the noise characteristics. A new method, namely universal compensation (UC), is proposed as a solution to the problem. The UC method is an extension of the missing-feature method, i.e., recognition based only on reliable data but robust to any corruption type, including full corruption in which the noise affects all time-frequency components of the speech representation. The UC technique achieves robustness to unknown, full noise corruption through a novel combination of the multicondition training method and the missing-feature method. Multicondition training is employed to convert fullband spectral corruption into partial-band spectral corruption, which is achieved by training the model using data involving simulated wide-band noise at different signal-to-noise ratios. The missing-feature principle is employed to reduce the effect of the remaining partial-band corruption on recognition by basing the recognition only on the matched or compensated spectral components from the multicondition training. The combination of these two strategies makes the new method potentially capable of dealing with arbitrary additive noise-with arbitrary temporal-spectral characteristics-based only on clean speech training data and simulated noise data, without requiring knowledge of the actual noise. Two databases, Aurora 2 and an E-set word database, have been used to evaluate the UC method. Experiments on Aurora 2 indicate that the new model has the potential to achieve a recognition performance close to the performance obtained by a multicondition baseline model trained using data involving the test environments. Further experiments for noise conditions unseen in Aurora 2 show significant performance improvement for the new model over the multicondition model. The experimental results on the E-set database demonstrate the ability of the UC model to deal with acoustically confusing recognition tasks.

We consider speaker identification involving background noise, assuming no knowledge about the noise characteristics. A new method, namely universal compensation (UC), is studied as a solution to the problem. The UC method is an extension of the missing-feature method, i.e. recognition based only on reliable data but robust to any corruption type, including full corruption that affects all time-frequency components of the speech representation. The UC technique achieves robustness to unknown, full noise corruption through a novel combination of the multi-condition training method and the missing-feature method. The combination of these two strategies makes the new method potentially capable of dealing with arbitrary additive noise - with arbitrary temporal-spectral characteristics - based only on clean speech training data and simulated noise data, without requiring knowledge about the actual noise. The SPIDRE database is used for the evaluation, assuming various corruptions from real-world noise data. The results obtained are encouraging.

TEST 02 - Elsevier's Scopus, the largest abstract and citation database of peer-reviewed literature. Search and access research from the science, technology, medicine, social sciences and arts and humanities fields.

This paper investigates the effect of modeling subband correlation for noisy speech recognition. Subband feature streams are assumed to be independent in many subband-based speech recognition systems. However, speech recognition experimental results suggest this assumption is unrealistic. In this paper, a method is proposed to incorporate correlation into subband speech feature streams. In the proposed method, all possible combinations of subbands are created and each combination is treated as a single frequency-band by calculating a single feature vector for it. The resulting feature vectors, therefore, capture information about every band in the combination, as well as the dependency across the bands. Although using the new features results in a higher computational complexity, our experimental results show that they effectively capture the correlation between the subbands while making minimal assumptions about the structure of the correlation. Experiments are conducted on the TIDigits database. The results demonstrate improved accuracy for clean speech recognition and improved robustness in the presence of both stationary and nonstationary band-selective noise, in comparison to a system assuming subband independence.

The paper investigates the effect of modeling sub-band correlation for noisy speech recognition. Sub-band data streams are assumed to be independent in many sub-band based speech recognition systems. However, the structure and operation of the human vocal tract suggests this assumption is unrealistic. A novel method is proposed to incorporate correlation into sub-band speech feature streams. All possible combinations of sub-bands are created and each combination is treated as a single frequency band by calculating a single feature vector for it. The resulting feature vectors capture information about every band in the combination as well as the dependency across the bands. Experiments conducted on the TIDigits database demonstrate significantly improved robustness in comparison to an independent sub-band system in the presence of both stationary and non-stationary noise.

We aim to develop an acoustic model for noisy speech recognition that is "trained once, suits all", in terms of offering a recognition performance close to the matched training-testing condition performance based only on clean speech training data. This paper describes such a method termed universal compensation (UC), for its ability to accommodate arbitrary additive noise without assuming any knowledge about the noise. The new UC method consists of two parts: 1) converting full-band spectral corruption into partial-band spectral corruption through compensations for simulated wide-band flat-spectrum noise at consecutive SNRs (signal-to-noise ratios); and 2) reducing the effect of the remaining partial frequency-band corruption on recognition by ignoring the severely mismatched spectral components and basing the recognition mainly on the matched or appropriately compensated spectral components. Experiments on Aurora 2 indicate that the new model, trained from clean data, has achieved a performance comparable to the performance obtained by the baseline system trained on multi-condition data; experiments with noises unseen in Aurora 2 have shown significant improvement for the new model over the baseline model with multi-condition training.

A new method, namely universal compensation (UC), is introduced for speech recognition involving additive noise assuming no knowledge about the noise. The UC method involves a novel combination of the principle of multi-condition training and the principle of the missing-feature method. Multicondition training is employed to convert full-band spectral corruption into partial-band spectral corruption through compensations for simulated wide-band noise, and the missing-feature principle is employed to reduce the effect of the remaining partial-band corruption on recognition by basing the recognition mainly on the matched or appropriately compensated spectral components. This combination makes the new method potentially capable of dealing with any additive noise – with arbitrary temporal-spectral characteristics – based only on clean speech training data and simulated noise data, without requiring knowledge about the noise. This paper describes the evaluation of the UC method on Aurora 2 and 3 and further, on noise conditions unseen in the Aurora tasks. The results show that the new model assuming no knowledge of noise has performed equally well as the baseline models trained for the specific tasks. The new model has outperformed the baseline when there exists a mismatch between the training and testing conditions.

This paper investigates speech recognition with partial feature corruption, assuming unknown, time-varying noise characteristics. We extend our previous probabilistic union model from a conditional-probability formulation to a posterior-probability formulation. The new formulation allows the order of the model to be optimized for every single frame, and therefore greatly enhances the capability of the model for dealing with nonstationary noise corruption. Experiments have been conducted on two databases: TIDigits with noise corruption and Aurora 2, to demonstrate the improved robustness for the new model. Examples are presented showing that the new model can co-exist with existing noise-reduction techniques to provide improved noise robustness.

The research on the basic characteristics of complex system has been done in this paper, in which brittleness as one of them is presented. Meanwhile, according to the theory of set pairing, concepts of brittle link function, brittle link entropy and basic element are defined. Finally, on the basis of data from the country from April to May, taking Beijing, and Neimenggu as examples, analysis have been carried out on the severe acute respiratory syndrome (SARS) based on the theory of brittleness. The results of simulation show that brittle entropy increase and brittle entropy of subsystems tend to ascend at the beginning of SARS interference. After subsystems obtain the negative entropy from the environment, the entropy will be decreased. Effective methods are the key factor of controlling the spread of SARS.

In this paper, we propose a combination of the probabilistic union model and the missing feature method to improve noise robustness in ASR. Specifically, the missing feature method is employed to eliminate the acoustic frames in which all frequency bands are corrupted and the union model is employed to extract useful information from the frames involving only partial frequency -band corruption. This combination enhances the capability of the union model for dealing with wide-band noise corruption, and moreover, it reduces the dependence of the missing-feature method on information about the noise - only a knowledge on the bandwidth of the noise is required. We have tested the new system based on the TIDigits database, corrupted by various types of wide-band and narrow-band noise. The experimental results show significant performance improvement by the proposed system, compared to a previous model without using the combination.

This paper introduces a new statistical approach, namely the probabilistic union model, for speech recognition involving partial, unknown frequency-band corruption. Partial frequency-band corruption accounts for the effect of a family of real-world noises. Previous methods based on the missing feature theory usually require the identity of the noisy bands. This identification can be difficult for unexpected noise with unknown, time-varying band characteristics. The new model combines the local frequency-band information based on the union of random events, to reduce the dependence of the model on information about the noise. This model partially accomplishes the target: offering robustness to partial frequency-band corruption, while requiring no information about the noise. This paper introduces the theory and implementation of the union model, and is focused on several important advances. These new developments include a new algorithm for automatic order selection, a generalization of the modeling principle to accommodate partial feature stream corruption, and a combination of the union model with conventional noise reduction techniques to deal with a mixture of stationary noise and unknown, nonstationary noise. For the evaluation, we used the TIDIGITS database for speaker-independent connected digit recognition. The utterances were corrupted by various types of additive noise, stationary or time-varying, assuming no knowledge about the noise characteristics. The results indicate that the new model offers significantly improved robustness in comparison to other models.

Real-world noise may be a mixture of known or trainable noise and unknown unexpected noise. This paper investigates the combination of the conventional noise-reduction techniques with the probabilistic union model to deal with this type of mixed noise for robust speech recognition. In particular, we have developed a multi-environment system to remove the known or trainable acoustic mismatch across different environments. The novelty of this system, in contrast to other multi-environment models, is that the acoustic model for each environment is built upon the probabilistic union model, so that this system is also capable of accommodating further unknown unexpected noise within a specific environment. We have tested the new system for connected digit recognition in different environments, each involving an environment-specific noise and some unknown untrained noise. The results indicate that the new system offers significantly improved performance for the environments involving unknown additional noise, in comparison to a baseline multi-environment system.

We present a new method of introducing domain knowledge into an n-gram language model. It is based on a combination of language models for individual word domains. Each word model is built from an individual corpus which is formed by extracting those subsets of the entire training corpus which contain that significant word. When testing, significant words are extracted from a cache and their models are combined with a global language model. Different methods of combining the models are described; one simple method based on combining frequencies rather than probabilities gives promising results and provides a relatively simple method of introducing domain information into an ngram language model. A 20% reduction in language model perplexity over the standard 3-gram approach is obtained which is similar to results obtained with other more complex domain models. The model also requires a small cache compared with other models requiring a cache.

A new statistical model is proposed for combining sub-band observations for robust speech recognition. This model characterizes the randomly corrupted sub-bands based on the union of random events. The new model has been incorporated into a hidden Markov model (HMM) and tested for recognizing a speaker-independent E-set, corrupted by various types of frequency-selective noise with unknown, time-varying statistics. The results show that the new model offers robustness to partial frequency corruption, requiring little or no knowledge about the noise characteristics.

Although the discrete cosine transform (DCT) is widely used for feature extraction in pattern recognition, it is shown that it converges slowly for most theoretically smooth functions. A modification of the DCT is described, based on a change of variable, which changes it to a new transform, called the discrete Chebyshev transform (DChT), which converges very rapidly for the same smooth functions. Although this rapid convergence is largely destroyed by the noise in real experimental data, the discrete Chebyshev transform is still generally better than the DCT when the sampling of the data can be selected at nonequidistant points. The improvement over the DCT gives a theoretical explanation for improved speech recognition obtained using Mel feature cepstral coefficients. These choose the sampling frequencies of a DCT to correspond to the human perception of pitch. It is shown that this sampling is similar to the sampling used in the discrete Chebyshev transform.

This paper provides a summary of our recent work on robust speech recognition based on a new statistical approach - the probabilistic union model. In particular, we considered speech recognition involving partial corruption in frequency bands, in time duration, and further in feature components. In all these situations, we assumed no prior knowledge about the corrupting noise, e.g. its band location, occurring time and statistical distribution. The new model characterizes these partial, unknown corruptions based on the union of random events. For the evaluation, we have conducted isolated-word recognition tasks by using both a speaker-independent E-set database and the TiDigits database, each being corrupted by various types of additive noise with unknown, time-varying statistics. The results indicate that the probabilistic union model offers robustness to partial corruption in speech utterances, requiring little or no knowledge about the noise characteristics.

Most current speech recognition systems are built upon a single type of model, e.g. an HMM or certain type of segment based model, and furthermore typically employs only one type of acoustic feature e.g. MFCCs and their variants. This entails that the system may not be robust should the modeling assumptions be violated. Recent research efforts have investigated the use of multi-scale/multi-band acoustic features for robust speech recognition. This paper described a multi-model approach as an alternative and complement to the multi-feature approaches. The multi-model approach seeks a combination of different types of acoustic models, thereby integrating the capabilities of each individual model for capturing discriminative information. An example system built upon the combination of the standard HMM technique with a segment-based modeling technique was implemented. Experiments for both isolated-word and continuous speech recognition have shown improved performances over each of the individual models considered in isolation.

A new method of using domains to improve a variable ngram language model is presented. Previous research on the use of domains using a 2-tier approach is extended through the addition of further hierarchical levels of domain abstraction. Our method is based upon the concept of domain association where semantically related domains exhibit similar n-gram distributions. Association allows us to cluster domains into a hierarchy of larger super-domains ultimately terminating when all domains have been included into one single superdomain. The resulting hierarchy has been used within a dynamic language model framework which tracks changes in text over time. Results with several corpora show improvements of 22% over traditional modelling techniques whilst providing a robust mechanism for handling topical changes.

This paper introduces a new statistical framework for constructing triphonic models from models of less context-dependency. This composition reduces the number of models to be estimated by higher than an order of magnitude and is therefore of great significance in relieving the data sparsity problem in triphone-based continuous speech recognition. The new framework is derived from Bayesian statistics, and represents an alternative to other triphone-by-composition techniques, particularly to the model-interpolation and quasitriphone approaches. The potential power of this new framework is explored by an implementation based on the hidden Markov modeling technique. It is shown that the new model structure includes the quasitriphone model as a special case, and leads to more efficient parameter estimation than the model-interpolation method. Phone recognition experiments show an increase in the accuracy over that obtained by comparable models.

A crucial issue in triphone based continuous speech recognition is the large number of models to be estimated against the limited availability of training data. This problem can be relieved by composing a triphone model from less context-dependent models. This paper introduces a new statistical framework, derived from the Bayesian principle, to perform such a composition. The potential power of this new framework is explored, both algorithmically and experimentally, by an implementation with hidden Markov modeling techniques. This implementation is applied to the recognition of the 39-phone set on the TIMIT database. The new model achieves 74.4% and 75.6% accuracy, respectively, on the core and complete test sets.

Two methods of improving conventional n-gram statistical language models are examined. The first involves using a new set of n-gram statistics that attempt to improve the ability of a system to identify phrases correctly. The second involves replacing the maximum likelihood unigram component with an optimised distribution. We test these approaches by incorporating them into weighted average [1] and deleted estimate [2] language models trained on a large newspaper corpus. The improvements lead to a reduction in perplexity of 4.5% and 4.9% respectively for these models.

An optimized hidden Markov model (HMM) with two kinds of inter-frame dependent observation structures, both built on the observation densities of a first-order dependent form, is presented to account for the statistical dependence between successive frames. In the first model, the dependence relation among the frames is determined optimally by maximizing the likelihood of the observations in both training and testing. In the second model, the dependence structure associated with each frame is described by a weighted sum of the conditional densities of the frame given individual previous frames. The segmental K-means and the forward-backward algorithms are implemented, respectively, for the estimation of the parameters of the two models. Experimental comparisons for an isolated word recognition task show that these models achieve better performance than both the standard continuous HMM and the bigram-constrained HMM.

Approaches to noisy speech recognition are developed by considering the direct extraction of the phonetic information from the noisy speech measurement. The proposed approaches are based on a statistical representation of the noisy measurement information directly in terms of the phonetic information to be recognized. Recognition is done based on a maximum a posteriori formulation of the noisy speech recognition problem.>