This paper proposes a novel facial image representation Block-based Local Contrast Patterns (BLCP) for illumination-robust face recognition. This method is based on an effective texture descriptor local contrast patterns (LCP). We use the directed and undirected difference masks to calculate three types of local intensity contrasts: directed, undirected, and maximum difference responses. These response images are divided into several nonoverlapping blocks. In each block these responses are quantized and encoded into specific patterns. A joint histogram of these patterns is computed for each block and then we concatenate all the blocks' histograms into an enhanced feature vector to be used as a face descriptor. The experimental results on Extended Yale-B and FERET databases illustrate the effectiveness of our proposed method in illumination-robust face recognition.

In this paper, a novel classifier, called superimposed sparse parameter (SSP) classifier is proposed for face recognition. SSP is motivated by two phase test sample sparse representation (TPTSSR) and linear regression classification (LRC), which can be treated as the extended of sparse representation classification (SRC). SRC uses all the train samples to produce the sparse representation vector for classification. The LRC, which can be interpreted as L2-norm sparse representation, uses the distances between the test sample and the class subspaces for classification. TPTSSR is also L2-norm sparse representation and uses two phase to compute the distance for classification. Instead of the distances, the SSP classifier employs the SSPs, which can be expressed as the sum of the linear regression parameters of each class in iterations, is used for face classification. Further, the fast SSP (FSSP) classifier is also suggested to reduce the computation cost. A mass of experiments on Georgia Tech face database, ORL face database, CVL face database, AR face database, and CASIA face database are used to evaluate the proposed algorithms. The experimental results demonstrate that the proposed methods achieve better recognition rate than the LRC, SRC, collaborative representation-based classification, regularized robust coding, relaxed collaborative representation, support vector machine, and TPTSSR for face recognition under various conditions.

This paper proposes a novel face image descriptor called local contourlet combined patterns (LCCP), based on the Non-Subsampled Contourlet Transform (NSCT), for face recognition. NSCT is a multiresolution analysis tool and can capture image information at multiple scales, orientations, and frequency bands. To adapt to the NSCT filter bank, a new encoding method named mean-based contrast patterns (MCP) is presented. We apply LBP and MCP to different levels' NSCT coefficient images respectively and then combine them to obtain a robust representation. Futhermore, block-based kernel Fisher linear discriminant (BKFLD) is used to select the most discriminative feature sets. Face recognition experiments on FERET database demonstrate the effectiveness of our proposed approach.

Recently, the linear regression classification (LRC) has been proposed. LRC uses linear regression representation of each class for classification. Based on LRC, a few improved methods are proposed, such as kernel linear regression classification (KLRC), kernel ridge regression classification (KRRC) and nearest regularized subspace (NRS). Motivated by the LRC, KLRC, KRRC and NRS, center-based weighted kernel linear regression (CWKLR) is proposed for image classification in this paper. CWKLR firstly constitutes the center-based kernel matrix based on the KLRC and KRRC. Next, CWKLR utilizes the Tikhonov Matrix to solve the weighted projection coefficients for classification. Experiments on Coil100 object database, Eth80 object database and GT face database are used to evaluate the proposed algorithms. The experimental results demonstrate that the proposed methods achieve better recognition rate than some state-of-the-art methods.

Bleed-through effect is one of the most common degradations in old documents, even in today's newspapers. This effect must be removed for hancing human and automatic readability. The two images scanned from the recto and verso pages of a document can be treated as a linear combination of clean text images from two sides. In this paper, we introduce a nonnegative least-correlation approach to demix bleed-through text images. Experiments have been conducted on both synthetic and real world images. In synthetic case, our method can recover the source text images exactly. Under real world conditions, our approach also performs well. In addition, our approach is computationally efficient and does not need any postprocessing task.

This paper proposes a novel face image descriptor local surface pattern (LSP) for illumination-robust face recognition. It is assumed that the discrete array of pixel values comes about by sampling an underlying smooth surface on the domain of the image. The proposed method efficiently estimates the underlying local surface information, which is approximately represented as linear projection coefficients of the pixels in a local patch. Thus, by filtering local image patches using the polynomial filters and binarizing the filter responses via thresholding, the method can compute a binary code for each pixel in the face image. Then the distribution of the code over suitable image regions is used for face representation. Furthermore, we prove that applying zero-mean filters in logdomain may enable the responses to be more robust to illumination variations. The experimental results on Extended Yale-B and FERET fc databases illustrate the effectiveness of our proposed method in illumination-robust face recognition.

This paper introduces a nonlinear vector-based feature mapping approach to extract robust features for automatic speech recognition (ASR) of overlapping speech using a microphone array. We explore different configurations and additional sources of information to improve the effectiveness of the feature mapping. First, we investigate the full-vector based mapping of different sources in a log mel-filterbank energy (log MFBE) domain, and demonstrate that retraining the acoustic model using the generated training data can help improve the recognition performance. Then we investigate the feature mapping between different domains. Finally in order to improve the qualities of the mapping inputs we propose a nonlinear mapping of the features from multiple beamformed sources, which are directed at the target and interfering speakers, respectively. We demonstrate the effectiveness of the proposed approach through extensive evaluations on the MONC corpus, which includes non-overlapping single speaker and overlapping multi-speaker conditions.

In this paper we address reducing the mismatch between training and testing conditions for robust distant-talking speech recognition under realistic reverberant environments. It is well known that the distortions caused by reverberation, background noise, etc., are highly nonlinear in the cepstral domain. In this paper we propose to capture the complex relationships between clean and reverberant speech via joint dictionary learning. Given a test reverberant speech with a sequence of feature vectors we first find their sparse representations, and then estimate the underlying clean feature vectors using the dictionary of clean speech. Based on speech recognition experiments conducted under realistic reverberation conditions, the proposed method is shown to perform very well, resulting in an average relative improvement of 59.1% compared with the baseline front-ends.

Recent research has shown that collaborative representation-based classifier (CRC) can lead to promising results for the classification of face images. However, CRC is conducted in the original image space rather than the nonlinear high dimensional feature space in which features belonging to the same class are better grouped together and thus can be easily separable. To address this problem, this paper presents a novel classifier, Kernel Collaborative Representation-based Classifier (KCRC), by incorporating the kernel trick into the framework of CRC. Extensive experiments on both the AT&T and the FERET face databases demonstrate the priority of KCRC to CRC and several state-of-the-art methods.

We address reducing the mismatch between training and testing conditions for hands-free in-car speech recognition. It is well known that the distortions caused by background noise, channel effects, etc., are highly nonlinear in the log-spectral or cepstral domain. This letter introduces a joint sparse representation (JSR) to estimate the underlying clean feature vector from a noisy feature vector. Performing a joint dictionary learning by sharing the same representation coefficients, the proposed method intends to capture the complex relationships (or mapping functions) between clean and noisy speech. Speech recognition experiments on realistic in-car data demonstrate that the proposed method shows excellent recognition performance with a relative improvement of 39.4% compared with the “baseline” frontends.

The log-energy parameter, typically derived from a full-band spectrum, is a critical feature commonly used in automatic speech recognition (ASR) systems. However, log-energy is difficult to estimate reliably in the presence of background noise. In this paper, we theoretically show that background noise affects the trajectories of not only the “conventional” log-energy, but also its delta parameters. This results in a poor estimation of the actual log-energy and its delta parameters, which no longer describe the speech signal. We thus propose a new method to estimate log-energy from a sub-band spectrum, followed by dynamic change enhancement and mean smoothing. We demonstrate the effectiveness of the proposed log-energy estimation and its post-processing steps through speech recognition experiments conducted on the in-car CENSREC-2 database. The proposed log-energy (together with its corresponding delta parameters) yields an average improvement of 32.8% compared with the baseline front-ends. Moreover, it is also shown that further improvement can be achieved by incorporating the new Mel-Frequency Cepstral Coefficients (MFCCs) obtained by non-linear spectral contrast stretching.

Face representation, including both feature extraction and feature selection, is the key issue for a successful face recognition system. In this paper, we propose a novel face representation scheme based on nonsubsampled contourlet transform (NSCT) and block-based kernel Fisher linear discriminant (BKFLD). NSCT is a newly developed multiresolution analysis tool and has the ability to extract both intrinsic geometrical structure and directional information in images, which implies its discriminative potential for effective feature extraction of face images. By encoding the the NSCT coefficient images with the local binary pattern (LBP) operator, we could obtain a robust feature set. Furthermore, kernel Fisher linear discriminant is introduced to select the most discriminative feature sets, and the block-based scheme is incorporated to address the small sample size problem. Face recognition experiments on FERET database demonstrate the effectiveness of our proposed approach.

Feature extraction is vital for a successful face recognition system. In this paper, we propose a computationally efficient, discriminative and robust feature descriptor for face images, named Patterns of Weber magnitude and orientation (PWMO), which encodes Weber magnitude and orientation with patch-based local binary pattern (p-LBP) and patch-based local XOR pattern (p-LXP), respectively. Furthermore, whitened PCA is introduced to reduce the feature dimensionality and select the most discriminative feature sets, and the block-based scheme is incorporated to address the small sample size problem. The effectiveness and robustness of our proposed approach has been demonstrated experimentally on the well-known FERET database.

Log energy and its delta parameters, typically derived from full-band spectrum, are commonly used in automatic speech recognition (ASR) systems.In this paper, we address the problem of estimating log energy in the presence of background noise (usually resulting in a reduction in dynamic ranges of spectral energies).We theoretically show that the background noise affects the trajectories of the "conventional" log energy and its delta parameters, resulting in very poor estimation of the actual log energy and its delta parameters, which no longer describe the speech signal.We thus propose to estimate log energy from the sub-band spectrum, followed by a dynamic range stretching.Based on speech recognition experiments conducted on CENSREC-2 in-car database, the proposed log energy (and its corresponding delta parameters) is shown to perform very well, resulting in an average relative improvement of 27.2% compared with the baseline front-ends.Moreover, it is also shown that further improvement can be achieved by incorporating those new MFCCs obtained through non-linear spectral contrast stretching.

Age progression and other sources of variations such as head pose change have adverse effects on the performance of face verification systems. In this paper, we propose to manage the influence of both the age progression and head pose change on the adult face verification system by a general Q-stack age modeling technique, which uses the age and head pose as class-independent quality measures together with the scores from baseline classifiers, in order to obtain better verification performance. This allows for improved long-term class separation by introducing a 2D parameterized decision boundary in the scores-age space using a short-term enrollment model. This new method, based on the concept of classifier stacking with age- and head pose aware decision boundary compares favorably with the conventional face verification approach, which uses age- and head-pose-independent decision threshold calculated only in the score space at the time of enrollment. In this paper, we also show the advantages of user-specific approach for the face verification task over user-independent approach. The proposed approach is evaluated on the MORPH database.

Weber's law suggests that for a stimulus, the ratio between the smallest perceptual change and the background is a constant, which implies stimuli are perceived not in absolute terms but in relative terms. Inspired from this, we exploit and analyze a novel illumination insensitive representation of face images under varying illuminations via a ratio image, called “Weber-face,” where a ratio between local intensity variation and the background is computed. Experimental results on both CMU-PIE and Yale B face databases show that Weber-face performs better than the existing representative approaches.

The problem of time validity of biometric models has received only a marginal attention from researchers. Actual and up-to-date at the time of their creation, extracted features and models relevant to a person's face may eventually become outdated, leading to a failure in the face identification task. If physical characteristics of the individual change over time, their classification model has to be updated. In this paper we present a mutli-classifier A-stack scheme, which is based on the concept of classifier stacking and makes use of the age information and scores of multiple baseline classifiers, in order to improve the identification performance during age progression. Our experiments on the MORPH database show that the use of the proposed multi-classifier stacking fusion allows for improving the identification accuracy as opposed to the baseline classifier and single-classifier A-stack method.

The influence of age progression on the performance of multi-classifier face verification systems is a challenging and largely open research problem that deserves more and more attention. In this paper, we propose to manage the aging influence on the adult face verification system by a multi-classifier Q-stack age modeling technique, which uses the age as a class-independent metadata quality measure together with scores from baseline classifiers, combining global and local patterns, in order to obtain better recognition rates. This allows for improved long-term class separation by introducing a 2D parameterized decision boundary in the scores-age space using a short-term enrollment model. This new method, based on the concept of classifier stacking and age-dependent decision boundary, compares favorably with the conventional face verification approach, which uses age-independent decision threshold calculated only in the score space at the time of enrollment. The proposed approach is evaluated on the MORPH database.

This paper investigates a non-linear mapping approach to extract robust features for ASR and speech separation of overlapping speech. Based on our previous studies, we continue to use two additional sound sources, namely from the target and interfering speakers. The focuses of this work are: 1) We investigate the feature mapping between different domains with the consideration of MMSE criterion and regression optimizations, demonstrating the mapping of log melfilterbank energies to MFCC can be exploited to improve the effectiveness of the regression; 2) We investigate the data-driven filtering for the speech separation by using the mapping method, which can be viewed as a generalized log spectral subtraction and results in better separation performance. We demonstrate the effectiveness of the proposed approach through extensive evaluations on the MONC corpus, which includes both non-overlapping single speaker and overlapping multi-speaker conditions.

In this paper, we address a beamforming application based on the capture of far-field speech data from a single speaker in a real meeting room. After the position of the speaker is estimated by a speaker tracking system, we construct a subband-domain beamformer in generalized sidelobe canceller (GSC) configuration. In contrast to conventional practice, we then optimize the active weight vectors of the GSC so as to obtain an output signal with maximum negentropy (MN). This implies the beamformer output should be as non-Gaussian as possible. For calculating negentropy, we consider the Gamma and the generalized Gaussian (GG) pdfs. After MN beamforming, Zelinski postfiltering is performed to further enhance the speech by removing residual noise. Our beamforming algorithm can suppress noise and reverberation without the signal cancellation problems encountered in the conventional beamforming algorithms. We demonstrate this fact through a set of acoustic simulations. Moreover, we show the effectiveness of our proposed technique through a series of far-field automatic speech recognition experiments on the Multi-Channel Wall Street Journal Audio Visual Corpus (MC-WSJ-AV), a corpus of data captured with real far-field sensors, in a realistic acoustic environment, and spoken by real speakers. On the MC-WSJ-AV evaluation data, the delay-and-sum beamformer with postfiltering achieved a word error rate (WER) of 16.5%. MN beamforming with the Gamma pdf achieved a 15.8% WER, which was further reduced to 13.2% with the GG pdf, whereas the simple delay-and-sum beamformer provided a WER of 17.8%. To the best of our knowledge, no lower error rates at present have been reported in the literature on this automatic speech recognition (ASR) task.

This paper presents new filter bank design methods for sub- band adaptive beamforming. In this work, we design analysis and synthesis prototypes for modulated filter banks so as to minimize each aliasing term individually. We then drive the total response error to null by constraining these prototypes to be Nyquist(M) filters. Thereafter those modulated filter banks are applied to a speech separation system which extracts a target speech signal. In our system, speech signals are first transformed into the subband domain with our filter banks, and the subband components are then processed with a beamforming algorithm. Following beamforming, post-filtering and binary masking are further performed to remove residual noises. We show that our filter banks can suppress the residual aliasing distortion more than conventional ones. Furthermore, we demonstrate the effectiveness of our design techniques through a set of automatic speech recognition experiments on the multi-channel speech data from the PASCAL Speech Separation Challenge. The experimental results prove that our beamforming system with the proposed filter banks achieves the best recognition performance, a 39.6 % word error rate (WER), with half the amount of computation of that of the conventional filter banks while the perfect reconstruction filter banks provided a 44.4 % WER.

This paper presents an adaptive beamforming application based on the capture of far-field speech data from a real single speaker in a real meeting room. After the position of a speaker is estimated by a speaker tracking system, we construct a subbanddomain beamformer in generalized sidelobe canceller (GSC) configuration. In contrast to conventional practice, we then optimize the active weight vectors of the GSC so that the distribution of an output signal is as non-Gaussian as possible. We consider kurtosis in order to measure the degree of non-Gaussianity. Our beamforming algorithms can suppress noise and reverberation without the signal cancellation problems encountered in conventional beamforming algorithms. We demonstrate the effectiveness of our proposed techniques through a series of farfield automatic speech recognition experiments on the Multi-Channel Wall Street Journal Audio Visual Corpus (MC-WSJ-AV). The beamforming algorithm proposed here achieved a 13.6 % WER, whereas the simple delay-and-sum beamformer provided a WER of 17.8%. Index Terms: far-field speech recognition, microphone array, beamforming 1.

This paper investigates a neural network based acoustic feature mapping to extract robust features for automatic speech recognition (ASR) of overlapping speech. In our preliminary studies, we trained neural networks to learn the mapping from log mel filter bank energies (MFBEs) extracted from the distant microphone recordings, including multiple overlapping speakers, to log MFBEs extracted from the clean speech signal. In this paper, we explore the mapping of higher order mel-filterbank cepstral coefficients (MFCC) to lower order coefficients. We also investigate the mapping of features from both target and interfering distant sound sources to the clean target features. This is achieved by using the microphone array to extract features from both the direction of the target and interfering sound sources. We demonstrate the effectiveness of the proposed approach through extensive evaluations on the MONC corpus, which includes both non-overlapping single speaker and overlapping multi-speaker conditions.

In this paper, we present a novel feature normalization method in the log-scaled spectral domain for improving the noise robustness of speech recognition front-ends. In the proposed scheme, a non-linear contrast stretching is added to the outputs of log mel-filterbanks (MFB) to imitate the adaptation of the auditory system under adverse conditions. This is followed by a two-dimensional filter to smooth out the processing artifacts. The proposed MFCC front-ends perform remarkably well on CENSREC-2 in-car database with an average relative improvement of 29.3\\% compared to baseline MFCC system. It is also confirmed that the proposed processing in log MFB domain can be integrated with conventional cepstral post-processing techniques to yield further improvements. The proposed algorithm is simple and requires only a small extra computation load.

We address issues for improving hands-free speech recognition performance in different car environments using a single distant microphone. In our previous work, we proposed a regression based enhancement method for in-car speech recognition. In this paper, we describe recent improvements and propose a data-driven adaptive regression based speech recognition system, in which both feature enhancement and model compensation are performed. Based on isolated word recognition experiments conducted in 15 real car environments, the proposed adaptive regression approach shows an advantage in average relative word error rate (WER) reductions of 52.5% and 14.8%, compared to original noisy speech and ETSI advanced front-end, respectively.

We present a two-stage noise spectra estimation approach. After the first-stage noise estimation using the improved minima controlled recursive averaging (IMCRA) method, the second-stage noise estimation is performed by employing a maximum a posteriori (MAP) noise amplitude estimator. We also develop a regression-based speech enhancement system by approximating the clean speech with the estimated noise and the original noisy speech. Evaluation experiments show that the proposed two-stage noise estimation method results in lower estimation error for all test noise types. Compared to the original noisy speech, the proposed regression-based approach obtains an average relative word error rate (WER) reduction of 65% in our isolated word recognition experiments conducted in 12 real car environments.

This letter addresses issues in improving hands-free speech recognition performance in different car environments. We propose a new speech-enhancement approach based on optimizing regression of the log-spectra, which is used to estimate the log-spectra of speech at a close-talking microphone by using multiple spatially distributed microphones. The regression weights can be adapted automatically for different noise environments. Compared to the nearest distant microphone and adaptive beamformer generalized sidelobe canceller (GSC), the proposed approach shows an advantage in the average relative word error rate (WER) reduction of 58.5 and 10.3%, respectively, for isolated word recognition under 15 real-car environments.

In this paper, we address issues in improving handsfree speech recognition performance in different car environments using multiple spatially distributed microphones. In previous work, we proposed multiple regression of the log-spectra (MRLS) for estimating the logspectra of speech at a close-talking microphone. In this paper, the idea is extended to nonlinear regressions. Isolated word recognition experiments under real car environments show that, compared to the nearest distant microphone, recognition accuracies could be improved by about 40% for very noisy driving conditions by using the optimizing regression method, The proposed approach outperforms linear regression methods and also outperforms adaptive beamformer by 8% and 3% respectively in terms of averaged recognition accuracies.

In this paper, we propose a speech enhancement method based on spectral magnitude estimation. We modify the noise estimation from the minimum statistics method and combine with a maximum a posterior (MAP) decomposition, using the Rice-conditional probability and a non-Gaussian statistic model of the speech. We derive two versions of magnitude decomposition and magnitude-phase decomposition and compare to spectral subtraction and other MAP methods based on the Gaussian statistic (MMSE, LSA). The experiments show the advantage of the proposed method in the improvement of both SNR (up to 12 dB) and recognition accuracy rate (up to 21 % to base line).