A study of rendering natural instrument sounds and vocals as an ensemble is presented. Using a single channel musical recording, a multi-channel signal is created by introducing delays and rendered in spatially distributed loudspeaker array. Three listeners participate in a listening experiment. We observed that the perceived width of the ensemble varies with the signal timbre. We find that spectro-temporal properties of the musical instruments play a significant role in the perception of source width. On an average, sustained sounds (like vocal) create wider percept than transients like percussion. Also, low pitched sounds (like male bass voice, saxophone, tuba) create wider percept than high pitched signals.

Multi-channel linear prediction (MCLP) can model the late reverberation in the short-time Fourier transform domain using a delayed linear predictor and the prediction residual is taken as the desired early reflection component. Traditionally, a Gaussian source model with time-dependent precision (inverse of variance) is considered for the desired signal. In this paper, we propose a Student's t-distribution model for the desired signal, which is realized as a Gaussian source with a Gamma distributed precision. Further, since the choice of a proper MCLP order is critical, we also incorporate a Gaussian distribution prior for the prediction coefficients and a higher order. We consider a batch estimation scenario and develop variational Bayes expectation maximization (VBEM) algorithm for joint posterior inference and hyper-parameter estimation. This has lead to more accurate and robust estimation of the late reverb component and hence its cancellation, benefitting the desired residual signal estimation. Along with these stochastic models, we formulate single channel output (MISO) and multi channel output (MIMO) schemes using shared priors for the desired signal precision and the estimated MCLP coefficients at each microphone. Experiments using real room impulse responses show improved late reverberation suppression with the proposed VBEM approach over the traditional methods, for different room conditions. Additionally, we achieve a sparse coefficient vector for the MCLP avoiding the criticality of manually choosing the model order. The MIMO formulation is easily extended to include spatial filtering of the enhanced signals, which further improves the estimation of the desired signal.

Multiple source localization using time-differences of arrival (TDOAs) is challenging because of the ambiguity involved in associating the TDOAs computed across microphone pairs to the sources. We show that the association ambiguity of the TDOAs can be effectively resolved using the concept of an inverse delay interval region (IDIR), which we introduce in this paper. By examining the association between a spatial domain and the TDOAs, we define IDIR as an interhyperboloidal spatial region corresponding to an interval of delays for a given pair of microphones. The proposed scheme for localizing multiple sources involves two stages. In the first stage, the given enclosure is partitioned into nonoverlapping elemental regions and the ones that contain a source are detected using a measure based on the generalized cross-correlation with phase transform and the IDIRs. In the second stage, the sources are finely localized within each of the detected elemental regions by identifying the IDIRs containing a single source and a novel region-constrained localization approach. We evaluate the performance of the proposed approach on real recordings from the AV16.3 corpus and in a simulated reverberation setting with a reverberation time RT60 of up to 500 ms, and show that the DOA estimation error with two active speakers is within 2° and the spatial localization error is less than 30 cm for each speaker.

We consider the joint estimation of time-varying linear prediction (TVLP) filter coefficients and the excitation signal parameters for the analysis of long-term speech segments. Traditional approaches to TVLP estimation assume linear expansion of the coefficients in a set of known basis functions only. But, excitation signal is also time-varying, which affects the estimation of TVLP filter parameters. In this letter, we propose a Bayesian approach, to incorporate the nature of excitation signal and also adapt regularization of the filter parameters. Since the order of the system is not known a priori, we formulate a Gaussian prior for the filter parameters, and the excitation signal is modeled as Gaussian with time-varying Gamma distributed precision. We develop an iterative algorithm for the maximum-likelihood estimation of the posterior distribution of filter parameters and the time-varying precision of the excitation signal, along with the parameters of the prior distribution. We show that the proposed method adapts to different types of excitation signals in speech, and also the time-varying system with unknown model order. The spectral modeling performance for synthetic speech-like signals, quantified using the absolute spectral difference shows that the proposed method estimates the system function more accurately compared to several of the traditional methods.

We address the problem of moving acoustic source localization and automatic camera steering using one-bit measurement of the time-difference of arrival (TDOA) between two microphones in a given array. Given that the camera has a finite field of view (FoV), an algorithm with a coarse estimate of the source location would suffice for the purpose. We use a microphone array and develop an algorithm to obtain a coarse estimate of the source using only one-bit information of the TDOA, the sign of it, to be precise. One advantage of the one-bit approach is that the computational complexity is lower, which aids in real-time adaptation and localization of the moving source. We carried out experiments in a reverberant enclosure with a 60 dB reverberation time of 600 ms (RT60 = 600 ms). We analyzed the performance of the proposed approach using a circular microphone array. We report comparisons with a point source localization based automatic camera steering algorithm proposed in the literature. The proposed algorithm turned out to be more accurate in terms of always having the moving speaker within the field of view.

In this paper, the effect of time-varying pitch-glides on timbre perception of vowels is studied experimentally. While earlier studies show that stationary pitch is observed to be a significant dimension of timbre, we explore the effect of non-stationary pitch changes on the perception of vowel timbre using synthesized vowels. In each trial, three listeners rate perceptual timbre change using a MUSHRA-like (MUltiple Stimuli with Hidden Reference and Anchor) methodology. Results show that perceived timbre change is affected by the pitch-shift and nature of pitch-glide. Timbre change is observed to be monotonic with pitch-shift for all types of pitch-glides, implying larger the pitch-shift, higher the perceived timbre change. When the excitation is a pitch-glide, the vowel spectral centroid and the nature of pitch-glide interact. We observe that for vowels of higher spectral centroid, decreasing pitch-glides cause higher timbre change than the timbre change caused by increasing pitch-glides; vice- versa for vowel of lower spectral centroid. Thus, “incoherent” pitch-glides cause higher timbre change than the “coherent” pitch-glides.

We formulate the problem of detecting the constituent instruments in a polyphonic music piece as a joint decoding problem. From monophonic data, parametric Gaussian Mixture Hidden Markov Models (GM-HMM) are obtained for each instrument. We propose a method to use the above models in a factorial framework, termed as Factorial GM-HMM (F-GM-HMM). The states are jointly inferred to explain the evolution of each instrument in the mixture observation sequence. The dependencies are decoupled using variational inference technique. We show that the joint time evolution of all instruments' states can be captured using F-GM-HMM. We compare performance of proposed method with that of Student's-t mixture model (tMM) and GM-HMM in an existing latent variable framework. Experiments on two to five polyphony with 8 instrument models trained on the RWC dataset, tested on RWC and TRIOS datasets show that F-GM-HMM gives an advantage over the other considered models in segments containing co-occurring instruments.

Estimating the parameters of moving sound sources using only the source signal is of interest in low-power, and contact-less source monitoring applications, such as, industrial robotics and bio-acoustics. The received signal embeds the motion attributes of the source via Doppler effect. In this paper, we analyze the Doppler effect on mixture of time-varying sinusoids. Focusing, on the instantaneous frequency (IF) of the received signal, we show that the IF profile composed of IF and its first two derivatives can be used to obtain source motion parameters. This requires a smooth estimate of IF profile. However, the numerical implementation of traditional approaches, such as analytic signal and energy separation approach, gives oscillatory behavior hence a non-smooth IF estimate. We devise an algorithm using non-uniformly spaced signal extrema samples of the received signal for smooth IF profile estimation. Using the smooth IF profiles for a source moving on a linear trajectory with constant velocity, an accurate estimate of moving source parameters is obtained. We see promise of this approach for an arbitrary trajectory motion parameter estimation.

Time-varying linear prediction has been studied in the context of speech signals, in which the auto-regressive (AR) coefficients of the system function are modeled as a linear combination of a set of known bases. Traditionally, least squares minimization is used for the estimation of model parameters of the system. Motivated by the sparse nature of the excitation signal for voiced sounds, we explore the time-varying linear prediction modeling of speech signals using sparsity constraints. Parameter estimation is posed as a 0-norm minimization problem. The re-weighted 1-norm minimization technique is used to estimate the model parameters. We show that for sparsely excited time-varying systems, the formulation models the underlying system function better than the least squares error minimization approach. Evaluation with synthetic and real speech examples show that the estimated model parameters track the formant trajectories closer than the least squares approach.

We address the problem of multi-instrument recognition in polyphonic music signals. Individual instruments are modeled within a stochastic framework using Student's-t Mixture Models (tMMs). We impose a mixture of these instrument models on the polyphonic signal model. No a priori knowledge is assumed about the number of instruments in the polyphony. The mixture weights are estimated in a latent variable framework from the polyphonic data using an Expectation Maximization (EM) algorithm, derived for the proposed approach. The weights are shown to indicate instrument activity. The output of the algorithm is an Instrument Activity Graph (IAG), using which, it is possible to find out the instruments that are active at a given time. An average F-ratio of 0.75 is obtained for polyphonies containing 2-5 instruments, on a experimental test set of 8 instruments: clarinet, flute, guitar, harp, mandolin, piano, trombone and violin.

We address the problem of identifying the constituent sources in a single-sensor mixture signal consisting of contributions from multiple simultaneously active sources. We propose a generic framework for mixture signal analysis based on a latent variable approach. The basic idea of the approach is to detect known sources represented as stochastic models, in a single-channel mixture signal without performing signal separation. A given mixture signal is modeled as a convex combination of known source models and the weights of the models are estimated using the mixture signal. We show experimentally that these weights indicate the presence/absence of the respective sources. The performance of the proposed approach is illustrated through mixture speech data in a reverberant enclosure. For the task of identifying the constituent speakers using data from a single microphone, the proposed approach is able to identify the dominant source with up to 8 simultaneously active background sources in a room with RT60= 250 ms, using models obtained from clean speech data for a Source to Interference Ratio (SIR) greater than 2 dB.

The classical approach to A/D conversion has been uniform sampling and we get perfect reconstruction for bandlimited signals by satisfying the Nyquist Sampling Theorem. We propose a non-uniform sampling scheme based on level crossing (LC) time information. We show stable reconstruction of bandpass signals with correct scale factor and hence a unique reconstruction from only the non-uniform time information. For reconstruction from the level crossings we make use of the sparse reconstruction based optimization by constraining the bandpass signal to be sparse in its frequency content. While overdetermined system of equations is resorted to in the literature we use an undetermined approach along with sparse reconstruction formulation. We could get a reconstruction SNR > 20dB and perfect support recovery with probability close to 1, in noise-less case and with lower probability in the noisy case. Random picking of LC from different levels over the same limited signal duration and for the same length of information, is seen to be advantageous for reconstruction.

A joint analysis-synthesis framework is developed for the compressive sensing (CS) recovery of speech signals. The signal is assumed to be sparse in the residual domain with the linear prediction filter used as the sparse transformation. Importantly this transform is not known apriori, since estimating the predictor filter requires the knowledge of the signal. Two prediction filters, one comb filter for pitch and another all pole formant filter are needed to induce maximum sparsity. An iterative method is proposed for the estimation of both the prediction filters and the signal itself. Formant prediction filter is used as the synthesis transform, while the pitch filter is used to model the periodicity in the residual excitation signal, in the analysis mode. Significant improvement in the LLR measure is seen over the previously reported formant filter estimation.

Latent variable methods, such as PLCA (Probabilistic Latent Component Analysis) have been successfully used for analysis of non-negative signal representations. In this paper, we formulate PLCS (Probabilistic Latent Component Segmentation), which models each time frame of a spectrogram as a spectral distribution. Given the signal spectrogram, the segmentation boundaries are estimated using a maximum-likelihood approach. For an efficient solution, the algorithm imposes a hard constraint that each segment is modelled by a single latent component. The hard constraint facilitates the solution of ML boundary estimation using dynamic programming. The PLCS framework does not impose a parametric assumption unlike earlier ML segmentation techniques. PLCS can be naturally extended to model coarticulation between successive phones. Experiments on the TIMIT corpus show that the proposed technique is promising compared to most state of the art speech segmentation algorithms.

We address the problem of robust formant tracking in continuous speech in the presence of additive noise. We propose a new approach based on mixture modeling of the formant contours. Our approach consists of two main steps: (i) Computation of a pyknogram based on multiband amplitude-modulation/frequency-modulation (AM/FM) decomposition of the input speech; and (ii) Statistical modeling of the pyknogram using mixture models. We experiment with both Gaussian mixture model (GMM) and Student's-t mixture model (tMM) and show that the latter is robust with respect to handling outliers in the pyknogram data, parameter selection, accuracy, and smoothness of the estimated formant contours. Experimental results on simulated data as well as noisy speech data show that the proposed tMM-based approach is also robust to additive noise. We present performance comparisons with a recently developed adaptive filterbank technique proposed in the literature and the classical Burg's spectral estimator technique, which show that the proposed technique is more robust to noise.

Using analysis-by-synthesis (AbS) approach, we develop a soft decision based switched vector quantization (VQ) method for high quality and low complexity coding of wideband speech line spectral frequency (LSF) parameters. For each switching region, a low complexity transform domain split VQ (TrSVQ) is designed. The overall rate-distortion (R/D) performance optimality of new switched quantizer is addressed in the Gaussian mixture model (GMM) based parametric framework. In the AbS approach, the reduction of quantization complexity is achieved through the use of nearest neighbor (NN) TrSVQs and splitting the transform domain vector into higher number of subvectors. Compared to the current LSF quantization methods, the new method is shown to provide competitive or better trade-off between R/D performance and complexity.

Compressive sensing (CS) has been proposed for signals with sparsity in a linear transform domain. We explore a signal dependent unknown linear transform, namely the impulse response matrix operating on a sparse excitation, as in the linear model of speech production, for recovering compressive sensed speech. Since the linear transform is signal dependent and unknown, unlike the standard CS formulation, a codebook of transfer functions is proposed in a matching pursuit (MP) framework for CS recovery. It is found that MP is efficient and effective to recover CS encoded speech as well as jointly estimate the linear model. Moderate number of CS measurements and low order sparsity estimate will result in MP converge to the same linear transform as direct VQ of the LP vector derived from the original signal. There is also high positive correlation between signal domain approximation and CS measurement domain approximation for a large variety of speech spectra.

Considering a general linear model of signal degradation, by modeling the probability density function (PDF) of the clean signal using a Gaussian mixture model (GMM) and additive noise by a Gaussian PDF, we derive the minimum mean square error (MMSE) estimator. The derived MMSE estimator is non-linear and the linear MMSE estimator is shown to be a special case. For speech signal corrupted by independent additive noise, by modeling the joint PDF of time-domain speech samples of a speech frame using a GMM, we propose a speech enhancement method based on the derived MMSE estimator. We also show that the same estimator can be used for transform-domain speech enhancement.

Segmental dynamic time warping (DTW) has been demonstrated to be a useful technique for finding acoustic similarity scores between segments of two speech utterances. Due to its high computational requirements, it had to be computed in an offline manner, limiting the applications of the technique. In this paper, we present results of parallelization of this task by distributing the workload in either a static or dynamic way on an 8-processor cluster and discuss the trade-offs among different distribution schemes. We show that online unsupervised pattern discovery using segmental DTW is plausible with as low as 8 processors. This brings the task within reach of today's general purpose multi-core servers. We also show results on a 32-processor system, and discuss factors affecting scalability of our methods.

We formulate a two-stage Iterative Wiener filtering (IWF) approach to speech enhancement, bettering the performance of constrained IWF, reported in literature. The codebook constrained IWF (CCIWF) has been shown to be effective in achieving convergence of IWF in the presence of both stationary and non-stationary noise. To this, we include a second stage of unconstrained IWF and show that the speech enhancement performance can be improved in terms of average segmental SNR (SSNR), Itakura-Saito (IS) distance and Linear Prediction Coefficients (LPC) parameter coincidence. We also explore the tradeoff between the number of CCIWF iterations and the second stage IWF iterations.

Traditional subspace based speech enhancement (SSE)methods \nuse linear minimum mean square error (LMMSE) estimation \nthat is optimal if the Karhunen Loeve transform (KLT) coefficients of speech and noise are Gaussian distributed. In this paper, we investigate the use of Gaussian mixture (GM) density for modeling the non-Gaussian statistics of the clean speech KLT coefficients. Using Gaussian mixture model (GMM), the optimum minimum mean square error (MMSE) estimator is found to be nonlinear and the traditional LMMSE estimator is shown to be a special case. Experimental results show that the proposed method provides better enhancement performance than the traditional subspace based methods.Index Terms: Subspace based speech enhancement, Gaussian mixture density, MMSE estimation.

Effective feature extraction for robust speech recognition is a widely addressed topic and currently there is much effort to invoke non-stationary signal models instead of quasi-stationary signal models leading to standard features such as LPC or MFCC. Joint amplitude modulation and frequency modulation (AM-FM) is a classical non-parametric approach to non-stationary signal modeling and recently new feature sets for automatic speech recognition (ASR) have been derived based on a multi-band AM-FM representation of the signal. We consider several of these representations and compare their performances for robust speech recognition in noise, using the AURORA-2 database. We show that FEPSTRUM representation proposed is more effective than others. We also propose an improvement to FEPSTRUM based on the Teager energy operator (TEO) and show that it can selectively outperform even FEPSTRUM

In this letter, we develop switched conditional PDF-based split vector quantization (SCSVQ) method using the recently proposed conditional PDF-based split vector quantizer (CSVQ). The use of CSVQ allows us to alleviate the coding loss by exploiting the correlation between subvectors, in each switching region. Using the Gaussian mixture model (GMM)-based parametric framework, we also address the rate-distortion (R/D) performance optimality of the proposed SCSVQ method by allocating the bits optimally among the switching regions. For the wideband speech line spectrum frequency (LSF) parameter quantization, it is shown that the optimum parametric SCSVQ method provides nearly 2 bits/vector advantage over the recently proposed nonparametric switched split vector quantization (SSVQ) method.

For vector quantization (VQ) of speech line spectrum frequency (LSF) parameters, we experimentally determine a mapping function between the mean square error (MSE) measure and the perceptually motivated average spectral distortion (SD) measure. Using the mapping function, we estimate the minimum bits/vector required for transparent quantization of telephone-band and wide-band speech LSF parameters, respectively, as 22 bits/vector and 36 bits/vector, where the distribution of LSF vector is modeled as a Gaussian mixture model (GMM).

In this letter, we develop an optimum transform domain split vector quantization (TrSVQ) method. We address both the issues of achieving best rate-distortion (R/D) performance and less complexity. For quantizing a multivariate Gaussian source, we derive the mean-square error (MSE) performance expression for the TrSVQ method using high rate theory and optimum bit allocation. Also, to reduce the complexity, we develop a binary split-based iterative algorithm and use the algorithm in a tree structured manner to find the optimum subvectors' dimensions (i.e., optimum splits).

We derive expressions for convolution multiplication properties of discrete cosine transform II (DCT II) starting from equivalent discrete Fourier transform (DFT) representations. Using these expressions, a method for implementing linear filtering through block convolution in the DCT II domain is presented. For the case of nonsymmetric impulse response, additional discrete sine transform II (DST II) is required for implementing the filter in DCT II domain, where as for a symmetric impulse response, the additional transform is not required. Comparison with recently proposed circular convolution technique in DCT II domain shows that the proposed new method is computationally more efficient.

We are addressing a new problem of improving automatic speech recognition performance, given multiple utterances of patterns from the same class. We have formulated the problem of jointly decoding K multiple patterns given a single Hidden Markov Model. It is shown that such a solution is possible by aligning the K patterns using the proposed Multi Pattern Dynamic Time Warping algorithm followed by the Constrained Multi Pattern Viterbi Algorithm. The new formulation is tested in the context of speaker independent isolated word recognition for both clean and noisy patterns. When 10 percent of speech is affected by a burst noise at −5 dB Signal to Noise Ratio (local), it is shown that joint decoding using only two noisy patterns reduces the noisy speech recognition error rate to about 51 percent, when compared to the single pattern decoding using the Viterbi Algorithm. In contrast a simple maximization of individual pattern likelihoods, provides only about 7 percent reduction in error rate.

A better performing product code vector quantization (VQ) method is proposed for coding the line spectrum frequency (LSF) parameters; the method is referred to as sequential split vector quantization (SeSVQ). The split sub-vectors of the full LSF vector are quantized in sequence and thus uses conditional distribution derived from the previous quantized sub-vectors. Unlike the traditional split vector quantization (SVQ) method, SeSVQ exploits the inter sub-vector correlation and thus provides improved rate-distortion performance, but at the expense of higher memory. We investigate the quantization performance of SeSVQ over traditional SVQ and transform domain split VQ (TrSVQ) methods. Compared to SVQ, SeSVQ saves 1 bit and nearly 3 bits, for telephone-band and wide-band speech coding applications respectively.

We develop a two stage split vector quantization method with optimum bit allocation, for achieving minimum computational complexity. This also results in much lower memory requirement than the recently proposed switched split vector quantization method. To improve the rate-distortion performance further, a region specific normalization is introduced, which results in 1 bit/vector improvement over the typical two stage split vector quantizer, for wide-band LSF quantization.

We propose a novel technique for robust voiced/unvoiced segment detection in noisy speech, based on local polynomial regression. The local polynomial model is well-suited for voiced segments in speech. The unvoiced segments are noise-like and do not exhibit any smooth structure. This property of smoothness is used for devising a new metric called the variance ratio metric, which, after thresholding, indicates the voiced/unvoiced boundaries with 75% accuracy for 0dB global signal-to-noise ratio (SNR). A novelty of our algorithm is that it processes the signal continuously, sample-by-sample rather than frame-by-frame. Simulation results on TIMIT speech database (downsampled to 8kHz) for various SNRs are presented to illustrate the performance of the new algorithm. Results indicate that the algorithm is robust even in high noise levels.

The commonly used split vector quantization (SVQ) method is inferior to unconstrained quantization due to independent coding of the split subvectors, resulting in a coding loss. In this paper, we propose a conditional pdf-based split vector quantization (CSVQ) method to recover the coding loss. The CSVQ method is developed assuming the line spectrum frequency source distribution as a multivariate Gaussian and the subvectors are quantized sequentially to exploit the correlation between the subvectors. The new CSVQ method is evaluated for wideband speech LSF quantization; CSVQ is shown to outperform traditional SVQ and provide comparable performance to the recently proposed switched split vector quantization (SSVQ) method. In addition, the transform domain SVQ method is also realized to show that its performance is limited by the distance measure used in the transform domain.

The split vector quantization (SVQ) method results in a “coding loss” due to the independent quantization of split subvectors. To recover the coding loss, conditional pdf-based split vector quantization methods were proposed recently.

Non-uniform sampling of a signal is formulated as an optimization problem which minimizes the reconstruction signal error. Dynamic programming (DP) has been used to solve this problem efficiently for a finite duration signal. Further, the optimum samples are quantized to realize a speech coder. The quantizer and the DP based optimum search for non-uniform samples (DP-NUS) can be combined in a closed-loop manner, which provides distinct advantage over the open-loop formulation. The DP-NUS formulation provides a useful control over the trade-off between bitrate and performance (reconstruction error). It is shown that 5-10 dB SNR improvement is possible using DP-NUS compared to extrema sampling approach. In addition, the close-loop DP-NUS gives a 4-5 dB improvement in reconstruction error

We present two discriminative language modelling techniques for Lempel-Ziv-Welch (LZW) based LID system. The previous approach to LID using LZW algorithm was to directly use the LZW pattern tables forlanguage modelling. But, since the patterns in a language pattern table are shared by other language pattern tables, confusability prevailed in the LID task. For overcoming this, we present two pruning techniques (i) Language Specific (LS-LZW)-in which patterns common to more than one pattern table are removed. (ii) Length-Frequency product based (LF-LZW)-in which patterns having their length-frequency product below a threshold are removed. These approaches reduce the classification score (Compression Ratio [LZW-CR] or the weighted discriminant score [LZW-WDS]) for non native languages and increases the LID performance considerably. Also the memory and computational requirements of these techniques are much less compared to basic LZW techniques.

We investigate the use of a two stage transform vector quantizer (TSTVQ) for coding of line spectral frequency (LSF) parameters in wideband speech coding. The first stage quantizer of TSTVQ, provides better matching of source distribution and the second stage quantizer provides additional coding gain through using an individual cluster specific decorrelating transform and variance normalization. Further coding gain is shown to be achieved by exploiting the slow time-varying nature of speech spectra and thus using inter-frame cluster continuity (ICC) property in the first stage of TSTVQ method. The proposed method saves 3-4 bits and reduces the computational complexity by 58-66%, compared to the traditional split vector quantizer (SVQ), but at the expense of 1.5-2.5 times of memory.

Further improvement in performance, to achieve near transparent quality LSF quantization, is shown to be possible by using a higher order two dimensional (2-D) prediction in the coefficient domain. The prediction is performed in a closed-loop manner so that the LSF reconstruction error is the same as the quantization error of the prediction residual. We show that an optimum 2-D predictor, exploiting both inter-frame and intra-frame correlations, performs better than existing predictive methods. Computationally efficient split vector quantization technique is used to implement the proposed 2-D prediction based method. We show further improvement in performance by using weighted Euclidean distance.

Speech spectral continuity is important in speech perception. We explore in this paper, the use of matrix quantization (MQ) to model spectral contours and impose time continuity in presence of noise. It is found that contours fitted over an optimum duration of 90-100 msec greatly improve speech quality. We show that Wiener filters derived from spectral contour matrices must operate in a time-varying manner and also propose a technique to achieve it through interpolation and STFT based reconstruction. In addition to conventional spectral distortion measures, we compare spectral transition measure profiles of clean and enhanced speech which indicate that MQ codebooks combined with time-varying Wiener filtering improve speech enhancement even at 0 dB SNR.

We explore new methods of determining automatically derived units for classification of speech into segments. For detecting signal changes, temporal features are more reliable than the standard feature vector domain methods, since both magnitude and phase information are retained. Motivated by auditory models, we have presented a method based on average level crossing rate (ALCR) of the signal, to detect significant temporal changes in the signal. An adaptive level allocation scheme has been used in this technique that allocates levels, depending on the signal pdf and SNR. We compare the segmentation performance to manual phonemic segmentation and also that provided by maximum likelihood (ML) segmentation for 100 TIMIT sentences. The ALCR method matches the best segmentation performance without a priori knowledge of number of segments, as in ML segmentation.

Speech and audio processing techniques are used along with statistical pattern recognition principles to solve the problem of music instrument recognition. Non-temporal, frame level features only are used so that the proposed system is scalable from the isolated notes to the solo instrumental phrases scenario without the need for temporal segmentation of solo music. Based on their effectiveness in speech, line spectral frequencies (LSF) are proposed as features for music instrument recognition. The proposed system has also been evaluated using MFCC and LPCC features. Gaussian mixture models and K-nearest neighbour model classifier are used for classification. The experimental dataset included the Ulowa MIS and the C Music corporation RWC databases. Our best results at the instrument family level is about 95% and at the instrument level is about 90% when classifying 14 instruments.

Segment vocoders play a special role in very low bitrate speech coding to achieve intelligible speech at bitrates of /spl sim/300 bits/sec. We explore the definition and use of automatically derived units for segment quantization in segment vocoders. We consider three automatic segmentation techniques, namely, spectral transition measures (STM), maximum-likelihood (ML) segmentation (unconstrained) and duration-constrained ML segmentation, towards defining diphone-like and phone-like units. We show that the ML segmentations realize phone-like units which are significantly better than those obtained by STM in terms of match accuracy with TIMIT phone segmentation as well as actual vocoder performance measured in terms of segmental SNR. Moreover, the phone-like units of ML segmentations also outperform the diphone-like units obtained using STM in early vocoders. We also show that the segment vocoder can operate at very high intelligibility when used in a single-speaker mode.

We present a new zero-crossing based algorithm for decomposing a bandpass signal into the amplitude modulation (AM) and frequency modulation (FM) components. In this sequential algorithm, the FM component is first estimated using zero-crossing instant information in a k-nearest neighbour (k-NN) framework. The AM component is estimated by coherent demodulation using a time-varying lowpass filter that uses the estimated instantaneous frequency. Simulation results show that the proposed algorithm gives more accurate envelope and frequency estimates compared to the discrete-energy separation algorithm (DESA) which uses the Teager energy operator. Using the proposed approach on bandpass filtered speech and music, we can extract the fine-structured modulations that occur on a micro-time scale, within an analysis frame.

A new feature set for ASR called Rate-Spectrum(RS) is proposed. RS is a spectral representation obtained using a computational auditory model. The feature is noise-robust and considerably speaker invariant. RS matches the smoothed log spectrum both in shape and dynamic range variation. DCT is used to reduce dimensionality. Comparison of the proposed features with MFCC is done using an Isolated word recognition experiment on the TI Digits database, for clean and noisy speech cases. For speakers seen during training, RS and RS-DCT outperform MFCC in noisy case while matching that of MFCC in the clean case. For unseen speakers, RS does better than MFCC in the clean case, RS-DCT outperforms MFCC in the noisy case. We have thus shown that the proposed feature for ASR is noise robust and speaker invariant.

A clean speech VQ codebook has been shown to be effective in imposing intraframe constraints in Iterative Wiener Filtering (CCIWF) for speech enhancement. However, for time-varying noises, the performance is sub-optimum. We propose a smoothed noise update technique that uses the estimated signal spectrum for subsequent signal estimation. This leads to a more effective solution than the soft-decision based noise estimate found in literature. Further, the CCIWF performance is improved using codebook constraints in the LAR domain instead of LPC domain. Also, a new iteration initialization method is proposed which results in better enhancement in over 70% of the frames. Thus, we show that a combination of a robust parameter space, an effective initialization and continuous spectrum update significantly improves the performance of speech enhancement. Speech recognition results show that the new combination provides 10-20% increase in word recognition scores whereas simple spectral subtraction results in an actual decrease in recognition score.

We discuss the problem of instantaneous frequency (IF) estimation of phase signals using their level-crossing (LC) instant information. We cast the problem to that of interpolating the instantaneous phase (IP), and hence finding the IF, from samples obtained at the level-crossing instants of the phase signal. These are inherently irregularly spaced and the problem essentially reduces to reconstructing a signal from the samples taken at irregularly sampled points for which we propose a 'line plus sum of sines' model. In the presence of noise, the temporal structure of the level-crossings can get distorted. To reduce the effects of noise, we use a short-time Fourier transform (STFT) based enhancement scheme. The performance of the proposed method is studied through Monte-Carlo simulations for a phase signal with composite IF for various SNRs. Different level-crossing based estimates are combined to obtain a new IF estimate. Simulation studies show that the estimates obtained using zero-crossing (ZC) and other very low level values perform better than those obtained with higher level values.

Parallel sub-word recognition (PSWR) is a new model that has been proposed for language identification (LID) which does not need elaborate phonetic labeling of the speech data in a foreign language. The new approach performs a front-end tokenization in terms of sub-word units which are designed by automatic segmentation, segment clustering and segment HMM modeling. We develop PSWR based LID in a framework similar to the parallel phone recognition (PPR) approach in the literature. This includes a front-end tokenizer and a back-end language model, for each language to be identified. Considering various combinations of the statistical evaluation scores, it is found that PSWR can perform as well as PPR, even with broad acoustic sub-word tokenization, thus making it an efficient alternative to the PPR system.

Automatic segmentation of speech is an important problem that is useful in speech recognition, synthesis and coding. We explore in this paper, the robust parameter set, weighting function and distance measure for reliable segmentation of noisy speech. It is found that the MFCC parameters, successful in speech recognition. holds the best promise for robust segmentation also. We also explored a variety of symmetric and asymmetric weighting lifters. from which it is found that a symmetric lifter of the form 1 + A sin1/2(πn/L), 0 ≤ n ≤ L − 1, for MFCC dimension L, is most effective. With regard to distance measure, the direct L2norm is found adequate.

We develop a parametric sinusoidal analysis/synthesis model which can be applied to both speech and audio signals. These signals are characterised by large amplitude variations and small frequency variation within a short analysis frame. The model comprises of a Gaussian mixture representation for the envelope and a sum of linear chirps for the frequency components. A closed form solution is derived for the frequency domain parameters of a chirp with Gaussian-mixture envelope, based on the spectral moments. An iterative algorithm is developed to select and estimate prominent chirps based on the psycho-acoustic masking threshold. The model can adaptively select the number of time-domain and frequency-domain parameters to suit a particular type of signal. Experimental evaluation of the technique has shown that about 2 to 4 parameters/ms is sufficient for near transparent quality reconstruction of a variety of wide-band music and speech signals.

Recently, adaptive subband coders based on wavelet packet decomposition and psychoacoustic modelling have been proposed to achieve transparent quality compression of audio signals [1],[2]. While these coders perform well for stationary signals, there is no special mechanism in the coder to prevent the pre-echo artifact when transient signals are encoded. In this paper, we propose a switched DPCM/Subband structure to remove the pre-echo problem. This is achieved through a novel temporally varying bit allocation scheme which is based on the temporal masking properties of the human auditory system. The proposed coder/decoder output is found to be free from the preecho artifact even at a lower bitrate than the adaptive subband coder.

This paper describes some experiments to reduce the load of the side information in the MPEG advanced audio coder (AAC) scheme using vector quantization (VQ) methods. The VQ replaces the existing differential and entropy coding of the scale factors. Various types of VQ are considered, such as sub-vector VQ, multi-stage VQ and tree-structured VQ which provide some advantages in the context of MPEG-4 AAC characteristics such as scaleability etc. However, the VQ being a lossy compression scheme, psycho-acoustic sensitivity of the losses is very important. These are studied using objective measures such as NMR and listening tests to make proper choices for the VQ design.

Speech enhancement using iterative Wiener filtering has been shown to require interframe and intraframe constraints in all-pole parameter estimation. We show that a clean speech VQ codebook is more effective in providing intraframe constraints and, hence, better convergence of the iterative filtering scheme. Satisfactory speech enhancement results are obtained with a small codebook of 128, and the algorithm is effective for both white noise and pink noise up to 0 dB SNR.

We present a novel scheme for phoneme recognition in continuous speech using inhomogeneous hidden Markov models (IHMMs). IHMMs can capture the temporal structure of phonemes and inter-phonemic temporal relationships effectively, with their duration dependent state transition probabilities. A two stage IHMM is proposed to capture the variabilities in speech effectively for phoneme recognition. The first stage models the acoustic and durational variabilities of all distinct sub-phonemic segments and the second stage models the acoustic and durational variability of the whole phoneme. In an experimental evaluation of the new scheme for recognizing a subset of alphabets comprising of the most confusing set of phonemes, spoken randomly and continuously, a phoneme recognition accuracy of 83% is observed.>

An analysis is presented of an auditory model of the inner ear to achieve robust spectral estimation. To obtain a better estimate of spectral magnitude, a new method of determining aggregate synchrony is developed. Using mathematical analysis, the first-stage bandpass filtering in the model is shown to directly determine spectral resolution and noise robustness. Statistical evaluation results are presented for the performance factors of bias, variance, resolution, and noise threshold of the auditory spectral estimator.>

A hybrid speech coding scheme is proposed based on the properties of the linear predictive coding (LPC)-residual. It is shown that the residual comprises characteristically different components, namely, high-energy pulses, a low-energy smooth component, and a random component. It is proposed that residue coding, as a composite of the respective functions corresponding to the components, is more effective and economical than the existing coding techniques which use only a single function to represent the entire residue. Results of simulating a hybrid coder with full quantization and also subjective evaluation of its performance are reported.>

The Wakita's area functions based on the linear prediction model, assume the vocal tract to be lossless. Due to this assumption, the actual non-zero losses and additional errors in estimating them affect the area functions significantly. To quantify these effects, the sensitivity of area functions to perturbations in band widths and formants is systematically studied and their relative importance is presented. To overcome the effects of errors in bandwidth estimation, a modification for the area function is proposed. The effectiveness of these modified area functions is demonstrated by a vowel recognition test using both Wakita's area functions and the modified area functions.