Abstract Network tomography aims at estimating source–destination traffic rates from link traffic measurements. This inverse problem was formulated by Vardi in 1996 for Poisson traffic over networks operating under deterministic as well as random routing regimes. In this article, we expand Vardi's second‐order moment matching rate estimation approach to higher‐order cumulant matching with the goal of increasing the column rank of the mapping and consequently improving the rate estimation accuracy. We develop a systematic set of linear cumulant matching equations and express them compactly in terms of the Khatri–Rao product. Both least squares estimation and iterative minimum I‐divergence estimation are considered. We develop an upper bound on the mean squared error (MSE) in least squares rate estimation from empirical cumulants. We demonstrate that supplementing Vardi's approach with the third‐order empirical cumulant reduces its minimum averaged normalized MSE in rate estimation by almost 20% when iterative minimum I‐divergence estimation was used.

Stochastic network calculus involves the use of a traffic bound or envelope to make admission control and resource allocation decisions for providing end-to-end quality-of-service guarantees. To apply network calculus in practice, the traffic envelope should: (i) be readily determined for an arbitrary traffic source, (ii) be enforceable by traffic regulation, and (iii) yield statistical multiplexing gain. Existing traffic envelopes typically satisfy at most two of these properties. A well-known traffic envelope based on the moment generating function (MGF) of the arrival process satisfies only the third property. We propose a new traffic envelope based on the MGF of the workload process obtained from offering the traffic to a constant service rate queue. We show that this traffic workload envelope can achieve all three properties and leads to a framework for a network service that provides stochastic delay guarantees. We demonstrate the performance of the traffic workload envelope with two bursty traffic models: Markov on-off fluid and Markov modulated Poisson Process (MMPP).

In a wireless network with dynamic spectrum sharing, tracking temporal spectrum holes across a wide spectrum band is a challenging task. We consider a scenario in which the spectrum is divided into a large number of bands or channels, each of which has the potential to provide dynamic spectrum access opportunities. The occupancy times of each band by primary users are generally non-exponentially distributed. We develop an approach to determine and parameterize a small selected subset of the bands with good spectrum access opportunities, using limited computational resources under noisy measurements. We model the noisy measurements of the received signal in each band as a bivariate Markov modulated Gaussian process, which can be viewed as a continuous-time bivariate Markov chain observed through Gaussian noise. The underlying bivariate Markov process allows for the characterization of non-exponentially distributed state sojourn times. The proposed scheme combines an online expectation-maximization algorithm for parameter estimation with a computing budget allocation algorithm. Observation time is allocated across the bands to determine the subset of G*out of G frequency bands with the largest mean idle times for dynamic spectrum access and at the same time to obtain accurate parameter estimates for this subset of bands. Our simulation results show that when channel holding times are non-exponential, the proposed scheme achieves a substantial improvement in the probability of correct selection of the best subset of bands compared to an approach based on a (univariate) Markov modulated Gaussian process model.

Providing end-to-end network delay guarantees in packet-switched networks such as the Internet is highly desirable for mission-critical and delay-sensitive data transmission, yet it remains a challenging open problem. Due to the looseness of the deterministic bounds, various frameworks for stochastic network calculus have been proposed to provide tighter, probabilistic bounds on network delay, at least in theory. However, little attention has been devoted to the problem of regulating traffic according to stochastic burstiness bounds, which is necessary in order to guarantee the delay bounds in practice. We propose and analyze a stochastic traffic regulator that can be used in conjunction with results from stochastic network calculus to provide probabilistic guarantees on end-to-end network delay. Numerical results are provided to demonstrate the performance of the proposed traffic regulator.11This work was supported in part by the U.S. National Science Foundation under Grant No. 1717033.

Under a dynamic spectrum access paradigm, a set of L spectrum bands licensed to primary users provide opportunities for an unlicensed secondary user to gain access to spectrum left idle by a primary user. We model the received noisy signal measurements on each band as a continuous-time Markov chain observed through a discrete-time Gaussian channel. Based on this model, we develop a scheme for estimating the parameters of the subset of L* <; L bands that offer the “best” opportunities for dynamic spectrum access in the sense of largest mean idle periods. Our approach consists of a Markov modulated Gaussian process model, an associated expectation-maximization algorithm, and a computing budget allocation scheme for allocating sensing effort across the spectrum bands over a sequence of observation intervals. The sensing effort allocation scheme maximizes the probability that the L* best bands will be determined from their parameter stimates obtained in the next observation interval. Simulation results are presented to demonstrate the performance of the proposed scheme.

Spectrum sensing enables secondary users in a cognitive radio network to opportunistically access portions of the spectrum left idle by primary users. Tracking spectrum holes jointly in time and frequency over a wide spectrum band is a challenging task. In one approach to wideband temporal sensing, the spectrum band is partitioned into narrowband subchannels of fixed bandwidth, which are then characterized via hidden Markov modeling using average power or energy measurements as observation data. Adjacent, correlated subchannels are recursively aggregated into channels of variable bandwidths, corresponding to the primary user signals. Thus, wideband temporal sensing is transformed into a multiband sensing scenario by identifying the primary user channels in the spectrum band. However, future changes in the configuration of the primary user channels in the multiband setup cannot generally be detected using an energy detector front end for spectrum sensing. We propose the use of a cepstral feature vector to detect changes in the spectrum envelope of a primary user channel. Our numerical results show that the cepstrum-based spectrum envelope detector performs well under moderate to high signal-to-noise ratio conditions.

In dynamic or opportunistic spectrum access, the primary user (PU) alternates between an idle and an active state, and a secondary user (SU) may access the channel during the idle periods. In multiband spectrum sensing, an SU tracks the PU state on a given set of channels to determine spectrum access opportunities. In this context, we address the following problem: Given amp;#924; channels, determine the best subset of amp;#925; amp;#8804; amp;#924; channels with respect to spectrum access opportunities and, at the same time, estimate the parameter of the PU state process for each channel within the selected subset. Specifically, we model the PU state on the given set of channels by amp;#924; independent, two-state continuous-time Markov chains. Over a given interval of time, our goal is to determine, with high probability, the amp;#925; channels with the largest mean idle periods and, at the same time, to accurately estimate the parameter of each channel in the selected subset. We adapt the optimal computing budget allocation (OCBA) methodology from the field of simulation optimization to allocate the total time budget for sensing the $N$ channels in order to perform the channel subset selection and parameter estimation. Simulation results are presented to demonstrate the performance of the proposed algorithm.

Estimation of link delay densities in a computer network, from source-destination delay measurements, is of great importance in analyzing and improving the operation of the network. In this paper, we develop a general approach for estimating the density of the delay in any link of the network, based on continuous-time bivariate Markov chain modeling. The proposed approach also provides the estimates of the packet routing probability at each node, and the probability of each source-destination path in the network. In this approach, the states of one process of the bivariate Markov chain are associated with nodes of the network, while the other process serves as an underlying process that affects statistical properties of the node process. The node process is not Markov, and the sojourn time in each of its states is phase-type. Phase-type densities are dense in the set of densities with non-negative support. Hence, they can be used to approximate arbitrarily well any sojourn time distribution. Furthermore, the class of phase-type densities is closed under convolution and mixture operations. We adopt the expectation-maximization (EM) algorithm of Asmussen, Nerman, and Olsson for estimating the parameter of the bivariate Markov chain. We demonstrate the performance of the approach in a numerical study.

Collaborative spectrum sensing exploits multiuser diversity by combining spectrum sensing information from multiple secondary users to make joint decisions about spectrum occupancy. In hard fusion schemes, each secondary user makes a hard decision on spectrum occupancy and a fusion center makes a final decision by combining the individual hard decisions according to a fusion rule. In soft fusion schemes, each secondary user provides a signal power measurement to the fusion center, which performs further processing on the collection of all observations to make a final decision. In this paper, we propose hard and soft fusion collaborative spectrum sensing schemes based on the online hidden bivariate Markov chain modeling of the signals received by secondary users. Compared with prior collaborative sensing schemes, the proposed model-based schemes do not rely on precomputed thresholds or weights, and achieve superior performance. The online estimation of hidden bivariate Markov models provides predictive information that can be used to improve the performance of the dynamic spectrum access. Numerical results are presented to demonstrate the performance and communication overhead tradeoffs of the proposed collaborative spectrum sensing schemes.

We develop a computationally efficient online parameter estimation algorithm for temporal spectrum sensing of a cognitive radio channel using a hidden bivariate Markov model. The online estimator is based on a block-recursive parameter estimation algorithm developed by Rydén for hidden Markov models. This approach requires the score function only. We develop an efficient method for computing the score function recursively and extend Rydén's approach to hidden bivariate Markov models. The advantage of the hidden bivariate Markov model over the hidden Markov model is its ability to characterize non-geometric state sojourn time distributions, which can be crucial in spectrum sensing. Based on the hidden bivariate Markov model, an estimate of the future state of the primary user can be obtained, which can be used to reduce harmful interference and improve channel utilization. Moreover, the online estimator can adapt to changes in the statistical characteristics of the primary user. We present numerical results that demonstrate the performance of temporal spectrum sensing using the proposed online parameter estimator.

A new statistical model, in the form of a hidden bivariate Markov chain observed through a Gaussian channel, is developed and applied to spectrum sensing for cognitive radio. We focus on temporal spectrum sensing in a single narrowband channel in which a primary transmitter is either in an idle or an active state. The main advantage of the proposed model, compared to a standard hidden Markov model (HMM), is that it allows a phase-type dwell time distribution for the process in each state. This distribution significantly generalizes the geometric dwell time distribution of a standard HMM. Measurements taken from real data confirm that the geometric dwell time distribution characteristic of the HMM is not adequate for this application. The Baum algorithm is used to estimate the parameter of the proposed model and a forward recursion is applied to online estimation and prediction of the state of the cognitive radio channel. The performance of the proposed model and spectrum sensing approach are demonstrated using numerical results derived from real spectrum measurement data.

Two aspects of Ryden's expectation-maximization algorithm for estimating the parameter of a Markov modulated Poisson process are addressed. First, a scaling procedure is developed for the forward-backward recursions that circumvents the need for customized floating-point software. Second, evaluation of integrals of matrix exponentials is facilitated by applying a result due to Van Loan. For an MMPP of order four, a speedup of over two orders of magnitude was observed.

Linear predictive hidden Markov modeling is compared with a simple form of the switching autoregressive process. The latter process captures existing signal correlation during transitions of the Markov chain. Parameter estimation is described using naturally stable forward-backward recursions. The switching autoregressive model outperformed the linear predictive model in a digit recognition task and provided comparable performance to a cepstral-based recognizer.

We derive an explicit expression for the covariance of the log-periodogram power spectral density estimator for a zero mean Gaussian process. We do not make the assumption that the spectral components of the process are uncorrelated. Applications to spectral estimation and to cepstral modeling in automatic speech recognition are discussed.

Speaker classification is seen as a hypothesis testing problem of J simple hypotheses and a composite hypothesis. The simple hypotheses represent target speakers while the composite hypothesis represents nontarget speakers. The simple hypotheses have well-defined distributions that are estimated from training signals. The distribution of the signal under the composite hypothesis is assumed to belong to a given family. The parameter of that distribution is assumed random with a prior distribution that is estimated from a large set of speakers. This formulation converts the problem to that of testing J+1 simple hypotheses. Signals corresponding to target and nontarget speakers are assumed Gaussian mixtures processes. Once the system has been trained, list decoding is applied in which a test signal is associated with a list of possible speakers. The probability that the correct speaker is on the list is maximized for a given average number of incorrect speakers on the list. Results from speaker identification and speaker verification experiments are reported. In speaker identification using a National Institute of Standards and Technology (NIST) database with 174 target speakers, over 77% correct identification was achieved for an average of less than two erroneous speakers on the list. Speaker verification experiments on a similar database yielded results, expressed in terms of the equal-error-rate, of 6.7% and 10.1% using two decision rules.

The signal subspace approach for speech enhancement is extended to colored-noise processes. Explicit forms for the linear time-domain- and spectral-domain-constrained estimators are presented. These estimators minimize the average signal distortion power for given constraints on the residual noise power in the time and spectral domains, respectively. Equivalent implementations of the two estimators using the whitening approach are described.

In 1989 a new algorithm for non-iterative global maximization of the joint likelihood function of state and observation sequences of left-right HMM's was developed by Faragó and Lugosi. The algorithm capitalizes on the fact that the state sequence of a left-right HMM is uniquely determined by the state duration occupancies, The algorithm is mostly suitable for parameter estimation from a single training sequence. Extensions to estimation from multiple sequences are possible but deemed impractical. Two alternatives are proposed for utilizing this algorithm in automatic speech recognition. The first is based on averaging the parameter estimates from individual sequences while the second uses the Faragó and Lugosi segmentation to initialize the segmental k-means or the Baum algorithm. We have implemented the algorithm and tested it in isolated digit recognition. Using the second approach, a reduction of the error rate from .65% to .36% was realized

An overview of statistical and information-theoretic aspects of hidden Markov processes (HMPs) is presented. An HMP is a discrete-time finite-state homogeneous Markov chain observed through a discrete-time memoryless invariant channel. In recent years, the work of Baum and Petrie (1966) on finite-state finite-alphabet HMPs was expanded to HMPs with finite as well as continuous state spaces and a general alphabet. In particular, statistical properties and ergodic theorems for relative entropy densities of HMPs were developed. Consistency and asymptotic normality of the maximum-likelihood (ML) parameter estimator were proved under some mild conditions. Similar results were established for switching autoregressive processes. These processes generalize HMPs. New algorithms were developed for estimating the state, parameter, and order of an HMP, for universal coding and classification of HMPs, and for universal decoding of hidden Markov channels. These and other related topics are reviewed.

The optimal linear preprocessor for classifying two zero-mean Gaussian discrete-time signals which have been corrupted by additive zero-mean Gaussian noise is studied. Conditions for existence of the optimal linear preprocessor that achieves the performance of the likelihood ratio test for the noisy signals are given and the preprocessor is explicitly derived.

Explicit expressions for the second-order statistics of cepstral components representing clean and noisy signal waveforms are derived. The noise is assumed additive to the signal, and the spectral components of each process are assumed statistically independent complex Gaussian random variables. The key result developed here is an explicit expression for the cross-covariance between the log-periodograms of the clean and noisy signals. In the absence of noise, this expression is used to show that the covariance matrix of cepstral components representing N signal samples, is a fixed signal independent matrix, which approaches a diagonal matrix at a rate of 1/N. In addition, the cross-covariance expression is used to develop an explicit linear minimum mean square error estimator for the clean cepstral components given noisy cepstral components. Recognition results on the English digits using the fixed covariance and linear estimator are presented.

Explicit expressions for the second order statistics of cepstral components representing clean and noisy signal waveforms are derived. The noise is assumed additive to the signal, and the spectral components of each process are assumed statistically independent complex Gaussian random variables. The key result developed here is an explicit expression for the cross-covariance between the log-spectra of the clean and noisy signals. In the absence of noise, this expression is used to show that the covariance matrix of cepstral components representing a vector of N signal samples, approaches a fixed, signal independent, diagonal matrix at a rate of 1/N/sup 2/. In addition, the cross-covariance expression is used to develop an explicit linear minimum mean square error estimator for the clean cepstral components given noisy cepstral components. Recognition results on the ten English digits using the fixed covariance and linear estimator are presented.

An adaptive beamformer which is robust to uncertainty in source DOA is derived. The beamformer is a weighted sum of minimum variance distortionless response (MVDR) beamformers pointed at a set of candidate DOAs, where the relative contribution of each MVDR beamformer is determined from a combination of observed data and prior knowledge about the DOA. When SNR is high, the MVDR beamformer whose look direction is closest to the source dominates, and nearly optimal performance is obtained. When SNR is low, the weighted combination of beamformers has a wider main beam which is robust to DOA uncertainty.

The Bayesian Ziv-Zakai bound on the mean square error (MSE) in estimating a uniformly distributed continuous random variable is extended for arbitrarily distributed continuous random vectors and for distortion functions other than MSE. The extended bound is evaluated for some representative problems in time-delay and bearing estimation. The resulting bounds have simple closed-form expressions, and closely predict the simulated performance of the maximum-likelihood estimator in all regions of operation.

The extended Ziv-Zakai bound for vector parameters is used to develop a lower bound on the mean square error (MSE) in estimating the two-dimensional bearing of a narrowband planewave signal using planar arrays of arbitrary geometry. The bound has a simple closed form expression which is a function of the signal wavelength, the signal-to-noise ratio (SNR), the number of data snapshots, the number of sensors in the array, and the array configuration. Analysis of the bound suggests that there are several regions of operation, and expressions for the thresholds separating the regions are provided. In the asymptotic region where the number of snapshots and/or SNR are large, the bound approaches the inverse Fisher information. In the a priori performance region where the number of snapshots or SNR is small, the bound approaches the a priori covariance. In the transition region, the bound varies smoothly between the two extremes. Results from simulation of the maximum likelihood estimator (MLE) demonstrate that the bound closely predicts the performance of the MLE in all regions.

Bounds on the MSE in estimating the bearing of a planewave signal is of considerable interest in many fields. Of particular importance is the ability of a bound to closely characterize performance in the small error or asymptotic region, and the large error or ambiguity region, and to accurately predict the location of the threshold between the regions. The vector Ziv-Zakai bound is applied to the problem of estimating two-dimensional bearing with planar arrays of arbitrary geometry. The bound is calculated for square and circular arrays, and compared with the Weiss-Weinstein (1983, 1984) bound. The Ziv-Zakai bound is shown to be tighter than the Weiss-Weinstein bound in the threshold and asymptotic regions.

The signal subspace approach for enhancing speech signals degraded by uncorrelated additive noise is studied. The underlying principle is to decompose the vector space of the noisy signal into a signal plus noise subspace and a noise subspace. Enhancement is performed by removing the noise subspace and estimating the clean signal from the remaining signal subspace. The decomposition can theoretically be performed by applying the Karhunen-Loeve transform to the noisy signal. Linear estimation of the clean signal is performed using a perceptually meaningful estimation criterion. The estimator is designed by minimizing signal distortion for a fixed desired spectrum of the residual noise. This criterion enables masking of the residual noise by the speech signal. The filter is implemented as a gain function which modifies the KLT components corresponding to the signal subspace. The gain function is solely dependent on the desired spectrum of the residual noise. Listening tests indicate that 14 out of 16 listeners strongly preferred the proposed approach over the spectral subtraction approach.

A comprehensive approach for nonparametric speech enhancement is developed. The underlying principle is to decompose the vector space of the noisy signal into a signal-plus-noise subspace and a noise subspace. Enhancement is performed by removing the noise subspace and estimating the clean signal from the remaining signal subspace. The decomposition can theoretically be performed by applying the Karhunen-Loeve transform (KLT) to the noisy signal. Linear estimation of the clean signal is performed using two perceptually meaningful estimation criteria. First, signal distortion is minimized while the residual noise energy is maintained below some given threshold. This criterion results in a Wiener filter with adjustable input noise level. Second, signal distortion is minimized for a fixed spectrum of the residual noise. This criterion enables masking of the residual noise by the speech signal. It results in a filter whose structure is similar to that obtained in the first case, except that now the gain function which modifies the KLT coefficients is solely dependent on the desired spectrum of the residual noise. The popular spectral subtraction speech enhancement approach is shown to be a particular case of the proposed approach. It is proven to be a signal subspace approach which is optimal in an asymptotic (large sample) linear minimum mean square error sense, assuming the signal and noise are stationary. Our listening tests indicate that 14 out of 16 listeners strongly preferred the proposed approach over the spectral subtraction approach.>

A time-varying linear dynamical system model for speech signals is proposed. The model generalizes the standard hidden Markov model (HMM) in the sense that vectors generated from a given sequence of states are assumed a first order Markov process rather than a sequence of statistically independent vectors. The reestimation formulas for the model parameters are developed using the Baum algorithm. The forward formula for evaluating the likelihood of a given sequence of signal vectors in speech recognition applications is also developed. The dynamical system model is used in developing minimum mean square error (MMSE) and maximum a posteriori (MAP) signal estimators given noisy signals. Both estimators are shown to be significantly more complicated than similar estimators developed earlier using the standard HMM. A feasible approximate MAP estimation approach in which the states of the signal and the signal itself are alternatively estimated using Viterbi decoding and Kalman filtering is also presented.>

Since the statistics of the speech signal as well as of the noise are not explicitly available, and the most perceptually meaningful distortion measure is not known, model-based approaches have recently been extensively studied and applied to the three basic problems of speech enhancement: signal estimation from a given sample function of noisy speech, signal coding when only noisy speech is available, and recognition of noisy speech signals in man-machine communication. Research on the model-based approach is integrated and put into perspective with other more traditional approaches for speech enhancement. A unified statistical approach for the three basic problems of speech enhancement is developed, using composite source models for the signal and noise and a fairly large set of distortion measures.>

The performance of a minimum mean-square error (MMSE) estimator for the output signal from a composite source model (CSM), which has been degraded by statistically independent additive noise, is analyzed for a wide class of discrete-time and continuous-time models. In both cases, the MMSE is decomposed into the MMSE of the estimator, which is informed of the exact states of the signal and noise, and an additional error term. This term is tightly upper and lower bounded. The bounds for the discrete-time signals are developed using distribution tilting and Shannon's lower bound on the probability of a random variable exceeding a given threshold. The analysis for the continuous-time signal is performed using Duncan's theorem. The bounds in this case are developed by applying the data processing theorem to sampled versions of the state process and its estimate, and by using Fano's inequality. The bounds in both cases are explicitly calculated for CSMs with Gaussian subsources. For causal estimation, these bounds approach zero harmonically as the duration of the observed signals approaches infinity.>

Motivation for using minimum mean square error (MMSE) estimation in noisy speech enhancement problems is given. An MMSE estimator which is based on hidden Markov modeling of the clean signal as well as the noise process is systematically developed. The MMSE estimator is tested and compared with the spectral subtraction estimator in vector quantization of noisy speech signals.>

Approximate maximum likelihood (ML) hidden Markov modeling using the most likely state sequence (MLSS) is examined and compared with the exact ML approach that considers all possible state sequences. It is shown that, for any hidden Markov model (HMM), the difference between the approximate and the exact normalized likelihood functions cannot exceed the logarithm of the number of states divided by the dimension of the output vectors (frame length). Furthermore, for Gaussian HMMs and a given observation sequence, the MLSS is typically the sequence of nearest-neighbor states in the Itakura-Saito sense, and the posterior probability of any state sequence which departs from the MLSS in a single time instant decays exponentially with the frame length. Hence, for a sufficiently large frame length the exact and approximate ML approaches provide similar model estimates and likelihood values.>

A Bayesian approach to classification of parametric information sources whose statistics are not explicitly given is studied and applied to recognition of speech signals based upon hidden Markov modeling. A classifier based on generalized likelihood ratios, which depends only on the available training and testing data, is developed and shown to be optimal in the sense of achieving the highest asymptotic exponential rate of decay of the error probability. The proposed approach is compared to the standard classification approach used in speech recognition, in which the parameters for the sources are first estimated from the given training data, and then the maximum and posteriori (MAP) decision rule is applied using the estimated statistics.>

A minimum mean square error (MMSE) estimation approach for enhancing speech signals degraded by statistically independent additive noise is developed, based upon Gaussian autoregressive (AR) hidden Markov modeling of the clean signal and Gaussian AR modeling of the noise process. The parameters of the models for the two processes are estimated from training sequences of clean speech and noise samples. It is shown that the MMSE estimator comprises a weighted sum of MMSE estimators for the individual output processes corresponding to the different states of the hidden Markov model for the clean speech. The weights at each time instant are the probabilities of the individual estimators to be the correct ones given the noisy speech. Typical signal-to-noise ratio (SNR) improvements achieved by this approach are 4.5-5.5 dB at 10-dB input SNR. >

An approach for designing a set of acoustic models for speech recognition applications which results in a minimal empirical error rate for a given decoder and training data is studied. In an evaluation of the system for an isolated word recognition task, hidden Markov models (HMMs) are used to characterize the probability density functions of the acoustic signals from the different words in the vocabulary. Decoding is performed by applying the maximum aposteriori decision rule to the acoustic models. The HMMs are estimated by minimizing a differentiable cost function, which approximates the empirical error rate function, using the steepest descent method. The HMMs designed by the minimum empirical error rate approach were used in multispeaker recognition of the English E-set words and compared to models designed by the standard maximum-likelihood estimation approach. The approach increased recognition accuracy from 68.2% to 76.2% on the training set and from 53.4% to 56.4% on an independent set of test data.>

Some relations among approaches that have been applied to estimating models for acoustic signals in speech recognition systems are examined. In particular, the modeling approaches based on maximum likelihood (ML), maximum mutual information (MMI), and minimum discrimination information (MDI) are studied. It is shown that all three approaches can be formulated uniformly as MDI modeling approaches for simultaneous estimation of the acoustic models for all words in the vocabulary and that none of the approaches requires any model correctness assumption. The three approaches differ in the effective source being modeled and in the probability distribution attributed to this source.>

A maximum a posteriori approach for enhancing speech signals which have been degraded by statistically independent additive noise is proposed. The approach is based upon statistical modeling of the clean speech signal and the noise process using long training sequences from the two processes. Hidden Markov models (HMMs) with mixtures of Gaussian autoregressive (AR) output probability distributions are used to model the clean speech signal. A low-order Gaussian AR model is used for the wideband Gaussian noise considered here. The parameter set of the HMM is estimated using the Baum or the EM (estimation-maximization) algorithm. The enhancement of the noisy speech is done by means of reestimation of the clean speech waveform using the EM algorithm. An approximate improvement of 4.0-6.0 dB in signal-to-noise ratio (SNR) is achieved at 10 dB input SNR.>

An iterative approach for minimum-discrimination-information (MDI) hidden Markov modeling of information sources is proposed. The approach is developed for sources characterized by a given set of partial covariance matrices and for hidden Markov models (HMMs) with Gaussian autoregressive output probability distributions (PDs). The approach aims at estimating the HMM which yields the MDI with respect to all sources that could have produced the given set of partial covariance matrices. Each iteration of the MDI algorithm generates a new HMM as follows. First, a PD for the source is estimated by minimizing the discrimination information measure with respect to the old model over all PDs which satisfy the given set of partial covariance matrices. Then a new model that decreases the discrimination information measure between the estimated PD of the source and the PD of the old model is developed. The problem of estimating the PD of the source is formulated as a standard constrained minimization problem in the Euclidean space. The estimation of a new model given the PD of the source is done by a procedure that generalizes the Baum algorithm. The MDI approach is shown to be a descent algorithm for the discrimination information measure, and its local convergence is proved.>

An algorithm is proposed for enhancing noisy speech which has been degraded by statistically independent additive noise. The algorithm is based on modeling the clean speech as a hidden Markov process with mixtures of Gaussian autoregressive (AR) output processes and modeling the noise as a sequence of stationary, statistically independent, Gaussian AR vectors. The parameter sets of the models are estimated using training sequences from the clean speech and the noise process. The parameter set of the hidden Markov model is estimated by the segmental k-means algorithm. Given the estimated models, the enhancement of the noisy speech is done by alternate maximization of the likelihood function of the noisy speech, one over all sequences of states and mixture components assuming that the clean speech signal is given, and then over all vectors of the original speech using the resulting most probable sequence of states and mixture components. This alternating maximization is equivalent to first estimating the most probable sequence of AR models for the speech signal using the Viterbi algorithm, and then applying these AR models for constructing a sequence of Wiener filters which are used to enhance the noisy speech.>

The authors examine the relations between maximum likelihood (ML), maximum mutual information (MMI), and minimum discrimination information (MDI) modeling approaches, which have been applied to estimating acoustic word models in speech recognition systems. The show that all three approaches can be uniformly formulated as MDI modeling approaches for estimating the acoustic models for all words simultaneously. The three approaches differ in either the probability distribution (PD) attributed to the source being modeled or in the model effectively being used. None of the approaches, however, assumes model correctness, i.e., that the source has the PD of the model. A new modeling approach is proposed, which, in contrast with the other approaches considered, directly aims at the minimization of the probability of error.>

Data compression by vector quantization is considered for sources which have been degraded by noise. It is shown that, by appropriately modifying the given distortion measure, the problem becomes a standard quantization problem for the noisy source and the modified distortion measure. For the special case of sources corrupted by statistically independent additive noise, the authors provide sufficient conditions on the original distortion measure and probability distributions of the source and the noise for convergence of the generalized Lloyd algorithm in designing the quantizers. The results are specialized to waveform and autoregressive model vector quantization using the weighted quadratic and the Itakura-Saito distortion measures, respectively.>

An explicit expression is derived for the minimum discrimination information (MDI) measure with respect to Gaussian priors for sources characterized by their mean and by any principal leading block of their covariance matrix. An explicit expression is provided for the MDI extension of the given partial covariance of the source with respect to a Gaussian prior. For zero-mean sources and zero-mean Gaussian priors that are asymptotically weakly stationary (AWS) processes, it is shown that the asymptotic MDI measure equals half the Itakura-Saito distortion measure between the asymptotic power spectral densities of the source and prior. Asymptotic MDI modelling of a given AWS source by autoregressive and autoregressive moving average models, which are AWS models, is considered, and conditions are given for convergence of the sample covariance estimator of the source to the stationary covariance used in the modelling.>

A new iterative approach for hidden Markov modeling of information sources which aims at minimizing the discrimination information (or the cross-entropy) between the source and the model is proposed. This approach does not require the commonly used assumption that the source to be modeled is a hidden Markov process. The algorithm is started from the model estimated by the traditional maximum likelihood (ML) approach and alternatively decreases the discrimination information over all probability distributions of the source which agree with the given measurements and all hidden Markov models. The proposed procedure generalizes the Baum algorithm for ML hidden Markov modeling. The procedure is shown to be a descent algorithm for the discrimination information measure and its local convergence is proved.

We investigate the performance of a recent algorithm for linear predictive (LP) modeling of speech signals, which have been degraded by uncorrelated additive noise, as a front-end processor in a speech recognition system. The system is speaker dependent, and recognizes isolated words, based on dynamic time warping principles. The LP model for the clean speech is estimated through appropriate composite modeling of the noisy speech. This is done by minimizing the Itakura-Saito distortion measure between the sample spectrum of the noisy speech and the power spectral density of the composite model. This approach results in a "filtering-modeling" scheme in which the filter for the noisy speech, and the LP model for the clean speech, are alternatively optimized. The proposed system was tested using the 26 word English alphabet, the ten English digits, and the three command words, "stop," "error," and "repeat," which were contaminated by additive white noise at 5-20 dB signal to noise ratios (SNR's). By replacing the standard LP analysis with the proposed algorithm, during training on the clean speech and testing on the noisy speech, we achieve an improvement in recognition accuracy equivalent to an increase in input SNR of approximately 10 dB.

A speech enhancement system which utilizes an optimal (in the minimum mean square error sense) short-time spectral amplitude estimator is described. The derivation of the optimal estimator is based on modeling speech as a quasi-periodic signal, and on applying spectral decomposition. The optimal spectral amplitude estimator and a recently developed vector spectral subtraction amplitude estimator, are round to be nearly equivalent. The optimal spectral amplitude estimator coincides with a Wiener spectral amplitude estimator at high signal to noise ratio (SNR) values, and is found to be superior to it at low SNR values. The enhanced speech obtained by using the proposed system, is less spectrally distorted, although contains some more residual noise, than the enhanced speech obtained by using the Wiener spectral amplitude estimator, in the same system. In addition, it is free of the "musical noise" characteristic to the spectral subtraction algorithm. Both systems, the proposed one and spectral subtraction, have approximately the same complexity.