The author describes the file system that was developed for the Entropic Signal Processing System (ESPS), a UNIX-based software package that supports speech and signal processing research and development. He discusses the major goals of the ESPS file system, namely, that it be efficient, that it fit well with UNIX, that it should maintain a history of parameters and processing steps, that users should be able to modify existing file types and introduce new types without having to rewrite existing programs, and that the file system should facilitate distributed computation on UNIX networks. He then describes the design and implementation of the ESPS file system with emphasis on the foregoing goals and on the use of modern software engineering techniques. He closes with a description of desirable additions and generalizations.>

An application of source coding to speaker recognition is described. The method is text-dependent - the text spoken is known, and the problem is to determine who said it. Each speaker is represented by a sequence of vector quantization codebooks; known input utterances are classified using these codebook sequences and the resulting classification distortion is compared to a rejection threshold. On a 16 speaker test population with an additional 111 imposters, this method achieved a false rejection rate of 0.8%, an imposter acceptance rate of 1.8%, and within the 16 speakers, an identification error rate of 0.0%.

The use of vector quantization (VQ) in isolated-word recognition of a 20-word vocabulary is examined. A separate sequence of VQ code books is designed for each word in the recognition vocabulary and input words are classified by performing VQ and finding the sequence of code books that achieve the smallest average distortion. In this paper, critical parameters are noted and the results of parameter studies are presented.

This paper presents a new multisignal spectrum-analysis method based on the relative-entropy principle ("principle of minimum cross entropy"). The method simultaneously estimates a number of power spectra given (1) an initial ("prior") estimate of each, (2) imprecise values of the autocorrelation function of their sum, and (3) estimates of the error in measurement of the autocorrelation values. One application is to the separate estimation of the spectra of a signal and independent additive noise, based on imprecise measurements of the autocorrelations of the signal plus noise. A derivation is sketched, beginning with a discussion of relative-entropy minimization subject to quadratic constraints that require certain linear constraints to hold approximately, to within a specified error bound. The new method is an extension of multisignal relative-entropy spectrum analysis (with exact autocorrelations). The two methods are compared, and connections with previous related work are indicated. Mathematical properties of the new method are discussed, and an illustrative numerical example is presented.

Maximum likelihood (ML) and minimum relative-entropy (MRE) (minimum cross-entropy) classification of samples from an unknown probability density when the hypotheses comprise an exponential family are considered. It is shown that ML and MRE lead to the same classification nde, and the result is illustrated in terms of a method for estimating covariance matrices recently developed by Burg, Luenberger, and Wenger, MRE classification applies to the general case in which it cannot be assumed that the samples were generated by one of the hypothesis densities. The common use of ML in this case is technically incorrect, but the equivalence of MRE and ML provides a theoretical justification.

The use of vector quantization (VQ) for isolated-word speech recognition of a 20-word vocabulary was shown in previous work to achieve more than 99% accuracy for speaker-dependent recognition and 87% accuracy for speaker-independent recognition. Separate VQ codebooks were designed for each word in the recognition vocabulary, and input words were classified by performing VQ and finding the codebook that achieves the smallest average distortion. The method obviates time-normalization and makes no use of time-sequence information. This paper presents results for a generalization that incorporates time-sequence information. The generalization, which was motivated by work of Martinez, Riveria, and Buzo, is more accurate and faster than the previous method. Words in the training and input sequences are normalized linearly to the same length and then divided into sections. Separate VQ codebooks are designed for each section of each vocabulary word. Each vocabulary word is then represented by a "multisection" codebook - a time-dependent sequence of section-codebooks. New words are classified by performing VQ and finding the multi-section codebook that achieves the smallest average distortion. Initial tests on a twenty-word vocabulary resulted in accuracies greater than 97% for speaker-independent recognition.

This paper presents results of a new spectrum-analysis method that estimates a number of power spectra when a prior estimate of each is available and new information is obtained in the form of values of the auto-correlation function of their sum. The method applies for instance when one obtains autocorrelation measurements for a signal with independent additive interference, and one has prior estimates of the signal and noise spectra. By incorporating prior estimates for both spectra, the method offers considerable flexibility for tailoring an estimator to the characteristics of a signal or noise. The new method, a generalization of Minimum-Cross-Entropy Spectrum Analysis (MCESA) and Maximum Entropy Spectrum Analysis (MESA), is called Multisignal MCESA. Its theoretical basis is reviewed, and results of experimental tests of an implementation are presented. The test data comprise digitized samples of speech corrupted with helicopter noise and tone interference.

An error in the subject paper is pointed out: when the axioms given there are restricted to the discrete case, they do not imply the discrete case of the principle of minimum cross-entropy. The principle is shown to follow, however, from the adoption of an additional axiom: if new information is consistent with a prior estimate of a probability distribution, then the posterior estimate equals the prior. Minor other improvements and corrections to the arguments in the paper are made.

The results of a new method are presented for discrete utterance speech recognition. The method is based on rate-distortion speech coding (speech coding by vector quantization), minimum cross-entropy pattern classification, and information-theoretic spectral distortion measures. Separate vector quantization code books are designed from training sequences for each word in the recognition vocabulary. Inputs from outside the training sequence are classified by performing vector quantization and finding the code book that achieves the lowest average distortion per speech frame. The new method obviates time alignment. It achieves99percent accuracy for speaker-dependent recognition of a20-word vocabulary that includes the ten digits, with higher accuracy for recognition of the digit subset. For speaker-independent recognition, the method achieves88percent accuracy for the20-word vocabulary and95percent for the digit subset. Background of the method, detailed empirical results, and an analysis of computational requirements are presented.

We present a new, fast method for discrete utterance recognition of telephone bandwidth speech. The method is based on speech coding by vector quantization and minimum cross-entropy pattern classification. Separate vector quantization codebooks are designed from training sequences for each word in the recognition vocabulary. Inputs from outside the training sequence are classified by performing vector quantization and finding the codebook that achieves the lowest average distortion per speech frame. The new method obviates time normalization and uses approximately 6000 bits to represent each utterance in the recognition vocabulary. Preliminary limited testing on speaker dependent digit recognition has demonstrated excellent performance. Detailed tests are now in progress.

This paper considers the problem of classifying an input vector of measurements by a nearest neighbor rule applied to a fixed set of vectors. The fixed vectors are sometimes called characteristic feature vectors, codewords, cluster centers, models, reproductions, etc. The nearest neighbor rule considered uses a non-Euclidean information-theoretic distortion measure that is not a metric, but that nevertheless leads to a classification method that is optimal in a well-defined sense and is also computationally attractive. Furthermore, the distortion measure results in a simple method of computing cluster centroids. Our approach is based on the minimization of cross-entropy (also called discrimination information, directed divergence, K-L number), and can be viewed as a refinement of a general classification method due to Kullback. The refinement exploits special properties of cross-entropy that hold when the probability densities involved happen to be minimum cross-entropy densities. The approach is a generalization of a recently developed speech coding technique called speech coding by vector quantization.

The principle of maximum entropy and the principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) have been applied recently to problems in queuing theory and computer-system performance modeling. These information-theoretic principles estimate probability distributions based on information in the form of known expected values. In the case of queuing theory and computer-system modeling, the known expected values arise from rate balance equations. This correspondence concerns situations in which the system state probabilities decompose into disjoint subsets and in which the known expected values are either expectations conditional on a specific subset or expectations involving aggregate subset probabilities. New properties of minimum cross-entropy distributions are derived and an efficient method of computing these distributions is derived. Computational examples are included. In the case of queuing theory and computer-system modeling, the disjoint subsets correspond to internal device states, and the aggregate probabilities correspond to overall device states. The results here apply when one has both rate balance equations for device equilibrium involving internal device state probabilities, as well as rate balance equations for system equilibrium involving aggregate device state probabilities.

An information theory approach to the theory and practice of linear predictive coded (LPC) speech compression systems is developed. It is shown that a traditional LPC system can be viewed as a minimum distortion or nearest-neighbor system where the distortion measure is a minimum discrimination information between a speech process model and an observed frame of actual speech. This distortion measure is used in an algorithm for computer-aided design of block source codes subject to a fidelity criterion to obtain a 750-bits/s speech compression system that resembles an LPC system but has a much lower rate, a larger memory requirement, and requires no on-line LPC analysis. Quantitative and informal subjective comparisons are made among our system and LPC systems.

The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) is a general method of inference about an unknown probability density when there exists a prior estimate of the density and new information in the form of constraints on expected values. Various fundamental properties of cross-entropy minimization are proven and collected in one place. Cross-entropy's well-known properties as an information measure are extended and strengthened when one of the densities involved is the result of cross-entropy minimization. The interplay between properties of cross-entropy minimization as an inference procedure and properties of cross-entropy as an information measure is pointed out. Examples are included and general analytic and computational methods of finding minimum cross-entropy probability densities are discussed.

Jaynes's principle of maximum entropy and Kullbacks principle of minimum cross-entropy (minimum directed divergence) are shown to be uniquely correct methods for inductive inference when new information is given in the form of expected values. Previous justifications use intuitive arguments and rely on the properties of entropy and cross-entropy as information measures. The approach here assumes that reasonable methods of inductive inference should lead to consistent results when there are different ways of taking the same information into account (for example, in different coordinate system). This requirement is formalized as four consistency axioms. These are stated in terms of an abstract information operator and make no reference to information measures. It is proved that the principle of maximum entropy is correct in the following sense: maximizing any function but entropy will lead to inconsistency unless that function and entropy have identical maxima. In other words given information in the form of constraints on expected values, there is only one (distribution satisfying the constraints that can be chosen by a procedure that satisfies the consistency axioms; this unique distribution can be obtained by maximizing entropy. This result is established both directly and as a special case (uniform priors) of an analogous result for the principle of minimum cross-entropy. Results are obtained both for continuous probability densities and for discrete distributions.

Recent interest in the application of Walsh functions suggests that Haar functions, close relatives of Walsh functions, may also be useful. In this primarily tutorial paper, Haar functions are reviewed briefly and the computational and memory requirements of the Haar transform are analyzed; applications are then discussed. It is concluded that, whereas Haar functions are unlikely to be useful in as many applications as Walsh functions may be, they seem especially well suited to data coding, edge detection, and, perhaps, multiplexing.