UNLABELLED: Biological network analysis can be enhanced by examining the connections between nodes and the rest of the network. For this purpose we have developed GraphletCounter, an open-source software tool for computing graphlet degree signatures that can operate on its own or as a plug-in to the network analysis environment Cytoscape. A unique characteristic of GraphletCounter is its ability to compute the graphlet signatures of network motifs, which can be specified by files generated by the motif-finding tool mfinder. GraphletCounter displays graphlet signatures for visual inspection within Cytoscape, and can output graphlet data for integration with larger workflows. AVAILABILITY AND IMPLEMENTATION: GraphletCounter is implemented in Java. It can be downloaded from the Cytoscape plugin repository, and is also available at http://sonmezsysbio.org/software/ graphletcounter.

There are currently a large number of "orphan" G-protein-coupled receptors (GPCRs) whose endogenous ligands (peptide hormones) are unknown. Identification of these peptide hormones is a difficult and important problem. We describe a computational framework that models spatial structure along the genomic sequence simultaneously with the temporal evolutionary path structure across species and show how such models can be used to discover new functional molecules, in particular peptide hormones, via cross-genomic sequence comparisons. The computational framework incorporates a priori high-level knowledge of structural and evolutionary constraints into a hierarchical grammar of evolutionary probabilistic models. This computational method was used for identifying novel prohormones and the processed peptide sites by producing sequence alignments across many species at the functional-element level. Experimental results with an initial implementation of the algorithm were used to identify potential prohormones by comparing the human and non-human proteins in the Swiss-Prot database of known annotated proteins. In this proof of concept, we identified 45 out of 54 prohormones with only 44 false positives. The comparison of known and hypothetical human and mouse proteins resulted in the identification of a novel putative prohormone with at least four potential neuropeptides. Finally, in order to validate the computational methodology, we present the basic molecular biological characterization of the novel putative peptide hormone, including its identification and regional localization in the brain. This species comparison, HMM-based computational approach succeeded in identifying a previously undiscovered neuropeptide from whole genome protein sequences. This novel putative peptide hormone is found in discreet brain regions as well as other organs. The success of this approach will have a great impact on our understanding of GPCRs and associated pathways and help to identify new targets for drug development.

Multiple recent studies have shown that speaker recognition performance using frame-based cepstral features is improved by adding higher-level information, including prosodic and lexical features. This paper explores the important question of finding a good kernel for a system that models syllable-based prosodic features using support vector machines (SVMs). The system has been the best performing of our high-level systems in the last two NIST evaluations, and gives significant improvements when combined with cepstral-based systems. We introduce two new methods for transforming the syllable-level features into a single high-dimensional vector that can be well modeled by SVMs, resulting in significant gains in speaker recognition performance.

We describe a computational framework that models spatial structure along the genomic sequence simultaneously with the temporal evolutionary path structure and show how such models can be used to discover new functional molecules through cross-genomic sequence comparisons. The framework incorporates a priori high-level knowledge of structural and evolutionary constraints in terms of a hierarchical grammar of evolutionary probabilistic models. In particular, we demonstrate a novel computational method for identifying novel prohormones and the processed peptide sites by producing sequence alignments across many species at the functional-element level. We present experimental results with an initial implementation of the algorithm used to identify potential prohormones by comparing the human and mouse proteins, resulting in high accuracy identification in a known set of proteins and a putative novel hormone from an unknown set. Finally, in order to validate the computational methodology, we present the basic molecular biological characterization of the novel putative peptide hormone, including identification in the brain and regional localizations. The success of this approach will have a great impact on our understanding of GPCRs and associated pathways, and help us identify new targets for drug development.

Most commonly used kernels are invariant to permutations of the feature vector components. This characteristic may make machine learning methods that use such kernels suboptimal in cases where the feature vector has an underlying structure. In this paper we will consider one such case, where the features are spatially related. We show a way to modify the objective function of the support vector machine (SVM) optimization problem to account for this structure. The new optimization problem can be implemented as a standard SVM using a particular smoothing kernel. Results are shown on a speaker verification task using prosodic features that are transformed using a particular implementation of the Fisher score. The proposed method leads to improvements of as much as 15 % in equal error rate (EER).

Recent work in speaker recognition has demonstrated the advantage of modeling stylistic features in addition to traditional cepstral features, but to date there has been little study of the relative contributions of these different feature types to a state-of-the-art system. In this paper we provide such an analysis, based on SRI's submission to the NIST 2005 speaker recognition evaluation. The system consists of 7 subsystems (3 cepstral 4 stylistic). By running independent N-way subsystem combinations for increasing values of N, we fines that (1) a monotonic pattern in the choice of the best N systems allows for the inference of subsystem importance; (2) the ordering of subsystems alternates between cepstral and stylistic; (3) syllable-based prosodic features are the strongest stylistic features, and (4) overall subsystem ordering depends crucially on the amount of training data (1 versus 8 conversation sides). Improvements over the baseline cepstral system, when all systems are combined, range from 47% to 67%, with larger improvements for the 8-side condition. These results provide direct evidence of the complementary contributions of cepstral and stylistic features to speaker discrimination

We summarize recent progress in automatic speech-to-text transcription at SRI, ICSI, and the University of Washington. The work encompasses all components of speech modeling found in a state-of-the-art recognition system, from acoustic features, to acoustic modeling and adaptation, to language modeling. In the front end, we experimented with nonstandard features, including various measures of voicing, discriminative phone posterior features estimated by multilayer perceptrons, and a novel phone-level macro-averaging for cepstral normalization. Acoustic modeling was improved with combinations of front ends operating at multiple frame rates, as well as by modifications to the standard methods for discriminative Gaussian estimation. We show that acoustic adaptation can be improved by predicting the optimal regression class complexity for a given speaker. Language modeling innovations include the use of a syntax-motivated almost-parsing language model, as well as principled vocabulary-selection techniques. Finally, we address portability issues, such as the use of imperfect training transcripts, and language-specific adjustments required for recognition of Arabic and Mandarin

Three research prototype speech recognition systems are described, all of which use recently developed methods from artificial intelligence (specifically support vector machines, dynamic Bayesian networks, and maximum entropy classification) in order to implement, in the form of an automatic speech recognizer, current theories of human speech perception and phonology (specifically landmark-based speech perception, nonlinear phonology, and articulatory phonology). All three systems begin with a high-dimensional multiframe acoustic-to-distinctive feature transformation, implemented using support vector machines trained to detect and classify acoustic phonetic landmarks. Distinctive feature probabilities estimated by the support vector machines are then integrated using one of three pronunciation models: a dynamic programming algorithm that assumes canonical pronunciation of each word, a dynamic Bayesian network implementation of articulatory phonology, or a discriminative pronunciation model trained using the methods of maximum entropy classification. Log probability scores computed by these models are then combined, using log-linear combination, with other word scores available in the lattice output of a first-pass recognizer, and the resulting combination score is used to compute a second-pass speech recognition output.

The paper describes our recent efforts in exploring longer-range features and their statistical modeling techniques for speaker recognition. In particular, we describe a system that uses discriminant features from cepstral coefficients, and systems that use discriminant models from word n-grams and syllable-based NERF n-grams. These systems together with a cepstral baseline system are evaluated on the 2004 NIST speaker recognition evaluation dataset. The effect of the development set is measured using two different datasets, one from Switchboard databases and another from the FISHER database. Results show that the difference between the development and evaluation sets affects the performance of the systems only when more training data is available. Results also show that systems using longer-range features combined with the baseline result in about a 31% improvement with 1-side training over the baseline system and about a 61% improvement with 8-side training over the baseline system.

Many recent performance improvements in speaker recognition using higher-level features, as demonstrated in the NIST Speaker Recognition Evaluation (SRE) task, rely on combinations of multiple systems modeling a large variety of features. The diversity of the large set of features starting from short-term acoustic spectrum features all the way to habitual word usage from a large set of speakers in a multitude of settings (acoustic environment, speaking style, quantities of enrollment/test data) results in a challenging model combination task. In this work, we are presenting a class-based score combination technique that relies on clustering of both the target models and the test utterances in a vector space defined by a set of speaker-specific transformation parameters estimated during transcription of the talker’s speech by automatic speech recognition (ASR). We show that significant performance gains are obtained by using the first few principal components of a model transform for clustering the speaker verification trials into classes for (target speaker, test utterance) pairs, and then training a separate combiner for each class. We report results on the NIST SRE 2004 and FISHER datasets.

We present a method for speaker recognition that uses the duration patterns of speech units to aid speaker classification. The approach represents each word and/or phone by a feature vector comprised of either the durations of the individual phones making up the word, or the HMM states making up the phone. We model the vectors using mixtures of Gaussians. The speaker specific models are obtained through adaptation of a “background” model that is trained on a large pool of speakers. Speaker models are then used to score the test data; they are normalized by subtracting the scores obtained with the background model. We find that this approach yields significant perfomance improvement when combined with a state-of-the-art speaker recognition system based on standard cepstral features. Furthermore, the improvement persists even after combination with lexical features. Finally, the improvement continues to increase with longer test sample durations, beyond the test duration at which standard system accuracy level off.

We present a method to combine the standard and throat microphone signals for robust speech recognition in noisy environments. Our approach is to use the probabilistic optimum filter (POF) mapping algorithm to estimate the standard microphone clean-speech feature vectors, used by standard speech recognizers, from both microphones' noisy-speech feature vectors. A small untranscribed "stereo" database (noisy and clean simultaneous recordings) is required to train the POF mappings. In continuous-speech recognition experiments using SRI International's DECIPHER recognition system, both using artificially added noise and using recorded noisy speech, the combined-microphone approach significantly outperforms the single-microphone approach.

We describe SRI’s recognition system as used in the 2001 DARPA Speech in Noisy Environments (SPINE) evaluation. The SPINE task involves recognition of speech in simulated military environments. The task had some unique challenges, including segmentation of foreground speech from noisy background, the need for robust acoustic models to handle noisy speech, and development of language models from limited training data. In developing the SRI evaluation system for this task, we addressed each of these challenges using a combination of state-of-the-art techniques, including several types of feature normalization, model adaptation, class-based language modeling, multi-pass segmentation and recognition, and word posterior-based decoding and system combination We describe SRI’s evaluation system for the October 2001 Speech in Noisy Environments (SPINE) task. The main aim of the paper is to present the key algorithms and components of a state-of-art speech recognition system and how they were combined into a system for optimal performance. We have organized the paper as follows. First, we provide a brief introduction to the SPINE task and its challenges. We then present the key components and algorithms used in our system and how the task features guided the design of the system. We then present the results on two test sets, the dry run and the evaluation sets of the 2001 SPINE evaluation. We conclude with a discussion of the results. SPINE is a relatively new task developed by the Naval Research Laboratory (NRL) to test the state of the art of speech recognition in military noise environments. The primary challenge of the task is recognition of speech with significant amounts of background noise as found in various military environments, such as fighter jet cockpits and aircraft carrier flightdecks. The data consists of dialogs between two participants playing a battleship-like game, with recorded military noises played back into the recording environment. The players use realistic microphones and headgear (e.g., fighter pilot helmets) as appropriate for the different scenarios. The language used comprises a mix of commands, status reports, and confirmations specific to this limited domain, involving an active vocabulary of about 2000 words. More details are available at [1]. Due to its focus on noisy environments, SPINE posed some unique challenges. One of the difffculties was to segment foreground speech from the noisy background, which, in some environments, included background speech. Another challenge was to develop robust acoustic features, models, and techniques capable of recognizing the noise-degraded speech. Yet another challenge was the limited amount of training data, particluarly for training the language model. Thus, the task posed challenges not only for research, but also for system development. To solve these issues of segmentation and robust acoustic and language modeling, we drew on a number of stateof-the-art algorithms as the building blocks of our system. In the following, we describe these components and how they were integrated into a system that achieved the lowest word error rate (WER) in the 2001 SPINE evaluation.

We consider adaptation by topologically smooth transformations with applications to environment and speaker adaptation for robust speech recognition. Specifically, the tradeoff between global affine transformations that fail to capture local variation but preserve topology and local class dependent bias transformations that have more resolution but may destroy the topology of the reference model is addressed. We cast the problem of topology preservation of the reference model in an information divergence geometry framework and derive a class of alternating minimization algorithms that aims to preserve topology explicitly during adaptation.

This paper is situated in a long line of phonetic studies that seek to determine and qualify the acoustic cues humans use to identify stop place. The present study draws from a database of 1500 CV tokens of American English and their values for the acoustic features thought to be cues for stop place identification, including (1) VOT, (2) energy of the burst and release, (3) spectrum at the burst, and (4) formant transitions into the following vowel. Decision trees are used to determine the relative invariance of these acoustic features, which indicates their potential to serve as useful cues for listeners cross-contextually. Decision trees thus allow the evaluation of vocalic effects on this hierarchy of features for the purpose of guiding classic perceptual confusion studies.

Predicting the degree of nativeness of a student's utterance is an important issue in computer-aided language learning. This task has been addressed by many studies focusing on the segmental assessment of the speech signal. Toachieve improved correlations between human and automatic nativeness scores, other aspects of speechshould also be considered, such as prosody. The goal of this study is to evaluate the use of prosodic information to help predict the degree of nativeness of pronunciation, independent of the text. A supervised strategy based on human grades is used in an attempt to select promising features for this task. Preliminary results show improvements in the correlation between human and automatic scores.

The aim of the work is to develop a flexible and efficient approach to the classification of the ratio of voiced to unvoiced excitation sources in continuous speech. To achieve this aim we adopt a probabilistic neural network approach. This is accomplished by designing a multilayer perceptron classifier trained by steepest descent minimization of the least relative entropy (LRE) cost function. By using the LRE cost function we can directly output the ratio, as a probability, of excitation source, voiced to unvoiced, for a given speech segment. These output probabilities can then be used directly in other applications, such as low bit rate coders.

The paper presents the results of a series of experiments to assess the capability of a multi-microphone sub-band adaptive (MMSBA) signal processing scheme for improving the intelligibility of speech corrupted with noise recorded in an automobile. The noise corrupted speech signals were presented to 15 normal hearing volunteer subjects at various SNRs, and numbers and distributions of sub-bands. The results of listening tests were analysed to determine: (i) The direction and statistical significance (DSS) of any effect of processing treatment on intelligibility. (ii) The DSS of any effect of processing treatment on perceived quality. (iii) The DSS of any effect of the numbers of sub-bands used in sub-band processing (SBP). (iv) The DSS of any effect of the spacing of sub-bands used in SBP. In the experimental cases considered, the MMSBA scheme employing diverse sub-band processing is shown to deliver a statistically significant improvement in terms of both speech intelligibility and perceived quality when compared with both the wide-band processed and the noisy unprocessed case.

Statistics of frame-level pitch have recently been used in speaker recognition systems with good results [1, 2, 3]. Although they convey useful long-term information about a speaker's distribution of f 0 values, such statistics fail to capture information about local dynamics in intonation that characterize an individual's speaking style. In this work, we take a first step toward capturing such suprasegmental patterns for automatic speaker verification. Specifically, we model the speaker's f 0 movements by fitting a piecewise linear model to the f 0 track to obtain a stylized f 0 contour. Parameters of the model are then used as statistical features for speaker verification. We report results on 1998 NIST speaker verification evaluation. Prosody modeling improves the verification performance of a cepstrum-based Gaussian mixture model system (as measured by a task-specific Bayes risk) by 10%. 1. INTRODUCTION Statistics of frame-level pitch have recently been shown to improve the perf...

Statistics of pitch have recently been used in speaker recognition systems with good results. The success of such systems depends on robust and accurate computation of pitch statistics in the presence of pitch tracking errors. In this work, we develop a statistical model of pitch that allows unbiased estimation of pitch statistics from pitch tracks which are subject to doubling and/or halving. We first argue by a simple correlation model and empirically demonstrate by QQ plots that "clean" pitch is distributed with a lognormal distribution rather than the often assumed normal distribution. Second, we present a probabilistic model for estimated pitch via a pitch tracker in the presence of doubling/halving, which leads to a mixture of three lognormal distributions with tied means and variances for a total of four free parameters. We use the obtained pitch statistics as features in speaker verification on the March 1996 NIST Speaker Recognition Evaluation data (subset of Switchboard) and ...

We introduce a unified statistical framework for real-time signal processing with neural networks by using a recent extension of maximum likelihood (ML) estimation, partial likelihood (PL) estimation theory, which allows for (i) dependent observations, and (ii) processing of data using only the information that is available at the time of processing. For a general neural network conditional distribution model, we establish a fundamental information-theoretic relationship for PL estimation, and obtain large sample properties of PL for the general case of dependent observations. We consider applications of PL to prediction and channel equalization.

The performance degradation as a result of acoustical environment mismatch remains an important practical problem in speech recognition. The problem carries a greater significance in applications over telecommunication channels, especially with the wider use of personal communications systems such as cellular phones which invariably present challenging acoustical conditions. In this work, we introduce a vector quantization (VQ) based compensation technique which both makes use of a priori information about likely acoustical environments and adapts to the test environment to improve recognition. The technique is progressive and requires neither simultaneously recorded speech from the training and the testing environments nor EM-type batch iterations. Instead of using simultaneously recorded data, the integrity of the updated VQ codebooks with respect to acoustical classes is maintained by endowing the codebooks with a topology and using transformations which preserve the topology of the reference environment. We report results on the McCaw Cellular Corpus where the technique decreases the word error for continuous ten digit recognition of cellular hands free microphone speech with land line trained models from 23.8% to 13.6% and the speaker dependent voice calling sentence error from 16.5% to 10.6%.

We present the general formulation for the adaptive equalization by distribution learning introduced by Adali (see Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, vol.3, p.297-300, April 1994) In this framework, adaptive equalization can be viewed as a parametrized conditional distribution estimation problem where the parameter estimation is achieved by learning on a multilayer perceptron (MLP). Depending on the definition of the conditioning event set either supervised or unsupervised (blind) algorithms in either recurrent or feedforward networks result. We derive the least relative entropy (LRE) algorithm for binary data communications and analyze its statistical and dynamical properties. Particularly, we show that LRE learning is consistent and asymptotically normal by working in the partial likelihood estimation framework, and that the algorithm can always recover from convergence at the wrong extreme as opposed to the MSE based MLP's by working within an extension of the well-formed cost functions framework of Wittner and Denker (1988). We present simulation examples to demonstrate this fact.

We derive estimators for the multiple lag process, a generalization of the lag process, via spectral representations of stationary processes by complex random spectral measures. We present estimators of transfer functions for the multiple lag model with a given vector of lags and derive a multiple-lag (quadratic) coherence which can be maximized to choose the best vector of lags in the minimum mean squared error sense from a given set of lag vectors. We also demonstrate the estimation scheme by a simulation example and point out possible applications for the multiple-lag model in speech processing.

We formulate the adaptive channel equalization as a conditional probability distribution learning problem. Conditional probability density function of the transmitted signal given the received signal is parametrized by a sigmoidal perceptron. In this framework, we use relative entropy (Kullback-Leibler distance) between the true and the estimated distributions as the cost function to be minimized. The true probabilities are approximated by their stochastic estimators resulting in a stochastic relative entropy cost function. This function is well-formed in the sense of Wittner and Denker (1988), therefore gradient descent on this cost function is guaranteed to find a solution. The consistency and asymptotic normality of this learning scheme are shown via maximum partial likelihood estimation of logistic models. As a practical example, we demonstrate that the resulting algorithm successfully equalizes multipath channels.>