In this paper, we introduce a novel entropy measure, termed epoch-based entropy. This measure quantifies disorder of EEG signals both at the time level and spatial level, using local density estimation by a Hidden Markov Model on inter-channel stationary epochs. The investigation is led on a multi-centric EEG database recorded from patients at an early stage of Alzheimer's disease (AD) and age-matched healthy subjects. We investigate the classification performances of this method, its robustness to noise, and its sensitivity to sampling frequency and to variations of hyperparameters. The measure is compared to two alternative complexity measures, Shannon's entropy and correlation dimension. The classification accuracies for the discrimination of AD patients from healthy subjects were estimated using a linear classifier designed on a development dataset, and subsequently tested on an independent test set. Epoch-based entropy reached a classification accuracy of 83% on the test dataset (specificity = 83.3%, sensitivity = 82.3%), outperforming the two other complexity measures. Furthermore, it was shown to be more stable to hyperparameter variations, and less sensitive to noise and sampling frequency disturbances than the other two complexity measures.

Steady-state visually evoked potentials (SSVEP) can be elicited by a large variety of stimuli. To the best of our knowledge, the size and shape effect of stimuli has never been investigated in the literature. We study the relationship between the visual parameters (size and shape) of the stimulation and the resulting brain response. A tentative physiological interpretation is proposed and the potential of the effect in a BrainComputer Interface is outlined.

Abstract The work presents advances in the implementation of an ultrasound based silent speech interface system. Use of a portable acquisition device, a visual speech recognizer system with a language model, and real time tests with the Julius system are described. Experiments with two types of visual feature extraction are also presented. Results show that good recognition and real time performance can be obtained with a portable silent speech interface employing a language model. Index Terms : silent speech interface, visual speech recognition, vocal tract imaging, ultrasound imaging 1. Introduction A silent speech interface (SSI) is intended to enable speech communication in the absence of an intelligible acoustic signal [1]. Several experimental SSI systems have been developed using a variety of different sensors [1]. The REVOIX project at the Sigma Laboratory in Paris is building an SSI meant to restore the voices of speech-impaired individuals in real-time. The technique chosen for REVOIX is to drive a recognizer-synthesizer system using ultrasound and video images of the tongue and lips. The REVOIX SSI thus consists of three modules operating sequentially: (1) an acquisition module to record simultaneous ultrasound and visual images of the vocal tract; (2) a word-level visual speech recognizer that uses Hidden Markov Models trained on features extracted from these images (HTK toolkit [7]), rather than from acoustic features; and (3) a speech synthesizer. To be genuinely useful, such a device will ultimately have to be lightweight, have good recognition and synthesis performance, and operate in real time. In this report, we build upon the groundwork laid in earlier research [2-6] by:  Introducing a new, portable acquisition system;  Comparing different types of visual feature extraction;  Introducing the use of a language model to improve the recognition accuracy;  Experimenting with a real time implementation of the recognition using the Julius system. Our results show that it is possible to obtain good recognition and real time performance using a portable SSI system employing a language model. The visual speech acquisition system and the acquired corpora are described in Section 2 and 3. In Section 4, two visual speech feature extraction techniques, namely the EigenTongues/EigenLips and the Discrete Cosine Transform (DCT), are presented. The experimental results are given in Section 5. Conclusions are drawn in Section 6.

Recent improvements are presented for phonetic decoding of continuous-speech from ultrasound and optical observations of the tongue and lips in a silent speech interface application. In a new approach to this critical step, the visual streams are modeled by context-dependent multi-stream Hidden Markov Models (CD-MSHMM). Results are compared to a baseline system using context-independent modeling and a visual feature fusion strategy, with both systems evaluated on a one-hour, phonetically balanced English speech database. Tongue and lip images are coded using PCA-based feature extraction techniques. The uttered speech signal, also recorded, is used to initialize the training of the visual HMMs. Visual phonetic decoding performance is evaluated successively with and without the help of linguistic constraints introduced via a 2.5k-word decoding dictionary. Index Terms: silent speech interface, visual speech recognition, multi-stream modeling 1.

This article presents a framework for a phonetic vocoder driven by ultrasound and optical images of the tongue and lips for a “silent speech interface” application. The system is built around an HMM-based visual phone recognition step which provides target phonetic sequences from a continuous visual observation stream. The phonetic target constrains the search for the optimal sequence of diphones that maximizes similarity to the input test data in visual space subject to a unit concatenation cost in the acoustic domain. The final speech waveform is generated using “Harmonic plus Noise Model” synthesis techniques. Experimental results are based on a onehour continuous speech audiovisual database comprising ultrasound images of the tongue and both frontal and lateral view of the speaker’s lips.

Latest results on continuous speech phone recognition from video observations of the tongue and lips are described in the context of an ultrasound-based silent speech interface. The study is based on a new 61-minute audiovisual database containing ultrasound sequences of the tongue as well as both frontal and lateral view of the speaker’s lips. Phonetically balanced and exhibiting good diphone coverage, this database is designed both for recognition and corpus-based synthesis purposes. Acoustic waveforms are phonetically labeled, and visual sequences coded using PCA-based robust feature extraction techniques. Visual and acoustic observations of each phonetic class are modeled by continuous HMMs, allowing the performance of the visual phone recognizer to be compared to a traditional acoustic-based phone recognition experiment. The phone recognition confusion matrix is also discussed in detail.

This paper addresses the problem of the optimal design of numerical experiments for the construction of nonlinear surrogate models. We describe a new method, called learner disagreement from experiment resampling (LDR), which borrows ideas from active learning and from resampling methods: the analysis of the divergence of the predictions provided by a population of models, constructed by resampling, allows an iterative determination of the point of input space, where a numerical experiment should be performed in order to improve the accuracy of the predictor. The LDR method is illustrated on neural network models with bootstrap resampling, and on orthogonal polynomials with leave-one-out resampling. Other methods of experimental design such as random selection and D-optimal selection are investigated on the same benchmark problems.

The article compares two approaches to the description of ultrasound vocal tract images for application in a "silent speech interface," one based on tongue contour modeling, and a second, global coding approach in which images are projected onto a feature space of Eigentongues. A curvature-based lip profile feature extraction method is also presented. Extracted visual features are input to a neural network which learns the relation between the vocal tract configuration and line spectrum frequencies (LSF) contained in a one-hour speech corpus. An examination of the quality of LSFs derived from the two approaches demonstrates that the Eigemongues approach has a more efficient implementation and provides superior results based on a normalized mean squared error criterion.

The article describes a video-only speech recognition system for a “silent speech interface” application, using ultrasound and optical images of the voice organ. A one-hour audiovisual speech corpus was phonetically labeled using an automatic speech alignment procedure and robust visual feature extraction techniques. HMM-based stochastic models were estimated separately on the visual and acoustic corpus. The performance of the visual speech recognition system is compared to a traditional acoustic-based recognizer.

The feasibility of a silent speech interface using ultrasound (US) imaging and lip profile video is investigated by examining the quality of line spectral frequencies (LSF) derived from the image sequences. It is found that the data do not at present allow reliable identification of silences and fricatives, but that LSF's recovered from vocalized passages are compatible with the synthesis of intelligible speech

"Jacobian Conditioning Analysis for Model Validation" by Rivals and Personnaz in this issue is a comment on Monari and Dreyfus (2002). In this reply, we disprove their claims. We point to flawed reasoning in the theoretical comments and to errors and inconsistencies in the numerical examples. Our replies are substantiated by seven counterexamples, inspired by actual data, which show that the comments on the accuracy of the computation of the leverages are unsupported and that following the approach they advocate leads to discarding valid models or validating overfitted models.

We present a novel approach to dealing with overfitting in black box models. It is based on the leverages of the samples, that is, on the influence that each observation has on the parameters of the model. Since overfitting is the consequence of the model specializing on specific data points during training, we present a selection method for nonlinear models based on the estimation of leverages and confidence intervals. It allows both the selection among various models of equivalent complexities corresponding to different minima of the cost function (e.g., neural nets with the same number of hidden units) and the selection among models having different complexities (e.g., neural nets with different numbers of hidden units). A complete model selection methodology is derived.

Discrete-time models of complex nonlinear processes, whether physical, biological, or economical, are usually under the form of systems of coupled difference equations. In analyzing such systems, one of the first tasks is to find a state-space description of the process—that is, a set of state variables and the associated state equations. We present a methodology for finding a set of state variables and a canonical representation of a class of systems described by a set of recurrent discrete-time, time-invariant equations. In the field of neural networks, this is of special importance since the application of standard training algorithms requires the network to be in a canonical form. Several illustrative examples are presented.

Multi-class classification problems can be efficiently solved by partitioning the original problem into sub-problems involving only two classes: for each pair of classes, a (potentially small) neural network is trained using only the data of these two classes. We show how to combine the outputs of the two-class neural networks in order to obtain posterior probabilities for the class decisions. The resulting probabilistic pairwise classifier is part of a handwriting recognition system which is currently applied to check reading. We present results on real world data bases and show that, from a practical point of view, these results compare favorably to other neural network approaches.

The paper first summarizes a general approach to the training of recurrent neural networks by gradient-based algorithms, which leads to the introduction of four families of training algorithms. Because of the variety of possibilities thus available to the "neural network designer," the choice of the appropriate algorithm to solve a given problem becomes critical. We show that, in the case of process modeling, this choice depends on how noise interferes with the process to be modeled; this is evidenced by three examples of modeling of dynamical processes, where the detrimental effect of inappropriate training algorithms on the prediction error made by the network is clearly demonstrated.

We present a model of the specialist olfactory system of selected moth species and the cockroach. The model is built in a semirandom fashion, constrained by biological (physiological and anatomical) data. We propose a classification of the response patterns of individual neurons, based on the temporal aspects of the observed responses. Among the observations made in our simulations a number relate to data about olfactory information processing reported in the literature; others may serve as predictions and as guidelines for further investigations. We discuss the effect of the stochastic parameters of the model on the observed model behavior and on the ability of the model to extract features of the input stimulation. We conclude that a formal network, built with random connectivity, can suffice to reproduce and to explain many aspects of olfactory information processing at the first level of the specialist olfactory system of insects.

The paper proposes a general framework that encompasses the training of neural networks and the adaptation of filters. We show that neural networks can be considered as general nonlinear filters that can be trained adaptively, that is, that can undergo continual training with a possibly infinite number of time-ordered examples. We introduce the canonical form of a neural network. This canonical form permits a unified presentation of network architectures and of gradient-based training algorithms for both feedforward networks (transversal filters) and feedback networks (recursive filters). We show that several algorithms used classically in linear adaptive filtering, and some algorithms suggested by other authors for training neural networks, are special cases in a general classification of training algorithms for feedback networks.

The supervised training of feedforward neural networks is often based on the error backpropagation algorithm. The authors consider the successive layers of a feedforward neural network as the stages of a pipeline which is used to improve the efficiency of the parallel algorithm. A simple placement rule is used to take advantage of simultaneous executions of the calculations on each layer of the network. The analytic expressions show that the parallelization is efficient. Moreover, they indicate that the performance of this implementation is almost independent of the neural network architecture. Their simplicity assures easy prediction of learning performance on a parallel machine for any neural network architecture. The experimental results are in agreement with analytical estimates.

The definition of the requirements for the design of a neural network associative memory, with on-chip training, in standard digital CMOS technology is addressed. Various learning rules that can be integrated in silicon and the associative memory properties of the resulting networks are investigated. The relationships between the architecture of the circuit and the learning rule are studied in order to minimize the extra circuitry required for the implementation of training. A 64-neuron associative memory with on-chip training has been manufactured, and its future extensions are outlined. Beyond the application to the specific circuit described, the general methodology for determining the accuracy requirements can be applied to other circuits and to other autoassociative memory architectures.

It is shown that neural network classifiers with single-layer training can be applied efficiently to complex real-world classification problems such as the recognition of handwritten digits. The STEPNET procedure, which decomposes the problem into simpler subproblems which can be solved by linear separators, is introduced. Provided appropriate data representations and learning rules are used, performance comparable to that obtained by more complex networks can be achieved. Results from two different databases are presented: an European database comprising 8700 isolated digits and a zip code database from the US Postal Service comprising 9000 segmented digits. A hardware implementation of the classifier is briefly described.

The proposed implementation is guaranteed to exhibit the same convergence behavior as the serial algorithm. Two models of parallelization, depending on the value of the temperature, are introduced and statistical models which can predict the speedup for any problem (as a function of the acceptance rate and of the number of processors), are derived. The performances are evaluated on a simple placement problem with a transputer-based network, and the models are compared with experiments.>

This paper presents the results of a systematic investigation of the thermodynamic ("simulated annealing") method applied to the placement of rectangular blocks on a chip. A new presentation of the fundamental ideas underlying this technique is proposed. It is shown that the analogies with physics, which have been at the origin of the method, may be partially forgotten, but that they are still useful to understand some results. Several simple examples are investigated, and the influence of various parameters is studied. Typical complex industrial applications are subsequently presented. Finally, an interactive implementation of the thermodynamic optimization algorithm, based on the results of the present investigation, is proposed.