In this paper, an explanation, based on simple data analysis of the visual environment, is given to the automatic capture and focusing of visual attention. A very simple automatic selection mechanism, supposed to be at work already in the ascending visual signal paths, may underlie several familiar phenomena that are usually attributed to visual attention.

Huge document collections can be organized according to textual similarities by the self-organizing map (SOM) algorithm, when statistical representations of the textual contents are used as the feature vectors of the documents. In a practical experiment we mapped 6,840,568 patent abstracts onto a 1,002,240-node SOM. For the feature vectors we selected 500-dimensional random projections of the weighted word histograms.

This article describes the implementation of a system that is able to organize vast document collections according to textual similarities. It is based on the self-organizing map (SOM) algorithm. As the feature vectors for the documents statistical representations of their vocabularies are used. The main goal in our work has been to scale up the SOM algorithm to be able to deal with large amounts of high-dimensional data. In a practical experiment we mapped 6,840,568 patent abstracts onto a 1,002,240-node SOM. As the feature vectors we used 500-dimensional vectors of stochastic figures obtained as random projections of weighted word histograms.

Point densities of model (codebook) vectors in self-organizing maps (SOMs) are evaluated in this article. For a few one-dimensional SOMs with finite grid lengths and a given probability density function of the input, the numerically exact point densities have been computed. The point density derived from the SOM algorithm turned out to be different from that minimizing the SOM distortion measure, showing that the model vectors produced by the basic SOM algorithm in general do not exactly coincide with the optimum of the distortion measure. A new computing technique based on the calculus of variations has been introduced. It was applied to the computation of point densities derived from the distortion measure for both the classical vector quantization and the SOM with general but equal dimensionality of the input vectors and the grid, respectively. The power laws in the continuum limit obtained in these cases were found to be identical.

The adaptive-subspace self-organizing map (ASSOM) is a modular neural network architecture, the modules of which learn to identify input patterns subject to some simple transformations. The learning process is unsupervised, competitive, and related to that of the traditional SOM (self-organizing map). Each neural module becomes adaptively specific to some restricted class of transformations, and modules close to each other in the network become tuned to similar features in an orderly fashion. If different transformations exist in the input signals, different subsets of ASSOM units become tuned to these transformation classes.

The self-organizing map (SOM) method is a new, powerful software tool for the visualization of high-dimensional data. It converts complex, nonlinear statistical relationships between high-dimensional data into simple geometric relationships on a low-dimensional display. As it thereby compresses information while preserving the most important topological and metric relationships of the primary data elements on the display, it may also be thought to produce some kind of abstractions. The term self-organizing map signifies a class of mappings defined by error-theoretic considerations. In practice they result in certain unsupervised, competitive learning processes, computed by simple-looking SOM algorithms. Many industries have found the SOM-based software tools useful. The most important property of the SOM, orderliness of the input-output mapping, can be utilized for many tasks: reduction of the amount of training data, speeding up learning nonlinear interpolation and extrapolation, generalization, and effective compression of information for its transmission.

In this paper we have applied the Learning Vector Quantization methods, including the latest developments [1, 3, 2, 4], to the task of Finnish speaker-dependent speech recognition. The main objective was to study the effect of radically increasing the dimensionality of the context vectors. The high-dimensional feature vectors in our work represent the whole phoneme and they are formed by both averaging and concatenating shorttime feature vectors within a time domain window. Excellent results are achieved in separate phoneme classification of Finnish speech. Moreover, we also show how this method can be applied in combined labeling and segmentation of continuos speech. In this task we use an additional segmentation LVQ-codebook, and combine the information using HMMs. 1 Introduction Theoretically, a single centralized classification decision is expected to be more accurate than any fusion of distributed local decisions due to information quantization effects. That is, in order to get ...

The author introduces a variant of (supervised) learning vector quantization (LVQ) and discusses practical problems associated with the application of the algorithms. The LVQ algorithms work explicitly in the input domain of the primary observation vectors, and their purpose is to approximate the theoretical Bayes decision borders using piecewise linear decision surfaces. This is done by purported optimal placement of the class codebook vectors in signal space. As the classification decision is based on the nearest-neighbor selection among the codebook vectors, its computation is very fast. It has turned out that the differences between the presented algorithms in regard to the remaining discretization error are not significant, and thus the choice of the algorithm may be based on secondary arguments, such as stability in learning, in which respect the variant introduced (LVQ2.1) seems to be superior to the others. A comparative study of several methods applied to speech recognition is included

An adaptive algorithm combining traditional linear equalization techniques and a self-organizing neural learning algorithm is presented. The results show that the performance of the neural equalizer is insensitive to nonlinear learning distortions in dynamic discrete-signal detection. Stabilization of the self-organizing map during undistorted transmission has to be further considered to decrease the absolute mean-square error (MSE) rate of the neural equalizer. The error is due to oscillations in the self-organizing map, mainly caused by the neighborhood learning. The oscillations can be decreased by taking more samples to the map before adapting themivalues and by decreasing the neighborhood learning parameter β

The self-organized map, an architecture suggested for artificial neural networks, is explained by presenting simulation experiments and practical applications. The self-organizing map has the property of effectively creating spatially organized internal representations of various features of input signals and their abstractions. One result of this is that the self-organization process can discover semantic relationships in sentences. Brain maps, semantic maps, and early work on competitive learning are reviewed. The self-organizing map algorithm (an algorithm which order responses spatially) is reviewed, focusing on best matching cell selection and adaptation of the weight vectors. Suggestions for applying the self-organizing map algorithm, demonstrations of the ordering process, and an example of hierarchical clustering of data are presented. Fine tuning the map by learning vector quantization is addressed. The use of self-organized maps in practical speech recognition and a simulation experiment on semantic mapping are discussed.>

Self-organizing maps have a bearing on traditional vector quantization. A characteristic that makes them more closely resemble certain biological brain maps, however, is the spatial order of their responses, which is formed in the learning process. A discussion is presented of the basic algorithms and two innovations: dynamic weighting of the input signals at each input of each cell, which improves the ordering when very different input signals are used, and definition of neighborhoods in the learning algorithm by the minimal spanning tree, which provides a far better and faster approximation of prominently structured density functions. It is cautioned that if the maps are used for pattern recognition and decision process, it is necessary to fine tune the reference vectors so that they directly define the decision borders.

A microprocessor-based real-time speech recognition system is described. It is able to produce orthographic transcriptions for arbitrary words or phrases uttered in Finnish or Japanese. It can also be used as a large-vocabulary isolated word recognizer. The acoustic processor of the system transcribing speech into phonemes is based on neural network principles. The so-called phonotopic maps constructed by a self-organizing process are employed. The coarticulation effects in phonetic transcriptions are compensated by means of automatically derived rules which describe the morphology of errors at the acoustic processor output. Without applying any language model, the recognition result is correct up to 92 or even 97 per cent referring to individual letters.>

An attempt is described to improve the overall accuracy of a phoneme-based speech recognizer using a grammar that corrects error in the so-called quasiphoneme strings. These strings result when speech frames are independently labeled to phonemic classes every hundredth of a second. A syntactic symbol string-correction method, the dynamically expanding context, is then applied. The method involves a large set of context-sensitive rules that describe the errors and coarticulation effects in the quasiphoneme strings and are constructed automatically from examples. This rule set is then able to cancel errors that have similar morphology. The overall accuracy of phonemic speech recognition can be increased by 1.5% to 8% over that achievable by other means.>

The Otaniemi Speech Recognition System is an isolated word recognition system for Finnish speech. The recognition is based on the use of phonemes as phonological subunits. This characteristic together with the recognition methods used makes the system advantageous with large vocabularies, especially in relation to limited computing resources. In this paper the on-line implementation of our recognition system is described. It has been developed drawing from our experiences with the earlier versions. In addition to the on-line recognition of isolated words the system can also be used for research and development purposes. The many interactive characteristics make it a very effective tool for testing new methods and algorithms.

A new pattern recognition scheme with learning ability is introduced, and its application to the labeling of phonemes is reported. The basic classification algorithm is known as the subspace method in which classes of patterns are defined as linear vector subspaces spanned by the prototypes, and the class affiliation of an unknown pattern vector is decided by comparison of its orthogonal projections on the various subspaces. This method is here modified in two ways. In one of them, the prototype patterns are selected conditionally according to classification results obtained during training. In the second modification the subspaces are rotated in proper directions in the training procedure, depending on the classification results. By means of these methods, for the average accuracy of classification with 15 phonemic classes from continuous Finnish speech, a value of about 80 per cent was obtained.

In the recognition of spoken words a frequently applied method is to first convert the acoustic waveforms into phonemic strings which are then compared with prototype strings stored in a dictionary, using some metric. A standard method is to use dynamic programming for comparison of strings with variable length. This procedure, however, is rather slow. A recently introduced principle of string comparison is based on redundant hash addressing, and it is computationally at least an order of magnitude lighter. This method is here applied using multiple prototypes of phonemic strings for each word in the dictionary. The matching criterium thereby applied in fact corresponds to a distance-weighted k-nearest-neighbor classifier which allows length variations in strings.

Temporal segmentation of the acoustic wave-form into distinct, recognizable units is an unavoidable task in machine recognition of continuous speech. In this paper it is demonstrated that a vector space projector named Novelty Filter can be used to perform the segmentation of speech into phonemes. A Spectral decomposition of the speech waveform, performed by an analog filter bank, is continuously analyzed at regular sampling intervals for its degree of "novelty" with respect to a set of stationary prototype phonemes. The distance of the sampled vectors from the subspace spanned by all the prototype vectors is given by the Novelty Filter, and the maxima of this distance then indicate the transition regions between successive phonemes.

In this paper, the subspace method of pattern recognition, in which method classification is decided by the largest projection of an unknown pattern vector onto subspaces corresponding to different classes, is applied to the recognition of continuous Finnish speech. Classification is based on phonemic power spectra produced by an analog filter bank. When compared, e.g., with the nearest-neighbor method and the method of direction cosines, the advantages of the subspace method are an improved stability of classification and a more balanced total classification accuracy of the different phonemic classes with respect to their relative frequencies of occurrence. The efficient spanning of the subspaces as well as their mutual orthogonalization are discussed. Furthermore, the close relationship between the phonemic labeling and segmentation when using the subspace method is pointed out.

It is shown that an analog associative memory with optimal selectivity can be formed in adaptive processes which use learning algorithms related to the gradient method. The information is distributed throughout the memory by a matrix transform.

It is shown that associated pairs of vectoral items (Q(r), X(r)) can be recorded by transforming them into a matrix operator M so that a particular stored vector X(r)can be reproduced by multiplying an associated cue vector Q(r)by M. If the number of pairs does not exceed the dimension of the cue and all cue vectors are linearly independent, then the recollections are perfect replicas of the recorded items and there will be no crosstalk from the other recorded items. If these conditions are not valid, the recollections are still linear least square approximations of the X(r). The relationship of these mappings to linear estimators is discussed. These transforms can be readily implemented by linear analog systems.

A new model for associative memory, based on a correlation matrix, is suggested. In this model information is accumulated on memory elements as products of component data. Denoting a key vector by q(p), and the data associated with it by another vector x(p), the pairs (q(p), x(p)) are memorized in the form of a matrix {see the Equation in PDF File} where c is a constant. A randomly selected subset of the elements of Mxqcan also be used for memorizing. The recalling of a particular datum x(r)is made by a transformation x(r)=Mxqq(r). This model is failure tolerant and facilitates associative search of information; these are properties that are usually assigned to holographic memories. Two classes of memories are discussed: a complete correlation matrix memory (CCMM), and randomly organized incomplete correlation matrix memories (ICMM). The data recalled from the latter are stochastic variables but the fidelity of recall is shown to have a deterministic limit if the number of memory elements grows without limits. A special case of correlation matrix memories is the auto-associative memory in which any part of the memorized information can be used as a key. The memories are selective with respect to accumulated data. The ICMM exhibits adaptive improvement under certain circumstances. It is also suggested that correlation matrix memories could be applied for the classification of data.