Assuming that patterns in memory are represented as two-dimensional arrays of local features, just as they are in primary visual cortices, pattern recognition can take the form of elastic graph matching (Lades et al., 1993 ). Neural implementation of this may be based on preorganized fiber projections that can be activated rapidly with the help of control units (Wolfrum, Wolff, Lücke, & von der Malsburg, 2008 ). Each control unit governs a set of projection fibers that form part of a coherent mapping. We describe a mathematical model for the ontogenesis of the underlying connectivity based on a principle of network self-organization as described by the Häussler system (Häussler & von der Malsburg, 1983 ), modified to be sensitive to pattern similarity and to support formation of multiple mappings, each under the command of a control unit. The process takes the form of a soft-winner-take-all, where units compete for the representation of maps. We show simulations for invariant point-to-point and feature-to-feature mappings.

We present a model for the emergence of ordered fiber projections that may serve as a basis for invariant recognition. After invariance transformations are self-organized, so-called control units competitively activate fiber projections for different transformation parameters. The model builds on a well-known ontogenetic mechanism, activity-based development of retinotopy, and it employs activity blobs of varying position and size to install different transformations. We provide a detailed analysis for the case of 1D input and output fields for schematic input patterns that shows how the model is able to develop specific mappings. We discuss results that show that the proposed learning scheme is stable for complex, biologically more realistic input patterns. Finally, we show that the model generalizes to 2D neuronal fields driven by simulated retinal waves.

Recently, experience-driven unsupervised learning was shown to create combinatorial parts-based representations in a model of hierarchical visual memory. Examining the memory's ability to recognize persons from a database of natural face images, we show that an off-line, sleep-like operating regime of the memory domain results in a significant improvement of the system's ability to generalize over novel face views. Surprisingly, the positive effect turns out to be independent of synapse-specific plasticity, relying entirely on a homeostatic mechanism equalizing the intrinsic excitability levels of the units within the memory network. We show that this excitability equalization is the main cause for the improvement of memory function. A possible relation to cortical off-line memory reprocessing during certain sleep stages is discussed.

We describe a neural network able to rapidly establish correspondence between neural feature layers. Each of the network's two layers consists of interconnected cortical columns, and each column consists of inhibitorily coupled subpopulations of excitatory neurons. The dynamics of the system builds on a dynamic model of a single column, which is consistent with recent experimental findings. The network realizes dynamic links between its layers with the help of specialized columns that evaluate similarities between the activity distributions of local feature cell populations, are subject to a topology constraint, and can gate the transfer of feature information between the neural layers. The system can robustly be applied to natural images, and correspondences are found in time intervals estimated to be smaller than 100 ms in physiological terms.

This letter presents an improved cue integration approach to reliably separate coherent moving objects from their background scene in video sequences. The proposed method uses a probabilistic framework to unify bottom-up and top-down cues in a parallel, "democratic" fashion. The algorithm makes use of a modified Bayes rule where each pixel's posterior probabilities of figure or ground layer assignment are derived from likelihood models of three bottom-up cues and a prior model provided by a top-down cue. Each cue is treated as independent evidence for figure-ground separation. They compete with and complement each other dynamically by adjusting relative weights from frame to frame according to cue quality measured against the overall integration. At the same time, the likelihood or prior models of individual cues adapt toward the integrated result. These mechanisms enable the system to organize under the influence of visual scene structure without manual intervention. A novel contribution here is the incorporation of a top-down cue. It improves the system's robustness and accuracy and helps handle difficult and ambiguous situations, such as abrupt lighting changes or occlusion among multiple objects. Results on various video sequences are demonstrated and discussed. (Video demos are available at http://organic.usc.edu:8376/ approximately tangx/neco/index.html .).

We propose a general Bayesian framework to simultaneously solve the problems of visual tracking and structure-from-motion using a sequential Monte Carlo method. We demonstrate the functionality of the system on some simple examples.

Analyzing the design of networks for visual information routing is an underconstrained problem due to insufficient anatomical and physiological data. We propose here optimality criteria for the design of routing networks. For a very general architecture, we derive the number of routing layers and the fanout that minimize the required neural circuitry. The optimal fanout l is independent of network size, while the number k of layers scales logarithmically (with a prefactor below 1), with the number n of visual resolution units to be routed independently. The results are found to agree with data of the primate visual system.

We present a system for the automatic interpretation of cluttered scenes containing multiple partly occluded objects in front of unknown, complex backgrounds. The system is based on an extended elastic graph matching algorithm that allows the explicit modeling of partial occlusions. Our approach extends an earlier system in two ways. First, we use elastic graph matching in stereo image pairs to increase matching robustness and disambiguate occlusion relations. Second, we use richer feature descriptions in the object models by integrating shape and texture with color features. We demonstrate that the combination of both extensions substantially increases recognition performance. The system learns about new objects in a simple one-shot learning approach. Despite the lack of statistical information in the object models and the lack of an explicit background model, our system performs surprisingly well for this very difficult task. Our results underscore the advantages of view-based feature constellation representations for difficult object recognition problems.

Adaptive vision applications that involve rapid object identification and moving object tracking, such as in envisioned augmented reality applications, increasingly place stringent upper bounds on processing latency. In addition to the requirement for low latency, many emerging vision models and algorithms involves operations that are parallel in nature, nonlinear in functionality, and both local and non-local in structure. The resulting computational complexity places correspondingly complex demands on envisioned hardware implementations. In order to satisfy these conjoint requirements, a hybrid electronic/photonic multichip module architecture that comprises multiple layers of silicon VLSI detection and processing circuitry, coupled in the vertical dimension with dense photonic fan-out/fan-in interconnections has been investigated. The interconnections are implemented by means of 2D arrays of either multiple quantum well modulators or VCSEL that are flip-chip bonded on a pixel-by-pixel basis to the silicon VLSI detector/processor array.

We study a model of the cortical macrocolumn consisting of a collection of inhibitorily coupled minicolumns. The proposed system overcomes several severe deficits of systems based on single neurons as cerebral functional units, notably limited robustness to damage and unrealistically large computation time. Motivated by neuroanatomical and neurophysiological findings, the utilized dynamics is based on a simple model of a spiking neuron with refractory period, fixed random excitatory interconnection within minicolumns, and instantaneous inhibition within one macrocolumn. A stability analysis of the system's dynamical equations shows that minicolumns can act as monolithic functional units for purposes of critical, fast decisions and learning. Oscillating inhibition (in the gamma frequency range) leads to a phase-coupled population rate code and high sensitivity to small imbalances in minicolumn inputs. Minicolumns are shown to be able to organize their collective inputs without supervision by Hebbian plasticity into selective receptive field shapes, thereby becoming classifiers for input patterns. Using the bars test, we critically compare our system's performance with that of others and demonstrate its ability for distributed neural coding.

We present an analysis of the representation of images as the magnitudes of their transform with complex-valued Gabor wavelets. Such a representation is a model for complex cells in the early stage of visual processing and of high technical usefulness for image understanding, because it makes the representation insensitive to small local shifts. We show that if the images are band limited and of zero mean, then reconstruction from the magnitudes is unique up to the sign for almost all images.

The Gestalt principle of collinearity (and curvilinearity) is widely regarded as being mediated by the long-range connection structure in primary visual cortex. We review the neurophysiological and psychophysical literature to argue that these connections are developed from visual experience after birth, relying on coherent object motion. We then present a neural network model that learns these connections in an unsupervised Hebbian fashion with input from real camera sequences. The model uses spatiotemporal retinal filtering, which is very sensitive to changes in the visual input. We show that it is crucial for successful learning to use the correlation of the transient responses instead of the sustained ones. As a consequence, learning works best with video sequences of moving objects. The model addresses a special case of the fundamental question of what represents the necessary a priori knowledge the brain is equipped with at birth so that the self-organized process of structuring by experience can be successful.

Ullman and Basri [1] have shown theoretically, that a three-dimensional object can be represented by a linear combination of two-dimensional images of the object. But they have applied their calculations to artificially created images only, like line drawings of cars. The application to images of real objects turns out to be difficult, because a crucial point of their algorithm is the knowlegde of correspondences in the sample views. In this article we describe a biologically inspired system which automatically provides correspondences between views of a three-dimensional object. This enables us to apply Ullman and Basri's linear combination approach to images of arbitrary, real objects. We give detailed formula of our linear combinations and examples for reconstructed object views.

A framework for learning an accurate and general parametric facial model from 2D images is proposed and its application for analyzing and synthesizing facial images with pose variation is demonstrated. Our parametric piecewise linear subspace method covers a wide range of pose variation in a continuous manner through a weighted linear combination of local linear models distributed in a pose parameter space. The linear design helps to avoid typical nonlinear pitfalls such as overfitting and time-consuming learning. Experimental results show sub-degree and sub-pixel accuracy within /spl plusmn/55 degree full 3D rotation and good generalization capability over unknown head poses when learned and tested for specific persons.

Sensory integration or sensor fusion -- the integration of information from different modalities, cues, or sensors -- is among the most fundamental problems of perception in biological and artificial systems. We propose a new architecture for adaptively integrating different cues in a self-organized manner. In Democratic Integration different cues agree on a result, and each cue adapts toward the result agreed on. In particular, discordant cues are quickly suppressed and recalibrated, while cues having been consistent with the result in the recent past are given a higher weight in the future. The architecture is tested in a face tracking scenario. Experiments show its robustness with respect to sudden changes in the environment as long as the changes disrupt only a minority of cues at the same time, although all cues may be disrupted at one time or another.

A computer vision system for person-independent recognition of hand postures against complex backgrounds is presented. The system is based on the elastic graph matching, which was extended to allow for combinations of different feature types at the graph nodes.

A method of manifold representation for human faces with pose variations is proposed. Our model consists of mappings between 3D head angles and facial images separately represented in shape and texture, via sub-space models spanned by principal components (PC). Explicit mappings to and from 3D head angles are used as processes of pose estimation and transformation, respectively. Generalization capability to unknown head poses enables our model to continuously cover pose parameter space, providing high approximation accuracy. The feasibility of this model is evaluated in a number of experiments. We also propose a novel pose-invariant face recognition system using our model as the entry format for a gallery of known persons. Experimental results with 3D facial models recorded by a Cyberware scanner show that our model provides a superior recognition performance against pose variations, and that the texture synthesis process is carried out correctly.

A mechanism for the self-organized integration of different adaptive cues is proposed. In democratic integration the cues agree on a result and each cue adapts towards the result agreed upon. A technical formulation of this scheme is employed in a face tracking system. The self-organized adaptivity lends itself to suppression and recalibration of discordant cues. Experiments show that the system is robust to sudden changes in the environment as long as the changes disrupt only a minority of cues at the same time, although all cues may be affected in the long run.

The viewing hemisphere of a 3-dimensional object can be partitioned into areas of similar views, termed view bubbles. We compare two procedures of generating view bubbles: tracking of object features, i.e., Gabor wavelet responses, by utilizing the continuity of successive views and matching of features in different views, which are assumed to be independent. Both pro-cedures proved to be appropriate to detect canonical views. We found no difference concerning the size of the view bubbles, but tracking provides more precise correspondences than matching. Tracking is more appropri-ate for recognizing changes of features, whereas matching is more suitable if features of the same appearance are to be found. 1 Subject of Investigation For pose-invariant object recognition and pose estimation of objects it is necessary to uti-lize an appropriate object representation. A naive representation might consist of densly spaced views of an object’s viewing sphere. Our aim is to reduce such a “full ” represen-

A framework for integrating the processes of object recognition and knowledge acquisition is proposed and applied to solve a task of automatic video indexing based on personal appearance events in a video stream. Spatiotemporal segmentation using multiple cues and example based adaptation of a known person gallery are combined in a prototype system which demonstrated successful results in our preliminary experiments.

We present a system for recognizing human faces from single images out of a large database containing one image per person. Faces are represented by labeled graphs, based on a Gabor wavelet transform. Image graphs of new faces are extracted by an elastic graph matching process and can be compared by a simple similarity function. The system differs from Lades et al. (1993) in three respects. Phase information is used for accurate node positioning. Object-adapted graphs are used to handle large rotations in depth. Image graph extraction is based on a novel data structure, the bunch graph, which is constructed from a small set of sample image graphs.

We present a system for recognizing human faces from single images out of a large database containing one image per person. Faces are represented by labeled graphs, based on a Gabor wavelet transform. Image graphs of new faces are extracted by an elastic graph matching process and can be compared by a simple similarity function. The system differs from the preceding one (Lades et al., 1993) in three respects. Phase information is used for accurate node positioning. Object-adapted graphs are used to handle large rotations in depth. Image graph extraction is based on a novel data structure, the bunch graph, which is constructed from a small get of sample image graphs.

A system for the classification of hand postures against complex backgrounds in grey-level images is presented. The system employs elastic graph matching, which has already been successfully employed for the recognition of faces. Our system reaches 86.2% correct classification on our gallery of 239 images of ten postures against complex backgrounds. The system is robust with respect to certain variations in size of hand and shape of posture.

We present a system for the interpretation of camera images of scenes composed of several known objects with mutual occlusion. The scenes are analyzed by the recognition of the objects present and by the determination of their occlusion relations. Objects are internally represented by stored model graphs. These are formed in a semi-automatic way by showing objects against a varying background. Objects are recognized by dynamic link matching. Our experiments show that our system is very successful in analyzing cluttered scenes. The system architecture goes beyond classical neural networks by making extensive use of flexible links between units, as proposed in the dynamic link architecture. The present implementation is, however, rather algorithmic in style and is to be regarded as a pilot study that is preparing the way for a detailed implementation of the architecture.

A large attraction of neural systems lies in their promise of replacing programming by learning. A problem with many current neural models is that with realistically large input patterns learning time explodes. This is a problem inherent in a notion of learning that is based almost entirely on statistical estimation. We propose here a different learning style where significant relations in the input pattern are recognized and expressed by the unsupervised self-organization of dynamic links. The power of this mechanism is due to the very general a priori principle of conservation of topological structure. We demonstrate that style with a system that learns to classify mirror symmetric pixel patterns from single examples.

An object recognition system based on the dynamic link architecture, an extension to classical artificial neural networks (ANNs), is presented. The dynamic link architecture exploits correlations in the fine-scale temporal structure of cellular signals to group neurons dynamically into higher-order entities. These entities represent a rich structure and can code for high-level objects. To demonstrate the capabilities of the dynamic link architecture, a program was implemented that can recognize human faces and other objects from video images. Memorized objects are represented by sparse graphs, whose vertices are labeled by a multiresolution description in terms of a local power spectrum, and whose edges are labeled by geometrical distance vectors. Object recognition can be formulated as elastic graph matching, which is performed here by stochastic optimization of a matching cost function. The implementation on a transputer network achieved recognition of human faces and office objects from gray-level camera images. The performance of the program is evaluated by a statistical analysis of recognition results from a portrait gallery comprising images of 87 persons.>

A feature-based approach to face recognition in which the features are derived from the intensity data without assuming any knowledge of the face structure is presented. The feature extraction model is biologically motivated, and the locations of the features often correspond to salient facial features such as the eyes, nose, etc. Topological graphs are used to represent relations between features, and a simple deterministic graph-matching scheme that exploits the basic structure is used to recognize familiar faces from a database. Each of the stages in the system can be fully implemented in parallel to achieve real-time recognition. Experimental results for a 128*128 image with very little noise are evaluated.>

The authors present an approach to feature detection, which is a fundamental issue in many intermediate-level vision problems such as stereo, motion correspondence, image registration, etc. The approach is based on a scale-interaction model of the end-inhibition property exhibited by certain cells in the visual- cortex of mammals. These feature detector cells are responsive to short lines, line endings, corners and other such sharp changes in curvature. In addition, this method also provides a compact representation of feature information which is useful in shape recognition problems. Application to face recognition and motion correspondence are illustrated.>

A neural system is presented for invariant object recognition. Its flexibility is demonstrated with freely taken camera images of human faces. The system is an application of the dynamic link architecture, which owes its strength to an enhancement of tradition neural networks by a new kind of variable to express the hierarchical grouping of neurons. This capability is used to group primitive local feature detectors (Gabor-based wavelets) into composite feature detectors (jets) and to preserve neighborhood relationships between jets when they lose position information on the way from the image domain to the object domain. Due to the potential for grouping, objects can be represented as attributed graphs, with jets serving as attributes. Recognition is formulated as graph matching and is implemented as a topologically constrained diffusion of image-object links. A hierarchical sequence of matches, from low-frequency components of jets to high-frequency components, is used. Size invariance is achieved by interposing diffusion steps in magnification space. The system is implemented on a network of transputers

The goal of this paper is to show how to modify associative memory such that it can discriminate several stored patterns in a composite input and represent them simultaneously. Segmention of patterns takes place in the temporal domain, components of one pattern becoming temporally correlated with each other and anticorrelated with the components of all other patterns. Correlations are created naturally by the usual associative connections. In our simulations, temporal patterns take the form of oscillatory bursts of activity. Model oscillators consist of pairs of local cell populations connected appropriately. Transition of activity from one pattern to another is induced by delayed self-inhibition or simply by noise.