This paper presents a method for visual object categorization based on encoding the joint textural information in objects and the surrounding background, and requiring no segmentation during recognition. The framework can be used together with various learning techniques and model representations. Here we use this framework with simple probabilistic models and more complex representations obtained using Support Vector Machines. We prove that our approach provides good recognition performance for complex problems for which some of the existing methods have difficulties. Additionally, we introduce a new extensive database containing realistic images of animals in complex natural environments. We assess the database in a set of experiments in which we compare the performance of our approach with a recently proposed method.

We investigate modeling and recognition of arm manipulation actions of different levels of complexity. To model the process, we are using a combination of discriminative support vector machines and generative hidden Markov models. The experimental evaluation, performed with 10 people, investigates both definition and structure of primitive motions as well as the validity of the modeling approach taken.

In this work, we perform an extensive statistical evaluation for learning and recognition of object manipulation actions. We concentrate on single arm/hand actions but study the problem of modeling and dimensionality reduction for cases where actions are very similar to each other in terms of arm motions. For this purpose, we evaluate a linear and a nonlinear dimensionality reduction techniques: principal component analysis and spatio-temporal isomap. Classification of query sequences is based on different variants of Nearest Neighbor classification. We thoroughly describe and evaluate different parameters that affect the modeling strategies and perform the evaluation with a training set of 20 people.

Figure-ground segmentation and recognition are two interrelated processes. In this paper we present a method for foveated segmentation and evaluate it in the context of a binocular real-time recognition system. Segmentation is solved as a binary labeling problem using priors derived from the results ofa simplistic disparity method. Doing so we are able to cope with situations when the disparity range is very wide, situations that has rarely been considered, but appear frequently for narrow-field camera sets. Segmentation and recognition are then integrated into a system able to locate, attend to and recognise objects in typical cluttered indoor scenes. Finally, we try to answer two questions: is recognition really helped by segmentation and what is the benefit of multiple cues for recognition?

Recognition in cluttered real world scenes is a challenging problem. To find a particular object of interest within a reasonable time, a wide field of view is preferable. However, as we will show with practical experiments, robust recognition is easier if the object is foveated and subtends a considerable partof the visual field. In this paper a binocular system able to overcome these two conflicting requirements will be presented. The system consists of two sets of cameras, a wide field pair and a foveal one. From disparities a number of object hypotheses are generated. An attentional process based on hue and 3D size guides the foveal cameras towards the most salient regions. With the object foveated and segmented in 3D, recognition is performed using scale invariant features. The system is fully automised and runs at real-time speed.

The aim of this paper is to find the best representation for the appearance of surfaces with Lambertian reflectance under varying illumination. Previous work using principal component analysis (PCA) found the best sub-space to represent all images of an object under a varying point light source. We extend this to images from any illumination distribution. Specifically we calculate the bases for all configurations of a point plus ambient light source and two point light sources, as well as from a database of captured real world illumination. We also reformulate the optimization criterion used in PCA. The resulting basis, we believe has higher representability and is better for analyzing images of shaded objects. The different bases are compared on a database of images to test the representability.

Statistical background modelling and subtraction has proved to be a popular and effective class of algorithms for segmenting independently moving foreground objects out from a static background, without requiring any a priori information of the properties of foreground objects. We present two contributions on this topic, aimed towards robotics where an active head is mounted on a mobile vehicle. In periods when the vehicle's wheels are not driven, camera translation is virtually zero, and background subtraction techniques are applicable. This is also highly relevant to surveillance and video conferencing. The first part presents an efficient probabilistic framework for when the camera pans and tilts. A unified approach is developed for handling various sources of error, including motion blur, subpixel camera motion, mixed pixels at object boundaries, and also uncertainty in background stabilisation caused by noise, unmodelled radial distortion and small translations of the camera. The second contribution regards a Bayesian approach to specifically incorporate uncertainty concerning whether the background has yet been uncovered by moving foreground objects. This is an important requirement during initialisation of a system. We cannot assume that a background model is available in advance since that would involve storing models for each possible position, in every room, of the robot's operating environment.. Instead the background mode must be generated online, very possibly in the presence of moving objects.

We present a framework for calculating low-dimensional bases to represent image irradiance from surfaces with isotropic reflectance under arbitrary illumination. By representing the illumination and the bidirectional reflectance distribution function (BRDF) in frequency space, a model for the image irradiance is derived. This model is then reduced in dimensionality by analytically constructing the principal component basis for all images given the variations in both the illumination and the surface material. The principal component basis are constructed in such a way that all the symmetries (Helmholtz reciprocity and isotropy) of the BRDF are preserved in the basis functions. Using the framework we calculate a basis using a database of natural illumination and the CURET database containing BRDFs of real world surface materials.

Stereo is an important cue for visually guided robots. While moving around in the world, such a robot can use dynamic fixation to overcome limitations in image resolution and field of view. In this paper, a binocular stereo system capable of dynamic fixation is presented. The external calibration is performed continuously taking temporal consistency into consideration, greatly simplifying the process. The essential matrix, which is estimated in real-time, is used to describe the epipolar geometry. It will be shown, how outliers can be identified and excluded from the calculations. An iterative approach based on a differential model of the optical flow, commonly used in structure from motion, is also presented and tested towards the essential matrix. The iterative method will be shown to be superior in terms of both computational speed and robustness, when the vergence angles are less than about 15. For larger angles, the differential model is insufficient and the essential matrix is preferably used instead.

We present a fully automatic algorithm for estimating the projected light source direction from a single image. The requirement is that there exists a segment of an occluding contour of an object with locally Lambertian surface reflectance in the image. The algorithm consists of three stages. First a heuristic algorithm picks out potential occluding contours using color and edge information. Secondly, for each contour the light source direction is estimated using a shading model. In the final stage the results from the estimations are fused together in a Bayesian network to arrive at the most likely light source direction. The probabilistic model takes into account that the contours from the first stage might not be occluding contours. Using the same framework the contours are also classified as occluding or not. Experiments test the second stage, estimating the light source direction from an occluding contour, as well as the full algorithm.

Pursuing the goals of absolute simplicity of a detection/recognition system, a pure learning approach to background-invariance and visual 3D object detection/recognition is proposed. The approach relies on learning from examples only, and does not encode any domain knowledge (e.g. in the form of intermediate representations, or by solving segmentation or correspondence problems). To make the pure learning approach practically feasible, we propose the BW training method for teaching an object recognition system background-invariance. The method consist of pedagogically training the system, once with a black background and once with a white background. The method is formulated within the framework of support vector learning. Evaluation is performed with the Columbia Image (COIL) database, that is extended with different classes of cluttered backgrounds. Using this pure learning approach, a system is proposed that is able to perform 3D object detection/recognition successfully in real-world scenes, with varying illuminations and backgrounds. The system is able to perform this task in real-time.

In the field of computer vision, the detection and tracking of moving objects from a moving observer is a complex and computationally demanding task. Using a laser range scanner instead of a camera, the problem can be simplified dramatically. An algorithm that identifies range readings in areas that was detected earlier as free is described. This is done without incorporating any gridmaps that are inherently memory and computationally consuming. The algorithm is robust from the real-time test in a furnished living room. It is able to track a moving person walking around, while consuming only about 2% of the available processing power.

A visually guided mobile platform acting in a dynamic environment needs information, delivered from cues such as stereo and motion. In this paper a binocular stereo system, capable of dynamic vergence and ego-motion estimation, will be presented. The ability to dynamically verge the cameras is essential for visual tasks such as tracking, recognition and manipulation, since the computation of 3D geometry will be considerably simplified at fixation. We will show that the epipolar geometry of the presented system can be estimated in real-time. The number of degrees of freedom has been minimized, without affecting the flexibility of the system. Reconstructed 3D data is used in order to determine the ego-motion of the platform, at a very low computational cost. Finally, it will be shown how disparity maps and estimated ego-motion can facilitate the localization of independently moving objects.

Many visual tasks, such as recognition and visually guided manipulation, benefit from the use of binocular stereo with dynamic vergence. In order to achieve this the epipolar geometry of the two cameras has to be known. In this article we present a system capable of estimating the epipolar geometry of a stereo-head and calculating the binocular disparities in real-time producing dense disparity maps that, for example, can be used for figure-ground segmentation. The number of degrees of freedom has been minimized, without affecting the flexibility of the stereo-head, speeding up the process considerably.

We propose an approach to determine the occurrence of low-parametric qualitative models from images by a hypothesis-and-test approach based on the coincidence of multiple cues, thereby avoiding complete reconstruction of the scene. A system is presented which applies the approach to finding instances of planar surfaces, as it is important in many tasks for mobile or manipulating robots. The system uses monocularly determined L-junctions and binocular disparities. A notable feature of the approach is that it finds the most conspicuous exemplar of the model first. This property seems quite relevant for an agent using vision to guide its behaviors, since the simplest solution becomes available early on.

We present a computational model for attention. It consists of an early parallel stage with preattentive cues followed by a later serial stage, where the cues are integrated. We base the model on disparity image flow and motion. As one of the several possibilities we choose a depth-based criterion to integrate these cues, in such a way that the attention is maintained to the closest moving object. We demonstrate the technique by experiments in which a moving observer selectively mask our different moving objects in real scenes.

We present a binocular active vision system that can attend to and fixate a moving target. Our system has an open and expandable design and it forms the first steps of a long term effort towards developing an active observer using vision to interact with the environment, in particular capable of figure-ground segmentation. We also present partial real-time implementations of this system and show their performance in real-world situations together with motor control. In pursuit we particularly focus on occlusions of other targets, both stationary and moving, and integrate three cues, ego-motion, target motion and target disparity, to obtain an overall robust behavior. An active vision system must be open, expandable, and operate with whatever data are available momentarily. It must also be equipped with means and methods to direct and change its attention. This system is therefore equipped with motion detection for changing attention and pursuit for maintaining attention, both of which run concurrently.>

It is well-known that active selection of fixation points in humans is highly context and task dependent. It is therefore likely that successful computational processes for fixation in active vision should be so too. We are considering active fixation in the context of recognition of man-made objects characterized by their shapes. In this situation the qualitative shape and type of observed junctions play an important role. The fixations are driven by a grouping strategy, which forms sets of connected junctions separated from the surrounding at depth discontinuities. We have furthermore developed a methodology for rapid active detection and classification of junctions by selection of fixation points. The approach is based on direct computations from image data and allows integration of stereo and accommodation cues with luminance information. This work form a part of an effort to perform active recognition of generic objects, in the spirit of Malik and Biederman, but on real imagery rather than on line-drawings.

Fixation is the link between the observer and the events in the outside world and is a key element in the control strategy of an active vision system. The authors describe how the fixation process functions in human beings in general and discuss one of the essential components of fixation, vergence, in particular. They discuss the components of the human fixation model and through its geometry and organization describe a machine fixation model, running in real-time on a head-eye system. The authors are motivated by the need for gaze control in head-eye systems, which entails the need to clarify some fundamental relations in primate vergence movements, namely, the relations between disparity and blur and between vergence and accommodation. Following the geometry behind vergence movements, it is also suggested that the mechanism for disparity detection can use the symmetry present in the pattern of the left and right images through isolating vergence from version.>

Active vision systems can be considered as systems that integrate visual sensing and action. Sensing includes detection of actions/events and results also in specific actions/manipulations. This paper mainly addresses the basic issues in the design of a head-eye system for the study of active-purposive vision. The design complexity of such a head is defined by the activeness of the visual system. Although we have not had the motivation to exactly reproduce the biological solutions in a robot, we claim that the designer should carefully consider the solutions offered by evolution. The flexibility of the behavioral pattern of the system is constrained by the mechanical structure and the computational architecture used in the control system of the head. The purpose of the paper is to describe the mechanical structure as well as the computational architecture of the KTH-head from this perspective.

An approach is presented for deriving qualitative descriptions of contours containing structures at different (unknown) scales. The descriptions are in terms of straight arcs, curved arcs with sign of curvature, corners, and points delimiting the arcs: inflexion points and transitions from straight to curved. Furthermore, the tangents at these points are derived. The approach is based on the construction of a hierarchic family of polygons, having the scale-space property of causality; structure can only disappear as scale goes from fine to coarse. Using the principle that structures that are stable over scale represent significant properties, the features of the descriptive representations are then derived.>

We present a multi-scale representation of grey-level shape, called scale-space primal sketch, that makes explicit features in scale-space as well as the relations between features at different levels of scale. The representation gives a qualitative description of the image structure that allows for extraction of significant image structure — stable scales and regions of interest — in a solely bottom-up data-driven manner. Hence, it can be seen as preceding further processing, which can then be properly tuned. Experiments on real imagery demonstrate that the proposed theory gives perceptually intuitive results. Scale-space theory [11, 3] provides a well-founded framework for dealing with image structures, which naturally occur at different scales. According to this theory one can from a given signal generate a family of derived

The authors present: (1) a multi-scale representation of gray-level shape, called a scale-space primal sketch, which makes explicit both features in scale-space and the relations between features at different levels of scales; (2) a theory for extraction of significant image structure from this representation; and (3) applications to edge detection, histogram analysis and junction classification demonstrating how the proposed method can be used for guiding later stage processing. The representation gives a qualitative description of the image structure that allows for detection of stable scales and regions of interest in a solely bottom-up data-driven way. In other words, it generates coarse segmentation cues and can be hence seen as preceding further processing, which can then be properly tuned. The authors argue that once such information is available many other processing tasks can become much simpler. Experiments on real imagery demonstrate that the proposed theory gives perceptually intuitive results.>

A new method for smoothing images by local weighted averaging is presented. The weights are computed as strengths of links between neighbors. These link strengths are averages of the variations in gray level over paths of length 2. It is shown that the method preserves edges, thin lines, and average gray levels. Moreover, it transforms the images into slowly varying regions with sharp borders, having histograms that tend to be composed of sharp peaks.

Probabilistic relaxation labeling processes are iterative parallel schemes that use contextual information to reduce local ambiguities. The behavior of these processes can be described by examining the rates of change and entropies of the probability vectors at each iteration. Examples are given comparing three relaxation processes as applied to several basic image analysis tasks.

Three approaches to reducing errors in multispectral pixel classification were compared: 1) postprocessing (iterated reclassification based on comparison with the neighbors' classes); 2) preprocessing (iterated smoothing, by averaging with selected neighbors, prior to classification); and 3) relaxation (probabilistic classification followed by iterative probability adjustment). In experiments using a color image of a house, the relaxation approach gave markedly superior performance; relaxation eliminated 4-8 times as many errors as the other methods did.

Complex waveforms may contain peaks and valleys at many different scales, ranging from very local to very global. This correspondence describes an approach to detecting these peaks and valleys by applying simple difference operators at each point to neighborhoods having a wide range of sizes, and making simple comparisons of the outputs of these operators for various sizes and positions.

Computer transposition of matrices larger than available main storage is a nontrivial problem. Undoubtedly, the solution to this problem, suggested by Schumann is tractable from a mathematical point of view. However, some comments relating to my correspondence1seem appropriate.