This paper proposes a new image signature, called Compass Rose, which is particularly suited to optical flow computation. It is differentiable, fast to compute, and robust to additive illumination changes, translation, and fast rotation. We design a sparse flow computation system based on the invariance of the Compass Rose signatures. This is then extended to dense motion estimation by the addition of an optional diffusion step. Quantitative testing on several benchmark sequences shows the Compass Rose attains higher accuracy than the traditional flow signatures under a range of conditions. Finally, we demonstrate its application to human motion estimation, which is challenging for optical flow methods due to fast limb rotation.

This paper proposes the use of an adaptive locally oriented coordinate frame when calculating an optical flow field. The coordinate frame is aligned with the least curvature direction in a local window about each pixel. This has advantages to both fitting the flow field to the image data and in imposing smoothness constraints between neighboring pixels. In terms of fitting, robustness is obtained to a wider variety of image motions due to the extra invariance provided by the coordinate frame. Smoothness constraints are naturally propagated along image boundaries which often correspond to motion boundaries. In addition, moving objects can be efficiently segmented in the least curvature direction. We show experimentally the benefits of the method and demonstrate robustness to fast rotational motion, such as what often occurs in human motion.

The paper presents mathematical underpinnings of the locally linear embedding technique for data dimensionality reduction. It is shown that a cogent framework for describing the method is that of optimization on a Grassmann manifold. The solution delivered by the algorithm is characterized as a constrained minimizer for a problem in which the cost function and all the constraints are defined on such a manifold. The role of the internal gauge symmetry in solving the underlying optimization problem is illuminated.

A robust approach to detection and tracking of multiple moving targets from a moving camera is presented. The main novelty of this approach is that objects are represented using efficient compact form of the colour correlogram. Like previous correlograms, this encodes both spatial pattern and appearance information about the target. However it is less complex to compute, making it applicable to real time target tracking. The correlogram representation is incorporated into a particle filter tracking framework. Robustness to camera motion is obtained by identifying homographies linking adjacent frames, and using them to align corresponding image areas to a common local reference frame. We demonstrate successful detection and tracking in real life image sequences.

Tracking objects in video using the mean shift (MS) technique has been the subject of considerable attention. In this work, we aim to remedy one of its shortcomings. MS, like other gradient ascent optimization methods, is designed to find local modes. In many situations, however, we seek the global mode of a density function. The standard MS tracker assumes that the initialization point falls within the basin of attraction of the desired mode. When tracking objects in video this assumption may not hold, particularly when the target's displacement between successive frames is large. In this case, the local and global modes do not correspond and the tracker is likely to fail. A novel multibandwidth MS procedure is proposed which converges to the global mode of the density function, regardless of the initialization point. We term the procedure annealed MS, as it shares similarities with the annealed importance sampling procedure. The bandwidth of the procedure plays the same role as the temperature in conventional annealing. We observe that an over-smoothed density function with a sufficiently large bandwidth is unimodal. Using a continuation principle, the influence of the global peak in the density function is introduced gradually. In this way, the global maximum is more reliably located. Since it is imperative that the computational complexity is minimal for real-time applications, such as visual tracking, we also propose an accelerated version of the algorithm. This significantly decreases the number of iterations required to achieve convergence. We show on various data sets that the proposed algorithm offers considerable promise in reliably and rapidly finding the true object location when initialized from a distant point.

The success of any Bayesian particle filtering based tracker relies heavily on the ability of the likelihood function to discriminate between the state that fits the image well and those that do not. This paper describes a general framework for learning probabilistic models of objects for exploiting these models for tracking objects in image sequences. We use a discriminative classifier to learn models of how they appear in images. In particular, we use a support vector machine (SVM) for training, which is able to extract useful non-linear information, and thus represent more complex characteristics of the tracked object and background. This is a particular advantage when tracking deformable objects and where appearance changes due to the unstable illumination and pose occur. A by-product of the SVM training procedure is the classification function, with which the tracking problem is cast into a binary classification problem. An object detector directly using the classification function is then available. To make the tracker robust, an object detector that directly uses the classification function is combined into the tracker for object verification. This provides the capability for automatic initialisation and recovery from momentary tracking failures. We demonstrate improved robustness in image sequences.

We address the problem of seeking the global mode of a density function using the mean shift algorithm. Mean shift, like other gradient ascent optimization methods, is susceptible to local maxima, and hence often fails to find the desired global maximum. In this work, we propose a multi-bandwidth mean shift procedure that alleviates this problem, which we term annealed mean shift, as it shares similarities with the annealed importance sampling procedure. The bandwidth of the algorithm plays the same role as the temperature in annealing. We observe that the over-smoothed density function with a sufficiently large bandwidth is uni-modal. Using a continuation principle, the influence of the global peak in the density function is introduced gradually. In this way the global maximum is more reliably located. Generally, the price of this annealing-like procedure is that more iteration is required since it is imperative that the computation complexity is minimal in real-time applications such as visual tracking. We propose an accelerated version of the mean shift algorithm. Compared with the conventional mean shift algorithm, the accelerated mean shift can significantly decrease the number of iterations required for convergence. The proposed algorithm is applied to the problems of visual tracking and object localization. We empirically show on various data sets that the proposed algorithm can reliably find the true object location when the starting position of mean shift is far away from the global maximum, in contrast with the conventional mean shift algorithm that will usually get trapped in a spurious local maximum.

Visual tracking is one of the key tasks in computer vision. The particle filter algorithm has been extensively used to tackle this problem due to its flexibility. However the conventional particle filter uses system transition as the proposal distribution, frequently resulting in poor priors for the filtering step. The main reason is that it is difficult, if not impossible, to accurately model the target's motion. Such a proposal distribution does not take into account the current observations. It is not a trivial task to devise a satisfactory proposal distribution for the particle filter. In this paper we advance a general augmented particle filtering framework for designing the optimal proposal distribution. The essential idea is to augment a second filter's estimate into the proposal distribution design. We then show that several existing improved particle filters can be rationalised within this general framework. Based on this framework we further propose variant algorithms for robust and efficient visual tracking. Experiments indicate that the augmented particle filters are more efficient and robust than the conventional particle filter.

Robustly tracking moving objects in video sequences is one of the key problems in computer vision. In this paper we introduce a computationally efficient nonlinear kernel learning strategy to find a discriminative model which distinguishes the tracked object from the background. Principal component analysis and linear discriminant analysis have been applied to this problem with some success. These techniques are limited, however, by the fact that they are capable only of identifying linear subspaces within the data. Kernel based methods, in contrast, are able to extract nonlinear subspaces, and thus represent more complex characteristics of the tracked object and background. This is a particular advantage when tracking deformable objects and where appearance changes due to the unstable illumination and pose occur. An efficient approximation to kernel discriminant analysis using QR decomposition proposed by Xiong et al. (2004) makes possible real-time updating of the optimal nonlinear subspace. We present a tracking method based on this result and show promising experimental results on real videos undergoing large pose and illumination changes.

Problems requiring accurate determination of parameters from image-based quantities arise often in computer vision. Two recent, independently developed frameworks for estimating such parameters are the FNS and HEIV schemes. Here, it is shown that FNS and a core version of HEIV are essentially equivalent, solving a common underlying equation via different means. The analysis is driven by the search for a nondegenerate form of a certain generalized eigenvalue problem and effectively leads to a new derivation of the relevant case of the HEIV algorithm. This work may be seen as an extension of previous efforts to rationalize and interrelate a spectrum of estimators, including the renormalization method of Kanatani and the normalized eight-point method of Hartley.

Camera calibration requires the identification of points in an image that correspond to known locations in the scene. These are typically determined through the use of a calibration pattern designed to facilitate feature localisation. We present in this paper a novel method of generating patterns such that each subregion is individually identifiable by its cross ratio. The method aims to minimise the probability of misidentifying a subregion. A key advantage of the method is the ability to place constraints on the size of the elements constituting the pattern. This allows a calibration object to be used in a wider variety of viewing conditions, increasing the flexibility of the calibration process.

Recovery of dynamic scene properties from multiple videos usually requires the manipulation of synchronous (simultaneously captured) frames. This paper is concerned with the automated determination of this synchrony when the temporal alignment of sequences is unknown. A cost function characterising departure from synchrony is first evolved for the case in which two videos are generated by cameras that may be moving. A novel voting method is then presented for minimising the cost function in the case where the ratio of the cameras' frame rates is unknown. Experimental results indicate this relatively general approach holds promise.

Hartley's eight-point algorithm has maintained an important place in computer vision, notably as a means of providing an initial value of the fundamental matrix for use in iterative estimation methods. In this paper, a novel explanation is given for the improvement in performance of the eight-point algorithm that results from using normalized data. It is first established that the normalized algorithm acts to minimize a specific cost function. It is then shown that this cost function I!; statistically better founded than the cost function associated with the nonnormalized algorithm. This augments the original argument that improved performance is due to the better conditioning of a pivotal matrix. Experimental results are given that support the adopted approach. This work continues a wider effort to place a variety of estimation techniques within a coherent framework.

In recent work the authors proposed a wide-ranging method for estimating parameters that constrain image feature locations and satisfy a constraint not involving image data. The present work illustrates the use of the method with experiments concerning estimation of the fundamental matrix. Results are given for both synthetic and real images. It is demonstrated that the method gives results commensurate with, or superior to, previous approaches, with the advantage of being fast.

We present a novel method for estimating the fundamental matrix, a key problem arising in stereo vision. The method aims to minimise a cost function that is derived from maximum likelihood considerations. The respective minimiser turns out to be significantly more accurate than the familiar algebraic least squares technique. Furthermore, the method is identical in accuracy to a Levenberg-Marquardt minimiser, while proving simpler and faster.

A new parameter estimation method is presented, applicable to many com-puter vision problems. It operates under the assumption that the data (typi-cally image point locations) are accompanied by covariance matrices charac-terising data uncertainty. An MLE-based cost function is first formulated and a new minimisation scheme is then developed. Unlike Sampson’s method or the renormalisation technique of Kanatani, the new scheme has as its theoret-ical limit the true minimum of the cost function. It also has the advantages of being simply expressed, efficient, and unsurpassed in our comparative test-ing. 1

We consider the problem of estimating parameters of a model described by an equation of special form. Specific models arise in the analysis of a wide class of computer vision problems, including conic fitting and estimation of the fundamental matrix. We assume that noisy data are accompanied by (known) covariance matrices characterizing the uncertainty of the measurements. A cost function is first obtained by considering a maximum-likelihood formulation and applying certain necessary approximations that render the problem tractable. A Newton-like iterative scheme is then generated for determining a minimizer of the cost function. Unlike alternative approaches such as Sampson's method or the renormalization technique, the new scheme has as its theoretical limit the minimizer of the cost function. Furthermore, the scheme is simply expressed, efficient, and unsurpassed as a general technique in our testing. An important feature of the method is that it can serve as a basis for conducting theoretical comparison of various estimation approaches.

A recursive method is presented for recovering 3D object shape and camera motion under orthography from an extended sequence of video images. This may be viewed as a natural extension of both the original and the sequential factorization methods. A critical aspect of these factorization approaches is the estimation of the so-called shape space, and they may in part be characterized by the manner in which this subspace is computed. If P points are tracked through F frames, the recursive least-squares method proposed in this paper updates the shape space with complexity O(P) per frame. In contrast, the sequential factorization method updates the shape space with complexity O(P/sup 2/) per frame. The original factorization method is intended to be used in batch mode using points tracked across all available frames. It effectively computes the shape space with complexity O(FP/sup 2/) after F frames. Unlike other methods, the recursive approach does not require the estimation or updating of a large measurement or covariance matrix. Experiments with real and synthetic image sequences confirm the recursive method's low computational complexity and good performance, and indicate that it is well suited to real-time applications.

We show that the self-calibration of a stereo head corresponding points in an image pair is in certain circumstances prone to considerable error. A novel error analysis reveals that the automated determination of relative orientation and focal length is adversely affected when the cameras verge inwards a similar amount, and when the principal point locations have a horizontal error. This analysis is facilitated by the adoption of closed-form solutions for self-calibration from previous work of the authors. It is also shown that estimation of the fundamental matrix associated with a stereo head image pair is improved when a domain-specific parameterisation and associated computational techniques are adopted. Experiments conducted with such image pairs suggest that, given cognisance of sensitive configurations and adoption of the revised method of fundamental matrix estimation, robust reconstructions are attainable. This is demonstrated on the problem of metrically reconstructing a scene from two pairs of images obtained by an uncalibrated stereo head undergoing unknown ground-plane motion.

Detecting background changes in scenes containing significant numbers of moving objects has several applications in video surveillance. One important example is the detection of suspicious packages left in busy airport terminals or train stations. The paper outlines a statistical approach to automatically detecting long term changes to the stationary component of a scene, and describes a prototype system which has been used to successfully demonstrate the feasibility of this approach.

A direct method for the recovery of surface shape from image shading is presented. The method in its basic form is applicable to shading patterns obtained when a distant overhead point light-source illuminates a convex or concave Lambertian surface possessing a single extremal point at which surface height has either a local maximum or local minimum. A key quality of the method is its ability to operate without explicit prerequisite information concerning the shape of the sought-after surface. Consideration is also given to the manipulation of images of more complex surface shapes.