This paper presents a novel approach for pedestrian detection using oriented line scans of gradients computed from a gray level image. Three feature types are proposed that can be generated easily from oriented gradients and an effective use of integral lines and integral images. A scalable cascaded classifier is built by combining oriented gradients with the oriented line scan features in a boosting framework. The detector's performance is comparable to the state of the art results and achieves about 3 to 5 fps on 320 x 240 resolution images making the proposed method suitable for real time applications. Detector performance is also represented as PUR, percentage of uncertainty removed.

Metric learning methods, for person re-identification, estimate a scaling for distances in a vector space that is optimized for picking out observations of the same individual. This paper presents a novel approach to the pedestrian re-identification problem that uses metric learning to improve the state-of-the-art performance on standard public datasets. Very high dimensional features are extracted from the source color image. A first processing stage performs unsupervised PCA dimensionality reduction, constrained to maintain the redundancy in color-space representation. A second stage further reduces the dimensionality, using a Local Fisher Discriminant Analysis defined by a training set. A regularization step is introduced to avoid singular matrices during this stage. The experiments conducted on three publicly available datasets confirm that the proposed method outperforms the state-of-the-art performance, including all other known metric learning methods. Further-more, the method is an effective way to process observations comprising multiple shots, and is non-iterative: the computation times are relatively modest. Finally, a novel statistic is derived to characterize the Match Characteristic: the normalized entropy reduction can be used to define the 'Proportion of Uncertainty Removed' (PUR). This measure is invariant to test set size and provides an intuitive indication of performance.

Automatic video analysis from urban surveillance cameras is a fast-emerging field based on computer vision techniques. We present here a comprehensive review of the state-of-the-art computer vision for traffic video with a critical analysis and an outlook to future research directions. This field is of increasing relevance for intelligent transport systems (ITSs). The decreasing hardware cost and, therefore, the increasing deployment of cameras have opened a wide application field for video analytics. Several monitoring objectives such as congestion, traffic rule violation, and vehicle interaction can be targeted using cameras that were typically originally installed for human operators. Systems for the detection and classification of vehicles on highways have successfully been using classical visual surveillance techniques such as background estimation and motion tracking for some time. The urban domain is more challenging with respect to traffic density, lower camera angles that lead to a high degree of occlusion, and the variety of road users. Methods from object categorization and 3-D modeling have inspired more advanced techniques to tackle these challenges. There is no commonly used data set or benchmark challenge, which makes the direct comparison of the proposed algorithms difficult. In addition, evaluation under challenging weather conditions (e.g., rain, fog, and darkness) would be desirable but is rarely performed. Future work should be directed toward robust combined detectors and classifiers for all road users, with a focus on realistic conditions during evaluation.

This paper proposes and demonstrates a novel method for the detection and classification of individual vehicles and pedestrians in urban scenes. In this scenario, shadows, lights and various occlusions compromise the accuracy of foreground segmentation and hence there are challenges with conventional silhouette-based methods. 2D features derived from histograms of oriented gradients (HOG) have been shown to be effective for detecting pedestrians and other objects. However, the appearance of vehicles varies substantially with the viewing angle and local features may be often occluded. In this paper, a novel method is proposed that overcomes limitations in the use of 2D HOG. Full 3D models are used for the object categories to be detected and the feature patches are defined over these models. A calibrated camera allows an affine transform of the observation into a normalised representation from which ‘3DHOG ’ features are defined. A variable set of interest points is used in the detection and classification processes, depending on which points in the 3D model are visible. Experiments on real CCTV data of urban scenes demonstrate the proposed method. The 3DHOG feature is compared with features based on FFT and simple histograms. A baseline method using overlap between wire-frame models and motion silhouettes is also included. The results demonstrate that the proposed method achieves comparable performance. In particular, an advantage of the proposed method is that it is more robust than motion silhouettes which are often compromised in real data by variable lighting, camera quality and occlusions from other objects. 1.

Video-based methods such as tracking and appearance recognition can be used in the public transport domain to re-acquire an estimate of the whereabouts of a passenger of interest. Starting with the end user requirements, it is argued that the most appropriate performance metric with which to evaluate these methods, is to measure the information gained through their use. This is equivalent to the reduction in uncertainty about the passenger's whereabouts, after tracking and appearance-based measurements are used to update the estimates of which detected persons are likely to be the passenger of interest. The prior estimates are significantly conditioned on the statistics for the likely distributions for entry and traversal through the network: a simple model is described for this purpose. The suitability of tracking and appearance based matching methods are discussed in this context.

In this paper, model-based approaches for real-time 3-D soccer ball tracking are proposed, using image sequences from multiple fixed cameras as input. The main challenges include filtering false alarms, tracking through missing observations, and estimating 3-D positions from single or multiple cameras. The key innovations are: 1. incorporating motion cues and temporal hysteresis thresholding in ball detection; 2. modeling each ball trajectory as curve segments in successive virtual vertical planes so that the 3-D position of the ball can be determined from a single camera view; and 3. introducing four motion phases (rolling, flying, in possession, and out of play) and employing phase-specific models to estimate ball trajectories which enables high-level semantics applied in low-level tracking. In addition, unreliable or missing ball observations are recovered using spatio-temporal constraints and temporal filtering. The system accuracy and robustness are evaluated by comparing the estimated ball positions and phases with manual ground-truth data of real soccer sequences.

This paper builds on previous work to propose a meta-data standard for video surveillance. The motivation is to promote interoperability. The starting point is the set of requirements under consideration for a Multimedia Application Format. These requirements cover a description of the surveillance system and of the activity in the scene. In addition to this set, appropriate descriptions for the relation between camera and scene are also considered. To improve interoperability between systems and between components of a system, two types of restrictions are proposed. The first proposal is a restricted subset of the MPEG-7 elements that are applicable to the surveillance domain. The second proposal is to use the MPEG-7 tools to include domain-specific taxonomies to restrict the names of elements used in the semantic descriptions. Both proposals are incorporated into examples which demonstrate the use of the standard.

In this paper, we discuss the different approaches used for evaluating the results of tracking algorithms, and in particular for analysis of football (soccer) tracking results. The focus of the study is on systems with multiple static cameras. The appropriate data representation and ground truth capture methods are discussed, and evaluation measures that indicate the performance of any given automatic tracker are presented. The evaluation method is demonstrated to compare results of an implemented multi-camera tracker.

Systems that automatically assess student programming assignments have been designed and used for over forty years. Systems that objectively test and mark student programming work were developed simultaneously with programming assessment in the computer science curriculum. This article reviews a number of influential automatic assessment systems, including descriptions of the earliest systems, and presents some of the most recent developments. The final sections explore a number of directions automated assessment systems may take, presenting current developments alongside a number of important emerging e-learning specifications.

Complex scenes such as underground stations and malls are composed of static occlusion structures such as walls, entrances, columns, turnstiles and barriers. Unless this occlusion landscape is made explicit such structures can defeat the process of tracking individuals through the scene. This paper describes a method of generating the probability density functions for the depth of the scene at each pixel from a training set of detected blobs, i.e., observations of detected moving people. As the results are necessarily noisy, a regularization process is employed to recover the most self-consistent scene depth structure. An occlusion reasoning framework is proposed to enable object tracking methodologies to make effective use of the recovered depth.

In a football stadium environment with multiple overhead floodlights, many protruding shadows can be observed originating from each of the targets. To successfully track individual targets, it is essential to achieve an accurate representation of the foreground. Unfortunately, many of the existing techniques are sensitive to shadows, falsely classifying them as foreground. In this work an unsupervised learning procedure that determines the RGB colour distributions of the foreground and shadow classes of feature data is proposed. A novel skelatonisation and spatial filtering process is developed for identifying components in the foreground segmentation that are most-likely to belong to each class of feature. A pixel classification mechanism is obtained at by approximating both classes of feature data by N Gaussian parametric models. To assess our technique’s performance and reliability, a comparison is made with other published works.

In a football stadium environment with multiple overhead floodlights, many protruding shadows can be observed originating from each of the targets. To track individual targets successfully, it is essential to achieve an accurate representation of the foreground. Many existing techniques are sensitive to shadows, falsely classifying shadows as foreground. The paper presents four different techniques associated with shadow classification. Three of the classifiers originate from the review material whilst the fourth is a novel application of a real-time implementation of the k-nearest neighbour algorithm to shadow identification. To assess the performance for each of the classifiers, four quantitative evaluation metrics are proposed. Using each of the evaluation metrics, we discuss the performance of each classifier's segmentation results and assess their impact on the tracking performance.

A system is described for tracking the positions of football players during a match. The input is eight video streams from static cameras, each processed to generate measurements of the players for a multi view tracker. The single view processing includes foreground detection and a bounding box tracker designed to split measurements of merged players. In the multi-view process, the measurements, each from a different camera and matched to the same player, are fused into an overall measurement which is then used in the estimate updating. Results are demonstrated on real data.

Tracking strategies usually employ motion and appearance models to locate observations of the tracked object in successive frames. The subsequent model update procedure renders the approach highly sensitive to the inevitable observation and occlusion noise processes. In this work, two robust mechanisms are proposed which rely on knowledge about the ground plane. First a highly constrained bounding box appearance model is proposed which is determined solely from predicted image location and visual motion. Second, tracking is performed on the ground plane enabling global real-world observation and dynamic noise models to be defined. Finally, a novel auto-calibration procedure is developed to recover the image to ground plane homography by simply accumulating event observations. 1

In typical visual surveillance implementations, observations of scene objects are extracted as regions of moving pixels identified by pixel differencing based motion detection algorithms. These observations are tracked to establish their temporal coherence by updating a state vector describing the projected 2D width and height as well as image trajectory. Such an approach is particularly vulnerable to fragmentation and occlusion process as there is essentially no appearance model. The objective of this work is to develop simple but highly discriminatory models of scene objects which indirectly use the depth of the object to model its projected width and height. Rather than relying on a time-consuming, labour-intensive and expert-dependent calibration procedure to recover the full image to ground-plane homography, the system relies on a simple learning procedure involving watching several hundred objects entering, passing through and leaving the monitored view volume to recover the relationship between the projected 2D width and height of an object and its image position and visual motion.

While the ability of point distribution models to model complex deformable shapes is highly attractive, recovering shape instances is difficult in images containing multiple occluded and occluding shapes located in background clutter. The standard local refinement approach employed within the literature relies on the availability of good initial estimates. A highly efficient search strategy is presented for generating all plausible initial solutions by embedding a Kalman filter in a breadth-first search algorithm to use candidate observations extracted from the image to update the shape parameters of shape hypotheses and constrain the position of subsequent observations.

Point distribution models are useful tools for modelling the variability of particular classes of shapes. A common approach is to apply a principle component analysis to the data, to reduce the dimensionality of the representation. However, a single multivariate Gaussian model of the probability density, estimated from the principle covariances, can be substantially inaccurate. We examine how the specificity of a model can be improved by using a mixture of Gaussians, trained with the expectation-maximization algorithm, with reference to hand and vehicle profiles.

Perceptual organisation can be defined as the ability to impose structural organisation on sensory data, so as to group sensory primitives arising from a common underlying cause. Our organisational philosophy is hierarchical, with complex organisations being formed from simpler ones. In this paper, directional features extracted from Gabor responses are used as the primitives for perceptual grouping. In previous work, we extracted Gabor features in 8 directions and then applied two SOMs, thus classifying each pixel in the image within a 8x10 neuronmap, each corner of which represents one of four main directions, (horizontal, vertical, left diagonal and right diagonal). In the present work we group pixels with similar directional features, thereby detecting salient structures within an image.

We present a method for the detection of periodic structure in images. Finding a global threshold to detect a significant frequency component, and then associating it with responsible features, is fragile. The method extracts features from the original signal, and uses their autocorrelation as the basis for an adaptive filter, with which they are convolved. This convolution represents the conformity of a local neighbourhood to the dominant spectral frequencies: combined with the original feature signal, it effectively incorporates the property of periodicity into a local feature strength. A recursive implementation is suggested. The method avoids the fragility associated with global thresholds, and is shown to work on real data.

This paper discusses ego motion recovery from planar scenes with a camera view parallel to the direction of movement. Reasons for the failure of two published algorithms are considered, and a novel method a successful recovery is presented.