Urban environments present unique challenges from the perspective of surveillance and security. Threat activity in urban environments tends to be very similar to background activity, while the volume of activity is often very high. The widespread geographical area presents issues from the perspective of response. These characteristics of urban environments create challenges to traditional applications of video analytics technologies and opens up opportunities for novel approaches. This paper explores the applicability of video analytics in various scenarios presented in urban surveillance situations. We also describe novel technical solutions to some of the challenges of urban surveillance.

Surveillance video is used in two key modes, watching for known threats in real-time and searching for events of interest after the fact. Typically, real-time alerting is a localized function, e.g. airport security center receives and reacts to a “perimeter breach alert”, while investigations often tend to encompass a large number of geographically distributed cameras like the London bombing, or Washington sniper incidents. Enabling effective search of surveillance video for investigation & preemption, involves indexing the video along multiple dimensions. This paper presents a framework for surveillance search which includes, video parsing, indexing and query mechanisms. It explores video parsing techniques which automatically extract index data from video, indexing which stores data in relational tables, retrieval which uses SQL queries to retrieve events of interest and the software architecture that integrates these technologies.

We describe a set of tools for retail analytics based on a combination of video understanding and transaction-log. Tools are provided for loss prevention (returns fraud and cashier fraud), store operations (customer counting) and merchandising (display effectiveness). Results are presented on returns fraud and customer counting.

Pervasive sensor based systems are transforming Information Technology systems from being transactional in nature to being observational in nature. Observational systems are inherently distributed and capture information at a much finer grain of space and time. Enabling and building such systems also poses many technology challenges, extracting information from sensor signals, indexing and searching sensor meta-data, data mining and scalability. In this paper we use S3: the IBM smart surveillance system as an example of an observational system to explore several of these issues through real world deployment examples.

As smart surveillance technology becomes a critical component in security infrastructures, the system architecture assumes a critical importance. This paper considers the example of smart surveillance in an airport environment. We start with a threat model for airports and use this to derive the security requirements. These requirements are used to motivate an open-standards based architecture for surveillance. We discuss the critical aspects of this architecture and its implementation in the IBM S3 smart surveillance system. Demo results from a pilot deployment in Hawthorne, NY are presented.

Presents a method, based on the properties of vector fields, for the estimation of a set of symbolic descriptors (node, saddle, star-node, improper-node, center, and spiral) from linear orientation fields. Planar first-order phase portraits are used to model the linear orientation fields. A weighted linear estimator is developed to estimate linear phase portraits, using only the flow orientation. A classification scheme for planar first-order phase portraits, based on their local properties: curl, divergence, and deformation is developed. The authors present results of experiments on noise-added synthetic flow patterns and real oriented textures.>

A method based on the properties of orientation fields is presented for the estimation of a set of symbolic descriptors from nondegenerate linear orientation fields, modeled by two-dimensional first-order phase portraits. It was previously shown that flow orientation is sufficient to characterize a flow pattern and locate its critical point position. A linear estimator is designed to estimate nondegenerate phase portraits. A classification scheme based on their local properties-curl, divergence, and deformation-is developed. Some experimental results are reported.>

Phase portraits are a powerful mathematical model for describing oriented textures. An isotangent-based approach is presented which is a linear formulation to the problem, to locate the critical points and compute the parameter sets of this model for the nonsingular two-dimensional first-order phase portraits. The authors classify flow patterns by Jordan canonical forms of the characteristic matrix made up of the estimated parameters. For these systems, they prove that all the isotangent curves are straight lines which intersect at a critical point. They also apply least median of squares (LMS) estimators to find the isotangent lines and locate the critical point. A linear regression technique is used to estimate the parameters of the two-dimensional first-order phase portrait of a given flow pattern. Results of applying the algorithm to synthetic and real images are presented.>