Feature selection and extraction is a key operation in video analysis for achieving a higher level of abstraction. In this paper, we introduce a general framework to extract a new spatio-temporal feature that represents the directions in which a video is regular, i.e., the pixel appearances change the least. We propose to model the directions of regular variations with a 3-D vector field, which is referred to as spatio-temporal regularity flow (SPREF). SPREF vectors are designed to have three cross-sectional parallel components Fx, Fy, and Ftfor convenient use in different applications. They are estimated using all the frames simultaneously by minimizing an energy functional formulated according to its definition. In this paper, we first introduce translational SPREF (T-SPREF) and then extend our framework to affine SPREF (A-SPREF). The successful use of SPREF in a few applications, including object removal, video inpainting, and video compression, is also demonstrated

We propose a new framework in wavelet video coding to improve the compression rate by exploiting the spatiotemporal regularity of the data. A sequence of images creates a spatiotemporal volume. This volume is said to be regular along the directions in which the pixels vary the least, hence the entropy is the lowest. The wavelet decomposition of regularized data results in a fewer number of significant coefficients, thus yielding a higher compression rate. The directions of regularity of an image sequence depend on both its motion content and spatial structure. We propose the representation of these directions by a 3-D vector field, which we refer to as the spatiotemporal regularity flow (SPREF). SPREF uses splines to approximate the directions of regularity. The compactness of the spline representation results in a low storage overhead for SPREF, which is a desired property in compression applications. Once SPREF directions are known, they can be converted into actual paths along which the data is regular. Directional decomposition of the data along these paths can be further improved by using a special class of wavelet basis called the 3-D orthonormal bandelet basis. SPREF -based video compression not only removes the temporal redundancy, but it also compensates for the spatial redundancy. Our experiments on several standard video sequences demonstrate that the proposed method results in higher compression rates as compared to the standard wavelet based compression.

This paper proposes a new technique in wavelet video compression that exploits the spatiotemporal regularity of the video. A sequence of frames is said to be regular along the directions in which the pixels vary the least. The directions of regularity of a sequence depend on both its motion content and its spatial structure. We model these directions by a 3D vector field, referred as the structural flow. This flow determines the paths of regularity along which the entropy of the data is smaller. We use these paths to construct a special class of wavelet basis, i.e., the 3D orthonormal bandelet basis for the directional decomposition of the sequence. Our experiments on several standard video sequences demonstrate the significant improvement in compression compared to the standard wavelet video coding.

In this paper, we present a wide area surveillance system that detects, tracks and classifies moving objects across multiple cameras. At the single camera level, tracking is performed using a voting based approach that utilizes color and shape cues to establish correspondence. The system uses the single camera tracking results along with the relationship between camera field of view (FOV) boundaries to establish correspondence between views of the same object in multiple cameras. To this end, a novel approach is described to find the relationships between the FOV lines of cameras. The proposed approach combines tracking in cameras with overlapping and/or non-overlapping FOVs in a unified framework, without requiring explicit calibration. The proposed algorithm has been implemented in a real time system. The system uses a client-server architecture and runs at 10 Hz with three cameras.