Recent advances in computer vision and artificial intelligence algorithms have allowed automatic extraction of metadata from video. This metadata can be represented by using the RDF/OWL ontology which can encode scene objects and their relationships in an unambiguous and well-formed manner. The encoded data can be queried using SPARQL. However, SPARQL has a steep learning curve and cannot be directly utilized by a general user for video content search. In this paper, we propose a method to bridge this gap by automatically translating user provided natural language query into an ontology-based SPARQL query for semantic video search. The proposed method consists of three major steps. First, semantically labeled training corpus of natural language query sentences is used for learning the Semantic Stochastic Context Free Grammar (SSCFG). Second, given a user provided natural language query sentence, we use the Earley-Stolcke parsing algorithm to determine the maximum likelihood semantic parsing of the query sentence. This parsing infers the semantic meaning for each word in the query sentence from which the SPARQL query is constructed. Third, the SPARQL query is executed to retrieve relevant video segments from the RDF-OWL video content database. The method is evaluated by running natural language queries on surveillance videos from maritime and land-based domains, though the framework itself is general and extensible to search videos from other domains.

This paper presents an unsupervised data driven scheme to automatically estimate the relative topology of overlapping cameras in a large visual sensor network. The proposed method learns the camera topology by employing the statistics of co-occurring observations (of moving targets) in each sensor. Since target observation data is typically very noisy in realistic scenarios, an efficient two step method is used for robust estimation of the planar homography between camera views. In the first step, modes in the co-occurrence data are learned using meanshift. In the second step, a RANSAC based procedure is used to estimate the homography from weighted co-occurrence modes. Note that the first step not only lessens the effects of noise but also reduces the search space for efficient calculation. Unlike most existing algorithms for overlapping camera calibration, the proposed method uses an update mechanism to adapt online to the changes in network topology. The method does not assume prior knowledge about the scene, target, or network properties. It is also robust to noise, traffic intensity, and the amount of overlap between the fields of view. Experiments and quantitative evaluation using both synthetic and real data are presented to support the above claims.

The fundamental matrix is a central construct in the analysis of images captured from a pair of cameras and many feature-based methods have been proposed for its computation. In this paper, we propose a direct method for estimating the fundamental matrix where the motion between the frames is small (e.g. between successive frames of a video). To achieve this, a warping function is presented for the fundamental matrix by using the brightness constancy constraint in conjunction with geometric constraints. Using this warping function, an iterative hierarchical algorithm is described to recover accurate estimates of the fundamental matrix. We present results of experimentation to evaluate the performance of the proposed approach and demonstrate improved accuracy in the computation of the fundamental matrix.

This paper presents a novel object-based video coding framework for videos obtained from a static camera. As opposed to most existing methods, the proposed method does not require explicit 2D or 3D models of objects and hence is general enough to cater for varying types of objects in the scene. The proposed system detects and tracks objects in the scene and learns the appearance model of each object online using incremental principal component analysis (IPCA). Each object is then coded using the coefficients of the most significant principal components of its learned appearance space. Due to smooth transitions between limited number of poses of an object, usually a limited number of significant principal components contribute to most of the variance in the object's appearance space and therefore only a small number of coefficients are required to code the object. The rigid component of the object's motion is coded in terms of its affine parameters. The framework is applied to compressing videos in surveillance and video phone domains. The proposed method is evaluated on videos containing a variety of scenarios such as multiple objects undergoing occlusion, splitting, merging, entering and exiting, as well as a changing background. Results on standard MPEG-7 videos are also presented. For all the videos, the proposed method displays higher Peak Signal to Noise Ratio (PSNR) compared to MPEG-2 and MPEG-4 methods, and provides comparable or better compression.