We propose a video content identification that uses the Viterbi algorithm for identifying unknown video content from a database of reference content. Both the source video in the database and exact frame index therein of each query frame are determined by our method. The Viterbi algorithm is effective for enforcing temporal consistency over frame-level hypotheses based on matching frame fingerprints. We have introduced an “unknown” state in the Viterbi state model to handle both missed detections and non-existent matches. Another novelty of our approach is the use of frame time stamps for computing transition probabilities between frames. This makes our method capable of handling even extreme frame rate changes between query and reference videos. We have provided experimental results demonstrating the effectiveness of the proposed method. This approach may conceivably be used for audio content identification as well.

Segmentation of moving objects in video sequences has many applications such as video surveillance, traffic monitoring, and object-based video coding. In this work, we propose a novel algorithm that separates locally moving objects (foreground) in a video from a globally moving background using both temporal and spatial contexts. The algorithm consists of four stages: frame alignment, pixel alignment, consensus filtering, and spatio-temporal refinement. The primary contributions of our work are the use of a neighborhood-based similarity metric for pixel alignment and a simple yet efficient spatio-temporal refinement method. Results are shown to be better than those of state-of-art median filtering-based segmentation algorithms.

In 3D video (3DV) applications, a reduced number of views plus depth maps are transmitted or stored. When there is a need to render virtual views in between the actual views, the technique of depth image based rendering (DIBR) can be used to generate the intermediate views. To address the problem of noisy depth information in 3DV systems, we propose novel methods that can be easily incorporated into DIBR to improve synthesized image quality. These include: (1) a heuristic scheme with adaptive spatting that blends multiple warped reference pixels based on their depth, warped pixel positions and camera parameters; (2) an approximation of the first scheme with up-sampling for fast processing; (3) boundary only splatting; and (4) view weighting based on hole distribution. Experiment results show that the proposed methods can improve synthesis quality significantly.

Captions are text or logos superimposed on videos during a postproduction process. Caption detection in videos is useful for a variety of applications. For many applications, temporal consistency and stability is very important. Most of the prior work adopts certain post-processing procedures to smooth detected caption bounding boxes over time. Although these approaches mitigate the effect of the temporal inconsistency problem, they are unable to eliminate the problem. In this paper, we present a new caption detection algorithm that detects the 3D bounding boxes of caption regions in spatiotemporal volume space. 2D bounding boxes are then created by slicing the 3D bounding boxes. Since all the 2D bounding boxes corresponding to a caption area are sliced from one 3D bounding box, they are identical over time, thus ensuring temporal consistency of the result. The experiment results show that our new approach not only generates temporally consistent results but also results in higher detection accuracy.

A method is proposed for reducing the visibility of "contour artifacts," i.e., false contours resulting from color quantization in digital images. The method performs a multiscale analysis on the neighborhood of each pixel, determines the presence and scale of contour artifacts, and probabilistically dithers (perturbs) the color of the pixel. The overall effect is to "break down" the false contours, making them less visible. The proposed method may be used to reduce contour artifacts at the same bit depth as the input image or at higher bit depths. The contour artifact detection mechanism ensures that artifact-free regions remain unaffected during the process.

High-quality digital video filtering applications pose interesting challenges. Noise reduction cannot come at the expense of losing picture detail or introducing post-filtering artifacts. We address these issues by proposing an adaptive spatio-temporal method for reducing noise in digital video. The novelty of this method is that it aims to achieve an "even" amount of noise reduction at each pixel while preserving picture detail. The emphasis on even noise reduction helps reduce post-filtering artifacts arising from uneven filtering. We further propose an algorithm for automatically estimating the key filtering parameters, given two intuitive high-level inputs, filtering strength and a tolerance value. The high-level inputs provide user-friendly "knobs" that could be adjusted to obtain the desired filtering performance.

A method is proposed for reducing the visibility of "banding artifacts," i.e. false contours resulting from color quantization in digital images. The method performs a multi-scale analysis on the neighborhood of each pixel, determines the presence and scale of banding artifacts, and probabilistically dithers (perturbs) the color of the pixel. The overall effect is to "break down" the false contours making them less visible. The proposed method may be used to reduce banding artifacts at the same bit depth as the input image or at higher bit depths. The banding detection mechanism ensures that artifact-free regions remain unaffected during the process.

We propose the use of texture motifs, or characteristic spatially recurrent patterns, for modeling and detecting geospatial objects. A method is proposed for learning a texture-motif model from object examples and detecting objects based on the learned model. The model is learned in a two-layered framework: the first learns the constituent "texture elements" of the motif and, the second, the spatial distribution of the elements. In the experimental session, we demonstrate the model training and selection methodology for objects given a set of training examples. The utility of such models for detecting the presence or absence of geospatial objects in large aerial image datasets comprising tens of thousands of image tiles is then emphasized

In this paper we present a new physically motivated curve descriptor based on the solution of Helmholtz’s equation. The descriptor satisfies the six principles set by MPEG-7: it has a good retrieval accuracy, it is compact, it can be applied in general contexts, it has a reasonable computational complexity, it is robust and provides an hierarchical representation of the curve from coarse to fine. Moreover this descriptor generalizes straightforwardly to three dimensional surfaces. We tested the performance of the descriptor in the context of affine invariant curve matching using the multiview curve dataset (MCD), which consists of 40 curves (extracted from the MPEG-7 shape dataset) imaged under 14 different points of view. The results we obtained show that the proposed descriptor satisfies the MPEG-7 requirements and presents some advantages over some of the commonly used curve descriptors. 1

In this paper, we propose, an affine-invariant-method for describing and matching curves. This is important since affine transformations are often used to model perspective distortions. More specifically, we propose a new definition of the shape of a curve that characterizes a curve independently of the effects introduced by affine distortions. By combining this definition with a rotation-invariant shape descriptor, we show how it is possible to describe a curve in an intrinsically affine-invariant manner. To validate our procedure we built a database of shapes subject to perspective distortions and plotted the precision-recall curve for this dataset. Finally an application of our method is shown in the context of wide baseline matching.

Texture has been recognized as an important visual primitive in image analysis. A widely used texture descriptor, which is part, of the MPEG-7 standard, is that computed using multiscale Gabor filters. The high dimensionality and computational complexity of this descriptor adversely affect the efficiency of content-based retrieval systems. We propose a modified texture descriptor that has comparable performance, but with nearly half the dimensionality and less computational expense. This gain is based on a claim that the distribution of (absolute values of) filter outputs have a strong tendency to be Rayleigh. Experimental results show that the dimensionality can be reduced by almost 50%, with a tradeoff of less than 3% on the error rate. Furthermore, it is easy to compute the new feature using the old, one, without having to repeat the computationally expensive filtering step. We also propose a new normalization method that improves similarity retrieval and indexing efficiency.

This work presents a novel approach to object localization in complex imagery. In particular, the spatial extents of objects characterized by distinct spatial signatures at multiple scales are estimated by using statistical models to control a simple region growing process. Texture motifs are used to model the spatial signatures at the smallest, or pixel, scale. Markov random fields are used to model the spatial signatures at the larger, or motif, scale. These models are used to iteratively expand a bounding box to approximate the spatial extent of an object. The approach is applied to localizing geo-spatial objects in high-resolution panchromatic aerial imagery.

A canonical model is proposed for object classes in aerial images. This model is motivated by the observation that geographic regions of interest are characterized by collections of texture motifs corresponding to geographic processes. Furthermore, the spatial arrangement of the motifs is an important discriminating characteristic. In our approach, the states of a hidden Markov model (HMM) correspond to the texture motifs and the state transitions correspond to the spatial arrangement of the motifs. A one-dimensional approach reduces the computational complexity. The model is shown to be effective in characterizing objects of interest in spatial datasets in terms of their underlying texture motifs. The potential of the model for identifying the classes of unlabeled objects is demonstrated.

What are the natural features of handwritten characters and how to arrive at them automatically? We apply independent components analysis on handwritten characters. Independent components analysis extracts the underlying statistically independent signals from a mixture of them. We expect strokes to be the independent components of handwritten characters. Our findings show that stroke-like features emerge as a result of the analysis confirming the above intuition. This finding is significant since it gives automatic procedures for extracting stroke-like features from multilingual character data sets. We use these features for handwritten digit recognition using a very simple classifier. The classifier is chosen to be simple so that the quality of the input feature set can be evaluated. The recognition results indicate that the features arrived at by independent component analysis are useful.