A method for unsupervised segmentation of color-texture regions in images and video is presented. This method, which we refer to as JSEG, consists of two independent steps: color quantization and spatial segmentation. In the first step, colors in the image are quantized to several representative classes that can be used to differentiate regions in the image. The image pixels are then replaced by their corresponding color class labels, thus forming a class-map of the image. The focus of this work is on spatial segmentation, where a criterion for "good" segmentation using the class-map is proposed. Applying the criterion to local windows in the class-map results in the "J-image," in which high and low values correspond to possible boundaries and interiors of color-texture regions. A region growing method is then used to segment the image based on the multiscale J-images. A similar approach is applied to video sequences. An additional region tracking scheme is embedded into the region growing process to achieve consistent segmentation and tracking results, even for scenes with nonrigid object motion. Experiments show the robustness of the JSEG algorithm on real images and video.

A compact color descriptor and an efficient indexing method for this descriptor are presented. The target application is similarity retrieval in large image databases using color. Colors in a given region are clustered into a small number of representative colors. The feature descriptor consists of the representative colors and their percentages in the region. A similarity measure similar to the quadratic color histogram distance measure is defined for this descriptor. The representative colors can be indexed in the three-dimensional (3-D) color space thus avoiding the high-dimensional indexing problems associated with the traditional color histogram. For similarity retrieval, each representative color in the query image or region is used independently to find regions containing that color. The matches from all of the query colors are then combined to obtain the final retrievals. An efficient indexing scheme for fast retrieval is presented. Experimental results show that this compact descriptor is effective and compares favorably with the traditional color histogram in terms of overall computational complexity.

Peer group image processing identifies a "peer group" for each pixel and then replaces the pixel intensity with the average over the peer group. Two parameters provide direct control over which image features are selectively enhanced: area (number of pixels in the feature) and window diameter (window size needed to enclose the feature). A discussion is given of how these parameters determine which features in the image are smoothed or preserved. We show that the Fisher discriminant can be used to automatically adjust the peer group averaging (PGA) parameters at each point in the image. This local parameter selection allows smoothing over uniform regions while preserving features like corners and edges. This adaptive procedure extends to multilevel and color forms of PGA. Comparisons are made with a variety of standard filtering techniques and an analysis is given of computational complexity and convergence issues.

In this work, a new approach to fully automatic color image segmentation, called JSEG, is presented. First, colors in the image are quantized to several representing classes that can be used to differentiate regions in the image. Then, image pixel colors are replaced by their corresponding color class labels, thus forming a class-map of the image. A criterion for "good" segmentation using this class-map is proposed. Applying the criterion to local windows in the class-map results in the "J-image", in which high and low values correspond to possible region boundaries and region centers, respectively. A region growing method is then used to segment the image based on the multi-scale J-images. Experiments show that JSEG provides good segmentation results on a variety of images.

In this work, an efficient low-dimensional color indexing scheme for region-based image retrieval is presented. The colors in each image region are first quantized so that only a small number of cluster centroids are needed to represent the region color information. The proposed color feature descriptor consists of these quantized colors and their percentages in the region. A similarity distance measure is defined and shown to be equivalent to the quadratic color histogram distance measure. The quantized colors are indexed in the 3-D color space so that high-dimensional indexing can be avoided. During the search process, each quantized color in the query is used as a separate cue to find matches containing that color. The matches from all the query colors are then joined to obtain the final retrievals. Experimental results show that the proposed scheme is fast and accurate compared to the color histogram approach.

We present a prototype video analysis and retrieval system, called NeTra-V, that is being developed to build an object-based video representation for functionalities such as search and retrieval of video objects. A region-based content description scheme using low-level visual descriptors is proposed. In order to obtain regions for local feature extraction, a new spatio-temporal segmentation and region-tracking scheme is employed. The segmentation algorithm uses all three visual features: color, texture, and motion in the video data. A group processing scheme similar to the one in the MPEG-2 standard is used to ensure the robustness of the segmentation. The proposed approach can handle complex scenes with large motion. After segmentation, regions are tracked through the video sequence using extracted local features. The results of tracking are sequences of coherent regions, called "subobjects". Subobjects are the fundamental elements in our low-level content description scheme, which can be used to obtain meaningful physical objects in a high-level content description scheme. Experimental results illustrating segmentation and retrieval are provided.

We present an implementation of a system for content-based search and retrieval of video based on low-level visual features. The system consists of three parts, automatic video partition, feature extraction, video search and retrieval. Three primary features, color texture and motion are used for indexing. They are represented by color histograms, Gabor texture features, and motion histograms. Most of the processing is done directly in the MPEG compressed domain. Testing on sports and movie databases have shown good retrieval performance.

In this paper, we present a wavelet-based multigrid approach to solve the perturbation equation encountered in optical tomography. With this scheme, the unknown image, the data, as well as the weight matrix are all represented by wavelet expansions, thus yielding a multiresolution representation of the original perturbation equation in the wavelet domain. This transformed equation is then solved using a multigrid scheme, by which an increasing portion of wavelet coefficients of the unknown image are solved in successive approximations. One can also quickly identify regions of interest (ROI's) from a coarse level reconstruction and restrict the reconstruction in the following fine resolutions to those regions. At each resolution level a regularized least squares solution is obtained using the conjugate gradient descent method. This approach has been applied to continuous wave data calculated based on the diffusion approximation of several two-dimensional (2-D) test media. Compared to a previously reported one grid algorithm, the multigrid method requires substantially shorter computation time under the same reconstruction quality criterion.