Detection CNN architectures often exhibit over-parameterization which results in excessive computational and storage overhead, but also undesired overfitting and reduced performance. In this work we focus on YOLOrs, a state-of-the-art CNN for target detection in remote sensing imagery, and counteract over-parameterization by enforcing Tensor-Train (TT) structure to its convolutional kernels. While TT has been successfully used before for compressing classification CNNs, this work is the first one that uses it to compress a detection CNN. We refer to the resulting network as YOLOrs-lite and compare its performance against standard YOLOrs as well as other state-of-the-art detection networks. Our numerical studies show that the proposed network attains superior detection performance, with storage savings as high as 70%. The proposed network combines light storage with real-time inference, making it quite promising for edge deployment.

Automatic recognition for complex scenes from aerial images and other sensor data (e.g. LiDAR) has become an active topic in the remote sensing community. In this paper, we proposed a novel framework that utilizes higher-order CRFs (HCRFs) to capture the spatial contextual information for the RGB aerial images along with their co-registered LiDAR data (DSMs). Our proposed HCRFs framework exploits the spatial contextual information in two levels. The first level encourages harmonic label co-existence within one segment, which can be generated by an unsupervised superpixel algorithm. The second level takes into account the local object co-occurrence among adjacent segments. We then show that how to apply the move making graph cuts algorithm to perform efficient inference for our proposed CRFs framework. Based on the experiments on a challenging data set, our proposed higher-order CRF framework generated state-of-the-art semantic segmentation results.

In this paper, we propose a Fast Text Detection (FTD) algorithm to automatically identify the text and line art pixels in scanned monochromatic and color documents for enhancing text in printed documents. To simulate the scanning operation, the input image is converted from the RGB space to a Luminance-Chrominance space and is then divided into individual strips. This is followed by a low pass filtering operation on the input strips for removing high frequency noise and to reduce processing time. An edge detection scheme is then applied to the blurred strips to generate corresponding edge-strips. The edge-strips are then subjected to a morphological operation and an edge-based adaptive thresholding simultaneously and the resulting two output strips are merged together to obtain the final candidate text plane. In the text enhancement operation, the original image is first converted into LCH space and the pixels corresponding to the detected text pixels are enhanced via a clipping operation in the L and C channel. This algorithm is highly efficient in terms of memory usage and processing speed and is thus suited to run effectively in low-cost embedded devices.

Classification plays a significant role in analyzing remotely sensed imagery. In order to obtain an optimized classier, following aspects are rather challenging: 1) complexity in dealing with the overwhelming amount of data information from an advanced high resolution hyperspectral imaging sensor; 2) difficulty in leveraging spectral and spatial information across the sensed wavelengths; 3) struggles in obtaining adequate dataset as in the same modalities with labeled ground truth in the training process. Therefore, we propose a novel classification approach to tackle these issues by utilizing probabilistic graphical model on super-pixel segmentation. This method is capable of compacting hyperspectral information efficiently which decreases computing complexity. Moreover, the employment of probabilistic graphical models that weighs the strong dependency in spatial and spectral neighbors improves accuracy. One of the most successful probabilistic graphical models is Conditional Random Fields (CRFs). Conventional methods utilize all spectral bands and assign the corresponding raw intensity values into the feature functions in CRFs and build the grid graph. These methods, however, require significant computational efforts and yield an ambiguous summary from the data. To mitigate these problems, we cooperate a non-linear kernel based classier to provide the meaningful probability features for CRFs and learn the non-grid graph from super pixel segmentation.

In this paper, we present an algorithm to determine the signature line in scanned documents through a fast and computationally efficient technique. To that effect, the algorithm consists of three modules, namely pre-processing, candidate line detection and classification. Preprocessing in this case consists of hard thresholding, which is essential to line detection in the next module. The Hough transform for a single angle (horizontal) is utilized in the line detection module. Connected component analysis is utilized to merge detected Hough lines belonging to the same line on the document. Finally, the classification module attempts to classify each line as a signature line or not. This is accomplished using Histogram of a Gradient (HoG) features and a Euclidean distance measure for comparison. Prior data for this similarity comparison consists of images of target text in different font sizes and styles. The proposed algorithm showed favorable results in terms of precision and accuracy.

Object detection in high resolution remote sensing images is a crucial yet challenging problem for many applications. With the development of satellite and sensor technologies, remote sensing images attain very high spatial resolution, giving rise to the employment of many computer vision algorithms. Therefore, the object detection is usually formalized as a supervised classification task. In this paper, we propose to apply the Gaussian process (GP) classification algorithm for our detection problem. Among different classifiers, the GP classifier is a Bayesian classification method that is able to make estimations in a probabilistic way. To demonstrate the performance of the proposed approach, we experiment the proposed framework with different feature extraction schemes and classification methods. We carry out a cross-validation experiment over an image dataset that consists of objects and non-objects to train an object detector, and apply the trained detector in an unobserved image scene to search for the objects of interest. Our results show that the GP classifier is competitive to support vector machines (SVM), which is considered state-of-the-art.

In the field of subpixel target detection in hyperspectral images, there is well-documented current interest for identifying preferred background covariance matrix estimates, to be used in the formation of matched-filter detectors. In this work, for the first time, we study the case of local background covariance matrix estimation from SLIC-superpixel based coherent regions. Interestingly, our experiments illustrate that the SLIC-based matched-filter detector can attain performance superior to that of contemporary alternatives which employ different, globally or locally estimated, background statistics.

Vehicle detection and recognition from aerial imagery provides useful information for local vehicle volume estimation and traffic monitoring. In this paper, we propose a method that accurately detects vehicles in urban environment using a probabilistic classification method followed by a refinement based on object segments. Both classification and segmentation methods make use of coregistered aerial RGB images and airborne LiDAR data. Pixel-wise vehicle probability estimation is achieved using Gaussian process (GP) classification and object segments are obtained by applying a gradient based segmentation algorithm (GSEG). The vehicle is then detected by refining the initial probability estimation with the following constraints: car size, statistical significance and 3D surface shape. Experimental results show our method achieves 90.8% precision and 93.7% recall, which outperforms the ones that only use size constraints.

In the current paper, we explore the use of a gradient based region growing method to segment Synthetic Aperture radar (SAR) imagery into disjoint regions and later categorize them into one of several known classes. Segment based features, as opposed to pixel based features, are utilized in the supervised classification stage. The method is tested on SAR images acquired from RADARSAT-2 and AIRSAR sensors. Here, SAR amplitude data and/or features from Pauli, Cloude-Pottier decompositions are used for segmentation and classification. Qualitative and quantitative results show good identification of open-water, terrain and agricultural areas.

On behalf of the ICIP 2012 Organizing Committee, I would like to take this opportunity to welcome you to the 19th IEEE International Conference on Image Processing (ICIP) in Orlando, Florida. Orlando is nicknamed “The City Beautiful,” and its symbol is the fountain at Lake Eola. This “City Beautiful” has more theme parks and entertainment attractions than any other city in the world. It is best known for the Walt Disney World Resort which was founded in 1971 and located approximately 21 miles/34 km southwest of downtown in Lake Buena Vista. It has currently grown to encompass four magnificent theme parks: Magic Kingdom, Epcot, Hollywood Studies and Animal Kingdom, two fabulous water parks: Blizzard Beach and Typhoon Lagoon and a variety of world class hotels, facilities and shopping areas. Another well-known resort is the Universal Orlando Resort, which consists of two parks, Universal Studios Florida and Islands of Adventure. Orlando is home to the SeaWorld theme park, Gatorland, and Wet 'n Wild Water Park. The city's famous attractions form the backbone of Orlando's tourism industry attracting over 51 million tourists a year.

We propose a new unsupervised technique for segmentation of digital video that partitions its constituents by identifying homogeneous sub-volumes within the data treated as a three dimensional (3-D) spatio-temporal volume. Our approach is commenced by subjecting the input video to a 3-D gradient detection method that determines the magnitude of color changes across the volume. The computed gradient is utilized to guide a volume growing procedure, initiated at spatio-temporal locations with small gradient magnitudes and concluded at locations with large gradient magnitudes, to yield an initial set of homogeneous sub-volumes. These partitions are further refined by integrating them with an entropy-based texture descriptor as well as color and gradient features in a multivariate volume merging procedure that fuses sub-volumes with similar attributes, to yield the final segmentation. Our approach was tested on several simple-to-complex video sequences with favorable results.

We propose a methodology for improved segmentation of images in a Bayesian framework by fusion of color, texture and gradient information. The proposed algorithm is initialized by subjecting the input image to an adaptive clustering scheme for initial region formation. Following this, a vector field approach is employed to split regions comprising of strong edges. Subsequently, all spatially independent regions are provided separate region labels, given that most clustering approaches neglect the spatial association among them. The resultant region map is integrated with textural features based on co-occurrence matrices, in a unique multivariate merging procedure hierarchically fusing regions with strong similarities. The merging process is eventually terminated using a Receiver Operating Characteristic analysis to determine the optimal number of segments in the final segmentation. Performance evaluation on the Berkeley segmentation dataset demonstrates that our approach outperforms a family of published techniques, with superior segmentation quality.

We propose a novel gradient driven methodology for three dimensional (3-D) segmentation of Computed Tomographic (CT) imagery. Our approach begins interactively where-in a user marks a set of voxels within the cross-section of a Sub-Volume Of Interest (SVOI), using a single slice of the CT volume. Subsequently, a 3-D gradient detection scheme is utilized to determine the radiodensity variations across the volume. The resultant gradient information is employed in an iterative volume growing procedure, which is initiated at voxels with small gradient magnitudes adjoining the user-selected voxels and culminates at voxels with large gradient magnitudes, to arrive at the final 3-D segmentation result of the SVOI. The aforementioned method was tested on multiple studies and the results show favorable performance against a state-of-the-art technique.

We propose a novel unsupervised segmentation method that efficiently exploits spectral intensity, gradient and textural information in remotely sensed imagery. Our approach begins in a multi-band gradient detection scheme of the input scene, utilized to determine the spectral intensity variations across it. The obtained gradient map is employed in an iterative region growth procedure that originates in pixel locations of gradual/low intensity variations and concludes in sites of abrupt intensity transitions, to yield an initial region map. This map is combined with co-occurrence matrices based textural descriptors using a multivariate region merging procedure that fuses regions with similar characteristics to yield the final segmentation. Our approach was tested on several multi/hyperspectral data sets with favorable results.

Maximization of mutual information (MI) (MMI) has a wide application in multimodal image registration. The conventional MMI algorithm is based on the statistics derived from image sampling and does not use spatial features. In this paper, we proposed an improved MMI registration algorithm that combines the spatial information through a feature-based selection mechanism. A Harris corner filter is chosen to classify the pixels. The classification results are used to focus sample selection, joint entropy estimation, and weighted MI information calculation. Although features play an important role, conventional feature matching is not used, which removes an important challenge to the use of feature-based information. The advantages of the proposed algorithm are improved robustness by incorporation of spatial features and significant reduction in computing time for registration. Experiments with challenging synthetic images and multimodal airborne infrared images are provided to demonstrate the improvement.

In this paper, we propose a segment-based triangulation method for building parallel-perspective stereo mosaics using ray interpolation. The currently available fast parallel ray interpolation for stereo mosaicing uses a uniform or regular triangulation technique that does not take advantage of the spatial information. This works well when the interframe displacement in the video is small. However, for large interframe displacements, hence large motion parallax, we observe visual artifacts in the mosaics when the source triangles in the video frames are warped into destination triangles in the mosaics using affine transformations. Our method uses the edge information present in the scene to ensure that triangles do not cross over from one planar facet to the other. The video frames are segmented, and the edges are obtained from the inner boundaries of different segments. A mesh is designed in each segment or region, over which motion between consecutive video frames is constant. Hence, this method avoids unwanted warping of objects with large motion parallax in the scene and reduces the visual artifacts. The performance of our proposed technique is demonstrated on a series of simple-to-complex video imagery collected by utilizing cameras and airborne sensors.

We propose an image segmentation methodology which exploits gradient information in a multiresolution framework. The proposed algorithm commences with a wavelet decomposition procedure to obtain a pyramidal representation of the input image, accompanied by an adaptive threshold generation scheme required for segregating regions of varying gradient densities. At low (coarse) resolution levels, progressive region growth, texture characterization, and region merging modules are integrated together to provide interim segmentations. These interim results are transferred from one resolution level to another as a-priori information, until the final result at the highest (original) resolution is achieved. Performance evaluation on several hundred images demonstrates that our algorithm computationally outperforms various published techniques, with superior segmentation quality.

Image segmentation is a fundamental task in many computer vision applications. In this paper, we propose a new unsupervised color image segmentation algorithm, which exploits the information obtained from detecting edges in color images in the CIE L *a *b * color space. To this effect, by using a color gradient detection technique, pixels without edges are clustered and labeled individually to identify some initial portion of the input image content. Elements that contain higher gradient densities are included by the dynamic generation of clusters as the algorithm progresses. Texture modeling is performed by color quantization and local entropy computation of the quantized image. The obtained texture and color information along with a region growth map consisting of all fully grown regions are used to perform a unique multiresolution merging procedure to blend regions with similar characteristics. Experimental results obtained in comparison to published segmentation techniques demonstrate the performance advantages of the proposed method.

Image segmentation is a fundamental task in many computer vision applications. In this paper, we present a novel unsupervised color image segmentation algorithm that utilizes color gradients, dynamic thresholding and texture modeling algorithms in a split and merge framework. To this effect, pixels without edges are clustered and labeled individually to identify the preliminary image content. Pixels that contain higher gradients are further classified by utilizing an iterative dynamic threshold generation technique and an appropriate entropy based texture model. The proposed algorithm was demonstrated successfully on an extensive database of images and benchmarked favorably against prior art.

Registration of maps and airborne or satellite images is an important problem for tasks such as map updating and change detection. This is a difficult problem because map features such as roads and buildings may be mislocated and features extracted from images may not correspond to map features, which is the challenge for the traditional feature-based registration algorithm. Our algorithm obtains a general global registration of maps and images by applying statistical techniques to map and image features. Finer analysis can then be used to find changes and local mismatches. The Maximization of Mutual Information (MMI) technique has proven to be very robust in image-to-image registration. This paper extends the MMI technique to the map-to-image registration problem through a focus-of-attention mechanism that forces MMI to utilize correspondences that have a high probability of being information rich and evaluates the accuracy and robustness in series experiments. The advantages and limitations are discussed in detail. The experimental results demonstrate map and image registration.

This paper presents a novel algorithm for improving the quality of edges in printed text. The algorithm is designed to add pixels at selected edge locations post halftoning. The extent of the correction is proportional to the "strength" of the edge, as determined by comparing the local differences in a four-pixel neighborhood to a dynamically generated threshold. The process is computationally efficient and requires minimal memory resources. The performance of our proposed algorithm is clearly demonstrated on several characters and lines. While the algorithm aims to improve the quality of printed text (edges), it is possible to extend its application to improvement of any edge identifiable in image documents.

In this paper, we propose a defect analysis system, which automatically aligns a digitized copy of a printed output to a reference electronic original and subsequently illustrates potential image quality artifacts. We focus on image defects or artifacts caused by shortfalls in mechanical or electrophotographic processes. In this method, log-polar transform and mutual information techniques are used for image registration. A confidence map is then calculated by comparing the contrast and entropy of the neighborhood for each pixel in both images. This confidence map results in a qualitative difference between printed documents and electronic originals. The algorithm was demonstrated successfully on a database of 94 images with 95.7% accuracy.

In this paper, we propose a novel algorithm for detecting needles and their corresponding implanted radioactive seed locations in the prostate during brachytherapy from transrectal ultrasound images. This is carried out efficiently using separable Gaussian filters in a probabilistic Gibbs random field framework. An approximation of the needle path through the prostate volume is obtained using a polynomial fit. The seeds are then detected and assigned to their corresponding needles by calculating local maxima. In our experiments, we were able to successfully localize over 85% of the implanted seeds

In this paper, we propose an algorithm for object tracking through frames of video using a fast partial shape matching technique. The tracking is divided into two modules: 1) moving object extraction followed by color/edge segmentation, and 2) tracking through frames using partial shape matching. The major challenges of object tracking, such as occlusions, splitting of one object and appearance and disappearance of objects, are effectively resolved. The proposed algorithm is tested on several synthetic and real life video sequences and has proven to be very effective in identifying and tracking moving objects independent of translations, rotations, scale variations and occlusions.

We retrieve samples from large image/video databases by means of learning by example. We propose a fast partial shape matching guided 2D object recognition algorithm to significantly accelerate recognition/matching of full (non-occluded) or partially occluded objects. The significant increase in speed comes from the fact that the search space is reduced to only those combinations of regions in the neighborhood of potential partial matches, as opposed to all combinations of regions as was done in our prior work (Xu et al. (2003)). Theoretical calculations and experimental results are provided to demonstrate the effectiveness of the proposed algorithm on real images.

This paper describes the design and development of a ping-pong robotic arm as an application of robotic vision. Displaced frame difference (DFD) is utilized to segment the ball motion from background motion. 3-D ball tracking using parametric calibration of single CCD camera is explained. This visual information is temporally updated and further employed to guide a robot arm to hit the ball at a specified location. The results signify the system development based on single camera tracking. System latency is measured as a function of the camera interface, processor architecture, and robot motion. Various hardware and software parameters that influence the real time system performance are also discussed.

We present a novel lossless (reversible) data-embedding technique, which enables the exact recovery of the original host signal upon extraction of the embedded information. A generalization of the well-known least significant bit (LSB) modification is proposed as the data-embedding method, which introduces additional operating points on the capacity-distortion curve. Lossless recovery of the original is achieved by compressing portions of the signal that are susceptible to embedding distortion and transmitting these compressed descriptions as a part of the embedded payload. A prediction-based conditional entropy coder which utilizes unaltered portions of the host signal as side-information improves the compression efficiency and, thus, the lossless data-embedding capacity.

We present a novel "dynamic learning" approach for an intelligent image database system to automatically improve object segmentation and labeling without user intervention, as new examples become available, for object-based indexing. The proposed approach is an extension of our earlier work on "learning by example", which addressed labeling of similar objects in a set of database images based on a single example (Saber et al. (2003)). It utilizes multiple example object templates to improve the accuracy of existing object segmentations and labels. We also propose to use Normalized Area of Symmetric Differences (NASD) as the similarity metric in "dynamic learning", due to its robustness to boundary noise that results from automatic image segmentation. The performance of the dynamic learning concept is demonstrated by experimental results.

We propose a system that employs low-level image segmentation followed by color and two-dimensional (2-D) shape matching to automatically group those low-level segments into objects based on their similarity to a set of example object templates presented by the user. A hierarchical content tree data structure is used for each database image to store matching combinations of low-level regions as objects. The system automatically initializes the content tree with only "elementary nodes" representing homogeneous low-level regions. The "learning" phase refers to labeling of combinations of low-level regions that have resulted in successful color and/or 2-D shape matches with the example template(s). These combinations are labeled as "object nodes" in the hierarchical content tree. Once learning is performed, the speed of second-time retrieval of learned objects in the database increases significantly. The learning step can be performed off-line provided that example objects are given in the form of user interest profiles. Experimental results are presented to demonstrate the effectiveness of the proposed system with hierarchical content tree representation and learning by color and 2-D shape matching on collections of car and face images.

We present a novel reversible (lossless) data hiding (embedding) technique, which enables the exact recovery of the original host signal upon extraction of the embedded information. A generalization of the well-known LSB (least significant bit) modification is proposed as the data embedding method, which introduces additional operating points on the capacity-distortion curve. Lossless recovery of the original is achieved by compressing portions of the signal that are susceptible to embedding distortion, and transmitting these compressed descriptions as a part of the embedded payload. A prediction-based conditional entropy coder which utilizes static portions of the host as side-information improves the compression efficiency, and thus the lossless data embedding capacity.

Several fragile watermarking schemes presented in the literature are either vulnerable to vector quantization (VQ) counterfeiting attacks or sacrifice localization accuracy to improve security. Using a hierarchical structure, we propose a method that thwarts the VQ attack while sustaining the superior localization properties of blockwise independent watermarking methods. In particular, we propose dividing the image into blocks in a multilevel hierarchy and calculating block signatures in this hierarchy. While signatures of small blocks on the lowest level of the hierarchy ensure superior accuracy of tamper localization, higher level block signatures provide increasing resistance to VQ attacks. At the top level, a signature calculated using the whole image completely thwarts the counterfeiting attack. Moreover, "sliding window" searches through the hierarchy enable the verification of untampered regions after an image has been cropped. We provide experimental results to demonstrate the effectiveness of our method.

Several fragile watermarking schemes presented in the literature are either vulnerable to vector quantization (VQ) counterfeiting attacks or sacrifice localization accuracy to improve security. Using a hierarchical structure, we propose a method that thwarts the VQ attack while sustaining the superior localization properties of blockwise independent watermarking methods. In particular, we propose dividing the image into blocks in a multi-level hierarchy and calculating block signatures in this hierarchy. While signatures of small blocks on the lowest level of the hierarchy ensure superior accuracy of tamper localization, higher level block signatures provide increasing resistance to VQ attacks. At the top level, a signature calculated using the whole image completely thwarts the counterfeiting attack. Moreover, "sliding window" searches through the hierarchy enable the verification of untampered regions after an image has been cropped.

Currently, image retrieval systems are based on low-level features of color, texture and shape, not on the semantic descriptions that are common to humans, such as objects, people, and place. In order to narrow down the gap between the low level and semantic level, object-based content analysis, which segments the semantically meaningful objects of images, is an essential step. In this study, we propose a learning process in order to perform effective automatic off-line analysis on a multi-level segmented image stack. Meaningful objects are extracted given certain user search patterns and interest profiles. Color and/or shape information of the objects is stored in the hierarchical content representations of the images. This information is utilized by a hierarchical matching scheme to improve the retrieval speed in the subsequent searches.

This paper proposes a new contour based shape matching approach capable of recognizing partially occluded objects in images. The process can be divided into the following steps: region formation from low level color image segmentation, B-spline filtering and feature point extraction, correspondence determination, least square estimation of affine transformation parameters, and similarity measuring. Once similarity is established, the information is retained using a hierarchical content description scheme, enabling expedient object based image retrieval at a later time.

This paper proposes a self-learning content-based image indexing and retrieval system that employs a hierarchical content representation (consisting of objects and regions) and a hierarchical content matching method for effective and efficient image/object retrieval. The “learning” behavior is enabled by our proposed hierarchical content representation which allows easy storage of combinations of regions that have resulted in successful matches to objects of interest as determined by user search patterns and profiles. The learning step effectively performs an automatic off-line analysis of database images into meaningful objects. Once the learning phase is complete, the speed of shape based retrieval of the learned objects in the database increases significantly. Experimental results are presented to show the effectiveness of the proposed hierarchical content representation, hierarchical matching, and the learning behavior on collections of car images.

Neugebauer modeling plays an important role in obtaining end-to-end device characterization profiles for halftone color printer calibration. This paper proposes total least square (TLS) regression methods to estimate the parameters of various Neugebauer models. Compared to the traditional least squares (LS) based methods, the TLS approach is physically more appropriate for the printer modeling problem because it accounts for errors in the measured reflectance of both the primaries and the modeled samples. A TLS method based on print measurements from single-colorant step-wedges is first developed. The method is then extended to incorporate multicolorant print measurements using an iterative algorithm. The LS and TLS techniques are compared through tests performed on two color printers, one employing conventional rotated halftone screens and the other using a dot-on-dot halftone screen configuration. Our experiments indicate that the TLS methods yield a consistent and significant improvement over the LS-based techniques for model parameter estimation. The gains from the TLS method are particularly significant when the number of patches for which measured data is available is limited.

The Neugebauer model is a powerful tool in obtaining end-to-end device characterization profiles for halftone color printer calibration. In this paper, we propose total least square (TLS) regression methods to estimate the parameters of Neugebauer models. Compared to the traditional least squares (LS) based methods, the TLS approach is a physically more appropriate procedure, because it accounts for errors in the measured reflectance of both the selected primaries and the modeled reflectance. The proposed TLS techniques are tested on a Xerox color printer with rotated halftone screen, and the results are compared with the LS based algorithms. Our experiments indicate that the TLS methods yield a significant improvement over the LS based techniques for model parameter estimation.

We describe an algorithm for detecting human faces and facial features, such as the location of the eyes, nose and mouth. First, a supervised pixel-based color classifier is employed to mark all pixels that are within a prespecified distance of “skin color”, which is computed from a training set of skin patches. This color-classification map is then smoothed by Gibbs random field model-based filters to define skin regions. An ellipse model is fit to each disjoint skin region. Finally, we introduce symmetry-based cost functions to search the center of the eyes, tip of nose, and center of mouth within ellipses whose aspect ratio is similar to that of a face.

We propose a new method for combined color image segmentation and edge linking. The image is first segmented based on color information only. The segmentation map is modeled by a Gibbs random field, to ensure formation of spatially contiguous regions. Next, spatial edge locations are determined using the magnitude of the gradient of the 3-channel image vector field. Finally, regions in the segmentation map are split and merged by a region-labeling procedure to enforce their consistency with the edge map. The boundaries of the final segmentation map constitute a linked edge map. Experimental results are reported.

We present a method for automatic image annotation and retrieval based on query-by-example by region-based shape matching. The proposed method consists of two parts: region selection and shape matching. In the first part, the image is partitioned into disjoint, connected regions with more-or-less uniform color, whose boundaries coincide with spatial edge locations. Each region or valid combinations of neighboring regions constitute “potential objects.” In the second part, the shape of each potential object is tested to determine whether it matches one from a set of given templates. To this effect, we propose a new shape matching method, which is translation-, rotation-, and isotropic scale-invariant, where the boundary of each potential object, as well as of each template, is represented by a B-spline. We, then, identify correspondences between the joint points of the B-splines of potential objects and templates by using a modal matching method. These correspondences are used to estimate the parameters of an affine mapping to register the object with the template. A proximity measure is then computed between the two contours based on the Hausdorff distance. We demonstrate the performance of the proposed method on a variety of images.

We propose a new method for combined color image segmentation and edge linking. The image is first segmented based on color information only. The segmentation map is modeled by a Gibbs random field, to ensure formation of spatially contiguous regions. Next, spatial edge locations are determined using the magnitude of the gradient of the 3-channel image vector field. Finally, regions in the segmentation map are split and merged by a region-labeling procedure to enforce their consistency with the edge map. The boundaries of the final segmentation map constitute a linked edge map. Experimental results are reported.

We present algorithms for automatic image annotation and retrieval based on pixel- or region-based color; region-based shape; and block- or region-based texture features and several schemes for integrating them. Automatic region selection is accomplished by integrating color and spatial edge features. Color, shape, and texture indexing may be knowledge-based (using appropriate training sets) or by example. The multi-feature integration algorithms are designed to: (1) offer the user a wide range of options and flexibilities to enhance the outcome of the search and retrieval operations, and (2) provide a compromise between accuracy and computational complexity. One of the novel features of this system is its ability to automatically segment images into meaningful regions, and provide the option of region (object)-based image indexing without user interaction.

This paper describes an algorithm for detecting human faces and subsequently localizing the eyes, nose, and mouth. First, we locate the face based on color and shape information. To this effect, a supervised pixel-based color classifier is used to mark all pixels which are within a prespecified distance of "skin color". This color-classification map is then subject to smoothing employing either morphological operations or filtering using a Gibbs random field model. The eigenvalues and eigenvectors computed from the spatial covariance matrix are utilized to fit an ellipse to the skin region under analysis. The Hausdorff distance is employed as a means for comparison, yielding a measure of proximity between the shape of the region and the ellipse model. Then, we introduce symmetry-based cost functions to locate the center of the eyes, tip of nose, and center of mouth within the facial segmentation mask. The cost functions are designed to take advantage of the inherent symmetries associated with facial patterns. We demonstrate the performance of our algorithm on a variety of images.

We describe a system which automatically annotates images with a set of prespecified keywords, based on supervised color classification of pixels intoNprespecified classes using simple pixelwise operations. The conditional distribution of the chrominance components of pixels belonging to each class is modeled by a two-dimensional Gaussian function, where the mean vector and the covariance matrix for each class are estimated from appropriate training sets. Then, a succession of binary hypothesis tests with image-adaptive thresholds has been employed to decide whether each pixel in a given image belongs to one of the predetermined classes. To this effect, a universal decision threshold is first selected for each class based on receiver operating characteristics (ROC) curves quantifying the optimum “true positive” vs “false positive” performance on the training set. Then, a new method is introduced for adapting these thresholds to the characteristics of individual input images based on histogram cluster analysis. If a particular pixel is found to belong to more than one class, a maximuma posterioriprobability (MAP) rule is employed to resolve the ambiguity. The performance improvement obtained by the proposed adaptive hypothesis testing approach over using universal decision thresholds is demonstrated by annotating a database of 31 images.