Computer vision and machine learning have great potential to aid in aesthetic judgments and exploration, particularly in the understanding of shapes. This paper presents our work in a well-defined but largely unexplored problem in this field: the automated recognition of apparel silhouette attributes for real-world products. Silhouette attributes, such as v-neck for dresses and open toe for shoes, are very important attributes for understanding the appearance of apparel but difficult to recognize automatically. We propose methods employing multi-modal features and supervised learning to automatically recognize silhouette attributes based on product images and the associated text. These algorithms are extensively tested on a large dataset of dresses, tops, and shoes provided by online retailers. The proposed silhouette recognition approach achieves high recognition accuracy on the nine silhouette categories. Our approach and experiments are also expected to stimulate future research on this topic.

We introduce a robust image segmentation method based on a variational formulation using edge flow vectors. We demonstrate the nonconservative nature of this flow field, a feature that helps in a better segmentation of objects with concavities. A multiscale version of this method is developed and is shown to improve the localization of the object boundaries. We compare and contrast the proposed method with well known state-of-the-art methods. Detailed experimental results are provided on both synthetic and natural images that demonstrate that the proposed approach is quite competitive.

This paper presents an algorithm for measuring hair and face appearance in 2D images. Our approach starts by using learned mixture models of color and location information to suggest the hypotheses of the face, hair, and background regions. In turn, the image gradient information is used to generate the likely suggestions in the neighboring image regions. Either Graph-Cut or Loopy Belief Propagation algorithm is then applied to optimize the resulting Markov network in order to obtain the most likely hair and face segmentation from the background. We demonstrate that our algorithm can precisely identify the hair and face regions from a large dataset of face images automatically detected by the state-of-the-art face detector.

Variational cost functions that are based on pairwise similarity between pixels can be minimized within level set framework resulting in a binary image segmentation. In this paper we extend such cost functions and address multi-region image segmentation problem by employing a multi-phase level set framework. For multi-modal images cost functions become more complicated and relatively difficult to minimize. We extend our previous work, proposed for background/foreground separation, to the segmentation of images in more than two regions. We also demonstrate an efficient implementation of the curve evolution, which reduces the computational time significantly. Finally, we validate the proposed method on the Berkeley Segmentation Data Set by comparing its performance with other segmentation techniques.

We present an efficient probabilistic method for identity recognition in personal photo albums. Personal photos are usually taken under uncontrolled conditions -the captured faces exhibit significant variations in pose, expression and illumination that limit the success of traditional face recognition algorithms. We show how to improve recognition rates by incorporating additional cues present in personal photo collections, such as clothing appearance and information about when the photo was taken. This is done by constructing a Markov random field (MRF) that effectively combines all available contextual cues in a principled recognition framework. Performing inference in the MRF produces markedly improved recognition results in a challenging dataset consisting of the personal photo collections of multiple people. At the same time, the computational cost of our approach remains comparable to that of standard face recognition approaches.

In this paper, we analyze the redundancy in calculating all pairs of geodesic distances on a rectangular grid. Fast marching method is an efficient way to estimate the geodesic distances from a point. But when calculated for all the points on the grid, this introduces certain redundancy. Our analysis shows that over 90% of the distances are actually recalculated. We propose a novel solution which exploits this redundancy to reduce the number of distances evaluated using the fast marching method and enforces the symmetry of the distance matrix. Experimental results show the improved accuracy obtained with our implementation.

We propose an algorithm to generate one multi-focus image from a set of images acquired at different focus settings. First images are registered to avoid large misalignments. Each image is tiled with overlapping neighborhoods. Then, for each region the tile that corresponds to the best focus is chosen to construct the multi-focus image. The overlapping tiles are then seamlessly mosaicked. Our approach is presented for images from optical microscopes and hand held consumer cameras, and demonstrates robustness to temporal changes and small misalignments. The implementation is computationally efficient and gives good results.

In this paper, we introduce new types of variational segmentation cost functions and associated active contour methods that are based on pairwise similarities or dissimilarities of the pixels. As a solution to a minimization problem, we introduce a new curve evolution framework, the graph partitioning active contours (GPAC). Using global features, our curve evolution is able to produce results close to the ideal minimization of such cost functions. New and efficient implementation techniques are also introduced in this paper. Our experiments show that GPAC solution is effective on natural images and computationally efficient. Experiments on gray-scale, color, and texture images show promising segmentation results.

A novel scheme for image segmentation is presented. An image segmentation criterion is proposed that groups similar pixels together to form regions. This criterion is formulated as a cost function. Using gradient-descent methods, which lead to a curve evolution equation that segments the image into multiple homogenous regions, minimizes this cost function. Homogeneity is specified through a pixel-to-pixel similarity measure, which is defined by the user and can be adaptive based on the current application. To improve the performance of the system, an edge function is also used to adjust the speed of the competing curves. The proposed method can be easily applied to vector valued images such as texture and color images without a significant addition to computational complexity.

An image segmentation scheme that utilizes image-based flow fields in a curve evolution framework is presented. Geometric curve evolution methods require an edge function and a vector field with certain characteristics that are obtained from the image itself. A vector field borrowed from the edgeflow segmentation (Ma and Manjunath 2000) method is utilized both to obtain an edge function and to guide the curve evolution towards the object boundaries. This vector field is computed from the image using intensity, texture and color features. The proposed method integrates well-tested image features with the well-studied curve evolution methods thus achieving better segmentation results.

This work presents a novel approach to content-based image retrieval in categorical multimedia databases. The images are indexed using a combination of text and content descriptors. The categories are viewed as semantic clusters of images and are used to confine the search space. Keywords are used to identify candidate categories. Content-based retrieval is performed in these categories using multiple image features. Relevance feedback is used to learn the user's intent-query specification and feature-weighting-with minimal user-interface abstraction. The method is applied to a large number of images collected from a popular categorical structure on the World Wide Web. Results show that efficient and accurate performance is achievable by exploiting the semantic classification represented by the categories. The relevance feedback loop allows the content descriptor weightings to be determined without exposing the calculations to the user.