Computer vision and machine learning have great potential to aid in aesthetic judgments and exploration, particularly in the understanding of shapes. This paper presents our work in a well-defined but largely unexplored problem in this field: the automated recognition of apparel silhouette attributes for real-world products. Silhouette attributes, such as v-neck for dresses and open toe for shoes, are very important attributes for understanding the appearance of apparel but difficult to recognize automatically. We propose methods employing multi-modal features and supervised learning to automatically recognize silhouette attributes based on product images and the associated text. These algorithms are extensively tested on a large dataset of dresses, tops, and shoes provided by online retailers. The proposed silhouette recognition approach achieves high recognition accuracy on the nine silhouette categories. Our approach and experiments are also expected to stimulate future research on this topic.

This paper presents methods to explore and fuse various clothing features for person clustering in family photos. Our approach automatically detects the clothing regions, extracts the global and localized features based on color and texture, and then computes multiple partitions of people based on different features, which are combined into the final consolidated person clustering with a cluster ensemble technique. The clothing clusters can be used for complementing face clusters and for other photo organization applications. The effectiveness of our approach is tested on real-world family photo datasets.

Automatic person clustering, which groups photos based on the individuals appearing in a photo collection, is a key component to facilitate photo management and sharing. Traditionally, person clusters are basically built by detecting faces and matching facial features. But these facial clusters can perform poorly when there are large pose variations and occlusions, which are not uncommon in consumer photos. In this paper, we propose an approach that employs contextual information to complement facial information in order to significantly improve the performance of person clustering. The proposed system is able to detect human skin, hair and clothing regions and extract features robustly. By matching the features, high-precision contextual clusters are obtained which can automatically link together multiple face clusters of the same person for efficient annotation. Promising results on family photo collections demonstrated the effectiveness of our approach.

Current work in object categorization discriminates among objects that typically possess gross differences which are readily apparent. However, many applications require making much finer distinctions. We address an insect categorization problem that is so challenging that even trained human experts cannot readily categorize images of insects considered in this paper. The state of the art that uses visual dictionaries, when applied to this problem, yields mediocre results (16.1% error). Three possible explanations for this are (a) the dictionaries are unsupervised, (b) the dictionaries lose the detailed information contained in each keypoint, and (c) these methods rely on hand-engineered decisions about dictionary size. This paper presents a novel, dictionary-free methodology. A random forest of trees is first trained to predict the class of an image based on individual keypoint descriptors. A unique aspect of these trees is that they do not make decisions but instead merely record evidence-i.e., the number of descriptors from training examples of each category that reached each leaf of the tree. We provide a mathematical model showing that voting evidence is better than voting decisions. To categorize a new image, descriptors for all detected keypoints are “dropped” through the trees, and the evidence at each leaf is summed to obtain an overall evidence vector. This is then sent to a second-level classifier to make the categorization decision. We achieve excellent performance (6.4% error) on the 9-class STONEFLY9 data set. Also, our method achieves an average AUC of 0.921 on the PASCAL06 VOC, which places it fifth out of 21 methods reported in the literature and demonstrates that the method also works well for generic object categorization.

Codebook-based representations are widely employed in the classification of complex objects such as images and documents. Most previous codebook-based methods construct a single codebook via clustering that maps a bag of low-level features into a fixed-length histogram that describes the distribution of these features. This paper describes a simple yet effective framework for learning multiple non-redundant codebooks that produces surprisingly good results. In this framework, each codebook is learned in sequence to extract discriminative information that was not captured by preceding codebooks and their corresponding classifiers. We apply this framework to two application domains: visual object categorization and document classification. Experiments on large classification tasks show substantial improvements in performance compared to a single codebook or codebooks learned in a bagging style.

Visual dictionaries have been successfully applied to “bags-of-points” image representations for generic object recognition. Usually the choice of low-level interest region detector and region descriptor (channel) has significant impact on the performance of visual dictionaries. In this paper, we propose a discriminative evaluation method — Maximum Mutual Information (MMI) curves to analyze the properties of the visual dictionaries built from different channels. Experimental results on benchmark datasets show that MMI curves can give us not only insight into the discriminative characteristics of the visual dictionaries, but also provide straightforward guidelines for the design of the image classifier.

Visual dictionaries are widely employed in object recognition to map unordered bags of local region descriptors into feature vectors for image classification. Most visual dictionaries have been constructed by unsupervised clustering. This paper presents an efficient discriminative approach, called Iterative Discriminative Clustering (IDC), for dictionary learning. In this approach, each dictionary entry is defined by a representative value and a learned distance metric. In IDC algorithm, the dictionary entries are initialized by unsupervised clustering and then locally adapted to improve their discriminative power. Motivated by studies of the characteristics of individual dictionary entries, we employ bagged decision lists (BDL) as our image classifier in order to explore the conjunctions of small number of informative dictionary entries for classification. Experiments on benchmark object recognition datasets show that the system based on the new discriminative dictionaries and BDL classifier give performance comparable or superior to the state-of-art generic object recognition approaches.

In this paper, we describe the architecture of the vision system for the Responsive Mirror, a novel system for retail fitting rooms that enables online social fashion comparisons in physical stores based on multi-camera perception. This vision system provides implicitly controlled real-time interaction for “self ” and “social ” clothing comparisons by automatically tracking user’s motion as she tries on clothes. We describe the key components of the motion-tracking and clothes-recognition systems and evaluate their effectiveness against images collected during a previous user study and a dataset of images representing content from a social fashion network. Author Keywords Ubiquitous computing, computer vision, multi-camera,

This paper presents a new structure-based interest region detector called principal curvature-based regions (PCBR) which we use for object class recognition. The PCBR interest operator detects stable watershed regions within the multi-scale principal curvature image. To detect robust watershed regions, we "clean" a principal curvature image by combining a grayscale morphological close with our new "eigenvectorflow" hysteresis threshold. Robustness across scales is achieved by selecting the maximally stable regions across consecutive scales. PCBR typically detects distinctive patterns distributed evenly on the objects and it shows significant robustness to local intensity perturbations and intra-class variations. We evaluate PCBR both qualitatively (through visual inspection) and quantitatively (by measuring repeatability and classification accuracy in real-world object-class recognition problems). Experiments on different benchmark datasets show that PCBR is comparable or superior to state-of-art detectors for both feature matching and object recognition. Moreover, we demonstrate the application of PCBR to symmetry detection.

This paper describes a fully automated stone fly-larvae classification system using a local features approach. It compares the three region detectors employed by the system: the Hessian-affine detector, the Kadir entropy detector and a new detector we have developed called the principal curvature based region detector (PCBR). It introduces a concatenated feature histogram (CFH) methodology that uses histograms of local region descriptors as feature vectors for classification and compares the results using this methodology to that of Opelt [Opelt, A, et.al., 2006.] on three stonefly identification tasks. Our results indicate that the PCBR detector outperforms the other two detectors on the most difficult discrimination task and that the use of all three detectors outperforms any other configuration. The CFH methodology also outperforms the Opelt methodology in these tasks