Computer vision and machine learning have great potential to aid in aesthetic judgments and exploration, particularly in the understanding of shapes. This paper presents our work in a well-defined but largely unexplored problem in this field: the automated recognition of apparel silhouette attributes for real-world products. Silhouette attributes, such as v-neck for dresses and open toe for shoes, are very important attributes for understanding the appearance of apparel but difficult to recognize automatically. We propose methods employing multi-modal features and supervised learning to automatically recognize silhouette attributes based on product images and the associated text. These algorithms are extensively tested on a large dataset of dresses, tops, and shoes provided by online retailers. The proposed silhouette recognition approach achieves high recognition accuracy on the nine silhouette categories. Our approach and experiments are also expected to stimulate future research on this topic.

We propose novel thinning conditions for grayscale 3D images that do not rely on image thresholding or segmentation. The resulting grayscale skeletonization routine extracts a topologically-consistent skeleton of curve-like features at all scales and intensities of the image. Our thinning conditions yield rich structural information that be used to identify and delete low-contrast shape features created by blurring and noise.

The popularity of camera phones enables many exciting multimedia applications. In this paper, we present a novel technology and several applications that allow users to interact with paper documents, books, and magazines. This interaction is in the form of reading and writing electronic information, such as images, web urls, video, and audio, to the paper medium by pointing a camera phone at a patch of text on a document. Our application does not require any special markings, barcodes, or watermarks on the paper document. Instead, we propose a document recognition algorithm that automatically determines the location of a patch of text in a large collection of document images given a small document image. This is very challenging because the majority of phone cameras lack autofocus and macro capabilities and they produce low quality images and video. We developed a novel algorithm, Brick Wall Coding (BWC), that performs image-based document recognition using the mobile phone video frames. Given a document patch image, BWC utilizes the layout, i.e. relative locations, of word boxes in order to determine the original file, page, and the location on the page. BWC runs real-time (4 frames per second) on a Treo 700w smartphone with a 312 MHz processor and 64MB RAM. Using our method we can recognize blurry document patch frames that contain as little as 4-5 lines of text and a video resolution as low as 176x144. We performed experiments by indexing 4397 document pages and querying this database with 533 document patches. Besides describing the basic algorithm, this paper also describes several applications that are enabled by mobile phone-paper interaction, such as inserting electronic annotations to paper, using paper as a tangible interface to collect and communicate multimedia data, and collaborative homework.

HotPaper enables users to electronically interact with paper documents by pointing their camera phones at a document. Electronic interaction is in the form of reading and/or writing images, audio and video clips, urls, notes, etc to the paper. HotPaper does not require any machine readable code, such as a barcode or watermark, to be present on the document. It utilizes our Brick Wall Coding document recognition algorithm [1] that uses a small document patch image to retrieve electronic versions of documents without performing OCR. The HotPaper demonstration runs on a Treo 700w 312 MHz mobile phone and performs document recognition at 4 frames per second using 176x144 video input.

An algorithm is presented that automatically generates ground-truthed video from a symbolic description for an object and a specification for the movement of a handheld video camera around that object. This provides a method to generate large amounts of training and test data for the development of computer vision algorithms. We describe an implementation of this technique for an imaging application in which a cell phone video camera is moved over a paper document. Experimental results demonstrate the similarity of images captured by the real camera to images generated by the proposed technique.

The advent of inexpensive digital image sensors and the ability to create photographs that combine information from a number of sensed images are changing the way we think about photography. In this paper, we describe a unique array of 100 custom video cameras that we have built, and we summarize our experiences using this array in a range of imaging applications. Our goal was to explore the capabilities of a system that would be inexpensive to produce in the future. With this in mind, we used simple cameras, lenses, and mountings, and we assumed that processing large numbers of images would eventually be easy and cheap. The applications we have explored include approximating a conventional single center of projection video camera with high performance along one or more axes, such as resolution, dynamic range, frame rate, and/or large aperture, and using multiple cameras to approximate a video camera with a large synthetic aperture. This permits us to capture a video light field, to which we can apply spatiotemporal view interpolation algorithms in order to digitally simulate time dilation and camera motion. It also permits us to create video sequences using custom non-uniform synthetic apertures.