Current work in object categorization discriminates among objects that typically possess gross differences which are readily apparent. However, many applications require making much finer distinctions. We address an insect categorization problem that is so challenging that even trained human experts cannot readily categorize images of insects considered in this paper. The state of the art that uses visual dictionaries, when applied to this problem, yields mediocre results (16.1% error). Three possible explanations for this are (a) the dictionaries are unsupervised, (b) the dictionaries lose the detailed information contained in each keypoint, and (c) these methods rely on hand-engineered decisions about dictionary size. This paper presents a novel, dictionary-free methodology. A random forest of trees is first trained to predict the class of an image based on individual keypoint descriptors. A unique aspect of these trees is that they do not make decisions but instead merely record evidence-i.e., the number of descriptors from training examples of each category that reached each leaf of the tree. We provide a mathematical model showing that voting evidence is better than voting decisions. To categorize a new image, descriptors for all detected keypoints are “dropped” through the trees, and the evidence at each leaf is summed to obtain an overall evidence vector. This is then sent to a second-level classifier to make the categorization decision. We achieve excellent performance (6.4% error) on the 9-class STONEFLY9 data set. Also, our method achieves an average AUC of 0.921 on the PASCAL06 VOC, which places it fifth out of 21 methods reported in the literature and demonstrates that the method also works well for generic object categorization.

This paper presents a new structure-based interest region detector called principal curvature-based regions (PCBR) which we use for object class recognition. The PCBR interest operator detects stable watershed regions within the multi-scale principal curvature image. To detect robust watershed regions, we "clean" a principal curvature image by combining a grayscale morphological close with our new "eigenvectorflow" hysteresis threshold. Robustness across scales is achieved by selecting the maximally stable regions across consecutive scales. PCBR typically detects distinctive patterns distributed evenly on the objects and it shows significant robustness to local intensity perturbations and intra-class variations. We evaluate PCBR both qualitatively (through visual inspection) and quantitatively (by measuring repeatability and classification accuracy in real-world object-class recognition problems). Experiments on different benchmark datasets show that PCBR is comparable or superior to state-of-art detectors for both feature matching and object recognition. Moreover, we demonstrate the application of PCBR to symmetry detection.

For many multi-part object classes, the set of parts can vary not only in location but also in type. For example, player formations in American football involve various subsets of player types, and the spatial constraints among players depend largely upon which subset of player types constitutes the formation. In this work, we study the problem of localizing and classifying the parts of such objects. Pictorial structures provide an efficient and robust mechanism for localizing object parts. Unfortunately, these models assume that each object instance involves the same set of parts, making it difficult to apply them directly in our setting. With this motivation, we introduce the mixture-of-parts pictorial structure (MoPPS) model, which is characterized by three components: a set of available parts, a set of constraints that specify legal part subsets, and a function that returns a pictorial structure for any legal part subset. MoPPS inference corresponds to jointly computing the most likely subset of parts and their positions. We propose a restricted, but useful, representation for MoPPS models that facilitates inference via branch-and-bound optimization, which we show is efficient in practice. Experiments in the challenging domain of American football show the effectiveness of the model and inference procedure.

This paper describes a fully automated stone fly-larvae classification system using a local features approach. It compares the three region detectors employed by the system: the Hessian-affine detector, the Kadir entropy detector and a new detector we have developed called the principal curvature based region detector (PCBR). It introduces a concatenated feature histogram (CFH) methodology that uses histograms of local region descriptors as feature vectors for classification and compares the results using this methodology to that of Opelt [Opelt, A, et.al., 2006.] on three stonefly identification tasks. Our results indicate that the PCBR detector outperforms the other two detectors on the most difficult discrimination task and that the use of all three detectors outperforms any other configuration. The CFH methodology also outperforms the Opelt methodology in these tasks

This paper presents a semi-automatic segmentation technique called Bayesian cut that formulates object boundary detection as the most probable explanation (MPE) of a Bayesian network’s joint probability distribution. A two-layer Bayesian network structure is formulated from a planar graph representing a watershed segmentation of an image. The network’s prior probabilities encode the confidence that an edge in the planar graph belongs to an object boundary while the conditional probability tables (CPTs) enforce global contour properties of closure and simplicity (i.e., no self-intersections). Evidence, in the form of one or more connected boundary points, allows the network to compute the MPE with minimal user guidance. The constraints imposed by CPTs also permit a linear-time algorithm to compute the MPE, which in turn allows for interactive segmentation where every mouse movement recomputes the MPE based on the current cursor position and displays the corresponding segmentation.

Matching points between multiple images of a scene is a vital component of many computer vision tasks. Point matching involves creating a succinct and discriminative descriptor for each point. While current descriptors such as SIFT can find matches between features with unique local neighborhoods, these descriptors typically fail to consider global context to resolve ambiguities that can occur locally when an image has multiple similar regions. This paper presents a feature descriptor that augments SIFT with a global context vector that adds curvilinear shape information from a much larger neighborhood, thus reducing mismatches when multiple local descriptors are similar. It also provides a more robust method for handling 2D nonrigid transformations since points are more effectively matched individually at a global scale rather than constraining multiple matched points to be mapped via a planar homography. We have tested our technique on various images and compare matching accuracy between the SIFT descriptor with global context to that without.

We introduce a confidence measure that estimates the assurance that a graph arc (or edge) corresponds to an object boundary in an image. A weighted, planar graph is imposed onto the watershed lines of a gradient magnitude image and the confidence measure is a function of the cost of fixed-length paths emanating from and extending to each end of a graph arc. The confidence measure is applied to automate the detection of object boundaries and thereby reduces (often greatly) the time and effort required for object boundary definition within a user guided image segmentation environment.

Intelligent Scissors and Intelligent Paint are complementary interactive image segmentation tools that allow a user to quickly and accurately select objects of interest. This demonstration provides a means for participants to experience the dynamic nature of these tools.

Intelligent Scissors is an interactive image segmentation tool that allows a user to select piece-wise globally optimal contour segments that correspond to a desired object boundary. We present a new and faster method of computing the optimal path by over-segmenting the image using tobogganing and then imposing a weighted planar graph on top of the resulting region boundaries. The resulting region-based graph is many times smaller than the previous pixel-based graph, thus providing faster graph searches and immediate user interaction. Further tobogganing provides an new systematic and predictable framework for computing edge model parameters, allowing subpixel localization as well as a measure of edge blur.

Summary form only given. We present a new method for representation and (lossy) compression of bitonal document images. The technique extracts a skeletal medial axis from each object using a true Euclidean distance map of the image and then finds piecewise linear breakpoints in the skeleton to create a breakpoint skeletal representation, b.p.s. The b.p.s. is encoded for each object as a set of triples. The original binary object is reconstructed by first reconstructing the skeleton using linear interpolation between breakpoints and then fractionally dilating each point on the skeleton with the (linearly interpolated) radius, r/sub i/. For noninteger r/sub i/ fractional dilation provides a natural antialiasing in the reconstructed image. Breakpoints can be extracted to preserve fine detail or a more coarse representation by tightening or relaxing the pruning radius respectively. If, in extracting breakpoints, the pruning radius is set to zero, the reconstruction is almost lossless, but the compression is worse.

We present a new, interactive tool called Intelligent Scissors which we use for image segmentation and composition.Fully automated segmentation is an unsolved problem, while manual tracing is inaccurate and laboriously unacceptable.However, Intelligent Scissors allow objects within digital images to be extracted quickly and accurately using simple gesture motions with a mouse.When the gestured mouse position comes in proximity to an object edge, a live-wire boundary "snaps" to, and wraps around the object of interest.Live-wire boundary detection formulates discrete dynamic programming (DP) as a two-dimensional graph searching problem.DP provides mathematically optimal boundaries while greatly reducing sensitivity to local noise or other intervening structures.Robustness is further enhanced with on-the-fly training which causes the boundary to adhere to the specific type of edge currently being followed, rather than simply the strongest edge in the neighborhood.Boundary cooling automatically freezes unchanging segments and automates input of additional seed points.Cooling also allows the user to be much more free with the gesture path, thereby increasing the efficiency and finesse with which boundaries can be extracted.Extracted objects can be scaled, rotated, and composited using live-wire masks and spatial frequency equivalencing.Frequency equivalencing is performed by applying a Butterworth filter which matches the lowest frequency spectra to all other image components.Intelligent Scissors allow creation of convincing compositions from existing images while dramatically increasing the speed and precision with which objects can be extracted.