We study fine-grained categorization, the task of distinguishing among (sub)categories of the same generic object class (e.g., birds), focusing on determining botanical species (leaves and orchids) from scanned images. The strategy is to focus attention around several vantage points, which is the approach taken by botanists, but using features dedicated to the individual categories. Our implementation of the strategy is based on {\it vantage feature frames}, a novel object representation consisting of two components: a set of coordinate systems centered at the most discriminating local viewpoints for the generic object class and a set of category-dependent features computed in these frames. The features are pooled over frames to build the classifier. Categorization then proceeds from coarse-grained (finding the frames) to fine-grained (finding the category), and hence the vantage feature frames must be both detectable and discriminating. The proposed method outperforms state-of-the art algorithms, in particular those using more distributed representations, on standard databases of leaves.

Automatic retrieval tools are becoming increasingly important in botany and agriculture due to the growing interest in biodiversity and the ongoing shortage of skilled taxonomists. Our work is motivated by a botanical field scenario where the basic unit of observation is a plant. We describe a novel, image-based retrieval system for both educational and decision-making purposes. Given multiple leaf images of the same plant, the algorithm displays a ranked list of the most relevant species, along with a varied set of representative images from each estimated species. We focus on leaves but the strategy is generic, based on a hierarchical representation of latent variables called identification keys (IdKeys) which embody domain knowledge about taxonomy and landmarks. For each query image, keys are estimated sequentially, proceeding from landmarks to the genus and finally to an estimated set of species. The results over multiple queries are then collated into a single ranked list of species. Experiments demonstrate that the proposed approach achieves excellent performance on several databases of uncluttered leaf images as well as providing an instructive interface for measuring diversity and identifying new species.