We address the problem of large-scale fine-grained visual categorization, describing new methods we have used to produce an online field guide to 500 North American bird species. We focus on the challenges raised when such a system is asked to distinguish between highly similar species of birds. First, we introduce "one-vs-most classifiers." By eliminating highly similar species during training, these classifiers achieve more accurate and intuitive results than common one-vs-all classifiers. Second, we show how to estimate spatio-temporal class priors from observations that are sampled at irregular and biased locations. We show how these priors can be used to significantly improve performance. We then show state-of-the-art recognition performance on a new, large dataset that we make publicly available. These recognition methods are integrated into the online field guide, which is also publicly available.

We present the design of a video event recounting system that takes YouTube-like videos, and identifies a minimal set of video segments and textual keyword descriptions in order to convince a user, in a time efficient manner, that the video contains an instance of a user-specificed human activity. The system is based on extensive user studies that have lead to nine design principles about human preferences and limits in semantic understanding. The processing pipeline locates the presence of user query keywords within the video, segments the video according to a model of human short-term memory for semantic similarities, selects those segments that best contain query terms, and abbreviates both the video and textual presentation. Speed-ups of a factor of 6 over simple video viewing time are achievable, without loss of semantic accuracy. In the 2013 Trecvid Multimedia Event Recounting competition, this system placed first in time efficiency, while remaining above average in description accuracy.