The advent of large image databases (>10000) has created a need for tools which can search and organize images automatically by their content. This paper focuses on the use of hierarchical tree-structures to both speed-up search-by-query and organize databases for effective browsing. The first part of this paper develops a fast search algorithm based on best-first branch and bound search. This algorithm is designed so that speed and accuracy may be continuously traded-off through the selection of a parameter lambda. We find that the algorithm is most effective when used to perform an approximate search, where it can typically reduce computation by a factor of 20-40 for accuracies ranging from 80% to 90%. We then present a method for designing a hierarchical browsing environment which we call a similarity pyramid. The similarity pyramid groups similar images together while allowing users to view the database at varying levels of resolution. We show that the similarity pyramid is best constructed using agglomerative (bottom up) clustering methods, and present a fast sparse clustering method which dramatically reduces both memory and computation over conventional methods.