Linguists use annotated text collections to validate, refute and refine a hypothesis about the written language. This research requires the creation and analysis of complex queries which are often above the technical expertise of the domain users. In this paper, we present a tool-design which enables language researchers to easily query annotated text corpora and conduct a comparative multi-faceted analysis on a single screen. The results of the iterative design process, including requirement analysis, multiple prototyping and user evaluation sessions, and expert reviews, are documented in detail. Our tool, called CorpSum, shows a 43.12 point increase in the mean SUS score in a randomized within-subjects test and an improvement of 3.18 times in mean task completion duration compared to a conventional solution. Two detailed case studies with linguists demonstrate a significant improvement for solving the real-world problems of the domain users.