Inquisitive probing questions come naturally to humans in a variety of settings, but is a challenging task for automatic systems.One natural type of question to ask tries to fill a gap in knowledge during text comprehension, like reading a news article: we might ask about background information, deeper reasons behind things occurring, or more.Despite recent progress with data-driven approaches, generating such questions is beyond the range of models trained on existing datasets.We introduce INQUISITIVE, a dataset of ∼19K questions that are elicited while a person is reading through a document.Compared to existing datasets, INQUISITIVE questions target more towards high-level (semantic and discourse) comprehension of text.We show that readers engage in a series of pragmatic strategies to seek information.Finally, we evaluate question generation models based on GPT-2 (Radford et al., 2019) and show that our model is able to generate reasonable questions although the task is challenging, and highlight the importance of context to generate INQUIS-ITIVE questions.