Naturally interacting robots must be able to understand natural human speech. As such, recent work has sought to allow robots to infer the intentions behind commonly used non-literal utterances such as indirect speech acts (ISAs). However, it is still unclear to what extent ISAs will actually be used in task-based human-robot dialogue, and to what extent robots could function without the ability to understand ISAs. In this paper, we present the results of a Wizard-of-Oz experiment that examined human ISA use in scenarios that did or did not have conventionalized social norms, and analyzed both ISA use and perceptions of robots when robots were or were not capable of understanding ISAs. Our results suggest that (1) ISAs are commonly used in task-based human-robot dialogues, even when robots show themselves unable to understand ISAs; (2) ISA use is more common in contexts with conventionalized social norms; and (3) a robot's inability to understand ISAs harms both the robot's task performance and human perception of the robot.