In human-to-human conversations, people sometimes interpret hesitations from their conversational partners as a clue for rejection (e.g., “ um I’ll tell you later if I could come to the party” may be interpreted as “I won’t come to the party”). This type of interpretation is deeply embedded in human talkers’ understanding of social etiquette and modeling of the state of mind of the interlocutors. In this study, we examine how human listeners interpret hesitations in robot speech in a human-robot interactive context, as compared to how they interpret human-produced hesitations. In Experiment 1, participants ( N = 63) watched videos of conversations between a humanoid robot talker and a human talker, where the robot talker would give responses, with or without hesitations, to the human talker’s requests or inquiries. The participants then completed a memory test of what they remembered from the conversations. The memory test results showed that participants were significantly more likely to interpret hesitant responses from the robot as rejections compared to completely fluent robot responses. The hesitation-triggered bias toward negative interpretations was replicated in Experiment 2 with a separate group of participants ( N = 59), who listened to the same conversations but as human-to-human interactions. Combined analysis found no difference in the magnitude of the hesitation bias between the two conditions. These results provide evidence that human listeners draw similar inferences from hesitant speech produced by robots and those by human talkers. This study offers valuable insights for the future design of conversational AI agents, highlighting the importance of subtle speech cues in human-machine interaction.