Inhibitory interneurons play critical roles in shaping the firing patterns of principal neurons in many brain systems. Despite difference in the anatomy or functions of neuronal circuits containing inhibition, two basic motifs repeatedly emerge: feed-forward and feedback. In the locust, it was proposed that a subset of lateral horn interneurons (LHNs), provide feed-forward inhibition onto Kenyon cells (KCs) to maintain their sparse firing--a property critical for olfactory learning and memory. But recently it was established that a single inhibitory cell, the giant GABAergic neuron (GGN), is the main and perhaps sole source of inhibition in the mushroom body, and that inhibition from this cell is mediated by a feedback (FB) loop including KCs and the GGN. To clarify basic differences in the effects of feedback vs. feed-forward inhibition in circuit dynamics we here use a model of the locust olfactory system. We found both inhibitory motifs were able to maintain sparse KCs responses and provide optimal odor discrimination. However, we further found that only FB inhibition could create a phase response consistent with data recorded in vivo. These findings describe general rules for feed-forward versus feedback inhibition and suggest GGN is potentially capable of providing the primary source of inhibition to the KCs. A better understanding of how inhibitory motifs impact post-synaptic neuronal activity could be used to reveal unknown inhibitory structures within biological networks.