Resistive random-access memory (ReRAM) offers a potential solution to accelerate the inference of deep neural networks by performing processing-in-memory. However, the peripheral circuits of ReRAM crossbars used to perform arithmetic operations consume significant amounts of power. Based on a power consumption analysis of the ReRAM crossbar circuits, we propose using the dynamic reference voltage scalable analog-to-digital circuits (ADCs) to conduct the dot product operation to enable the reconfigurability of the ReRAM-based neural network (NN) accelerator while maintaining accuracy. We propose a configurable ReRAM-based NN accelerator to provide various degrees of computing granularity with different levels of power consumption, creating a tradeoff between performance and power consumption in the given NN. Next, we develop an energy-efficient inference engine for the configurable ReRAM-based NN accelerator, EIF, to assign the operation unit (OU) size to perform vector-matrix multiplication (VMM) based on the data dependence of the NN. Our evaluation shows that the proposed EIF provided an energy savings of up to 36% over the state-of-the-art ReRAM-based accelerator while maintaining performance without resource duplication.