Online lot sizing for perishable materials in an uncertain environment is a fundamental problem for inventory planning and has been studied for several decades. In this article, we study a novel setting of thelotsizing problem, consideringperishable materials,multiple suppliers,uncertain demands and lead time (LS-PMU), which captures the inventory planning task in real life better than existing lot sizing problems. We present theoretical results of the best possible competitive ratio an online algorithm can achieve for the LS-PMU problem. We then develop a reinforcement learning-based algorithm calledRL4LSto intelligently choose the supplier and decide the order quantity in each time period. We conduct extensive experiments on both real and synthetic datasets to verify thatRL4LSoutperforms existing algorithms in terms of effectiveness and efficiency, e.g.,RL4LSimproves the effectiveness by 44% and runs two orders of magnitude faster than the state-of-the-art algorithmIBFA.