The rapid and accurate identification of plant diseases is a critical challenge in agriculture, impacting cost, crop health and agricultural productivity. Traditional methods of disease detection heavily rely on manual inspection, which can be time-consuming, subjective, and prone to human error. In this work, we propose an end-to-end system for automated detection of disease with severity level in tomato plants using a fusion of deep learning methods with image processing techniques. Farmers can reduce the needless and excessive use of pesticides by using targeted and optimized treatment plans by precisely measuring the severity level of the disease. We have developed a testbed to show the effectiveness of our proposed end-to-end spray control in terms of accuracy of disease classification, severity detection, and amount of pesticide reduction. In comparison to the traditional spray approach, our suggested setup resulted in 20% pesticide reductions and a 29.68% energy savings.