Capacity management of Enterprise Applications (EAs) encompasses critical IT processes that maximize the IT system’s performance while minimizing operational costs. Effective management of EAs capacity can be enhanced through precise anomaly detection and workload forecasting. Given sufficient historical monitoring data of EAs, Machine Learning (ML) algorithms like Isolation Forest and XGBoost can be applied to address anomaly detection and forecasting tasks. However, the performance of these algorithms can strongly depend on the selected hyper-parameter values. Hence, Hyper-Parameter Optimization (HPO) is crucial for successfully adopting ML methods to solve real-world problems. The existing tuning methods like manual tuning, Grid Search, and Random Search often tend to be computationally expensive and time-consuming, making them ineffective for high-dimensional data. On the other hand, recent successes of Deep Reinforcement Learning (DRL) in handling various optimization problems have sparked interest in exploring its potential. Therefore, this work investigates the adaptation of DRL as a novel HPO approach. We present two implementation scenarios deployed in two real-world use cases and compare the results to the state-of-the-art tuning algorithms. The experiments indicate that the DRL algorithms can be adopted as tuning techniques since they demonstrate consistent policy learning and overall reward maximization.