Accurate prediction of continuous targets often requires balancing model flexibility and generalisation. Classical regression models, while interpretable, struggle to capture complex nonlinearities and are highly sensitive to outliers, leading to biased estimates on heterogeneous data [13], [14]. Conversely, powerful ensemble-based regressors, such as gradient boosting methods (e.g., XGBoost, LightGBM, CatBoost) considered state-of-the-art in many tabular tasks, can fit intricate patterns but may suffer from high variance and sensitivity to outliers in low-data scenarios [9], [35]. To bridge this gap, we introduce a hybrid stacking framework that first discretises the response variable via quantiles—preserving distributional information—then learns high-level patterns with multiple classifiers, and finally reconciles these signals through a regularised polynomial meta-regressor. This design combines the robustness and interpretability of classical techniques with the representational power of modern ensembles while retaining classifier-agnostic flexibility.

Short-term electricity demand forecasting is a fundamental part of the decision-making process of entities involved in electricity consumption management, since it allows the development of strategies to deal with variations in electricity demand in short periods of time. Developing a highly-accurate predictive model is necessary to understand and reflect the consumption behavior, as well as to adjust the generation program essentially to meet the demand at each moment. Therefore, a performance comparison has been made between statistical, machine learning and deep learning models for short-term forecasting. The deep learning models are based on recurrent neural networks, incorporating attention mechanisms in some of them. Hyperparameter tuning was also applied using a Bayesian optimization algorithm. A dataset was also developed including historical electricity demand and external factors such as weather and calendar variables recorded in Paraguay from 2009 to 2022. Models were evaluated from a set of numerical experiments using classical error metrics such as: mean squared error, root mean square error, mean absolute error and mean percentage absolute error. In addition, new special measures were introduced to analyze the error in this type of applications: the percentage error at peak hour and the maximum percentage error of the day to analyze the error in certain events.