The virtualization of Radio Access Networks (vRAN) is rapidly becoming a reality, driven by the increasing need for flexible, scalable, and cost-effective mobile network solutions. To mitigate energy efficiency concerns in vRAN deployments, two approaches are gaining attention: ($i$) sharing computing infrastructure among multiple virtualized base stations (vBSs); and ($ii$) relying upon general-purpose, low-cost CPUs. However, effectively realizing these approaches poses several challenges. In this paper, we first conduct a comprehensive experimental campaign on a vRAN platform to characterize the impact of computing and radio resource allocation on energy consumption and performance across various network contexts. This analysis reveals several key issues. First, determining the optimal allocation of computing resources is difficult because it depends on the context of each vBS (e.g., traffic load, channel quality) in a non-trivial and non-linear manner. Second, suboptimal resource assignment can lead to increased energy consumption or, even worse, degradation of users' Quality of Service. Third, the high dimensionality of the solution space hinders the effectiveness of traditional optimization or learning methods. To tackle these challenges, we propose AegisRAN, a framework for optimizing computing resource allocation in vRAN. AegisRAN addresses the dual objective of minimizing energy consumption while maintaining high system reliability. Moreover, when computing resources are overbooked, our solution ensures a fair resource partition based on vBS performance. AegisRAN leverages a discrete soft actor-critic algorithm combined with several techniques, including multi-step decision-making, action masking, digital twin-based training, and a tailored reward signal that mitigates feedback sparsity. Our evaluations demonstrate that AegisRAN achieves near-optimal performance and offers high flexibility across diverse network contexts and varying numbers of vBSs, with up to 25% improvement in energy savings compared to baseline solutions in medium-scale scenarios.