We introduce SibylOpt, a system that applies deep reinforcement learning (DRL) to optimize the operation of a green data center (DC). SibylOpt uses an offline reinforcement learning algorithm, Advantage-Weighted Actor-Critic (AWAC), that requires historical data for training but does not depend on a DC simulator, which is effort intensive to build and keep updated as the DC evolves. SibylOpt then augments the offline training with online learning for refinement and adaptation to changes. We apply SibylOpt to the management of a small green DC that has onsite solar energy generation and a hybrid cooling system that includes "free-cooling." Evaluation results (using simulation) show that the offline trained SibylOpt achieves higher rewards trading off job wait time, cooling, and grid electricity consumption compared to two baseline policies. It is also competitive with PPO, a DRL approach that requires an accurate DC simulator for training.

Managing datacenters to maximize efficiency and sustain-ability is a complex and challenging problem. In this work, we explore the use of deep reinforcement learning (RL) to manage "green" datacenters, bringing a robust approach for designing efficient management systems that account for specific workload, datacenter, and environmental characteristics. We design and evaluate GreenDRL, a system that combines a deep RL agent with simple heuristics to manage workload, energy consumption, and cooling in the presence of onsite generation of renewable energy to minimize brown energy consumption and cost. Our design addresses several important challenges, including adaptability, robustness, and effective learning in an environment comprising an enormous state/action space and multiple stochastic processes. Evaluation results (using simulation) show that GreenDRL is able to learn important principles such as delaying deferrable jobs to leverage variable generation of renewable (solar) energy, and avoiding the use of power-intensive cooling settings even at the expense of leaving some renewable energy unused. In an environment where a fraction of the workload is deferrable by up to 12 hours, GreenDRL can reduce grid electricity consumption for days with different solar energy generation and temperature characteristics by 32--54% compared to a FIFO baseline approach. GreenDRL also matches or outperforms a management approach that uses linear programming together with oracular future knowledge to manage workload and server energy consumption, but leaves the management of the cooling system to a separate (and independent) controller. Overall, our work shows that deep RL is a promising technique for building efficient management systems for green datacenters.