For the past two decades, sustainability and carbon reduction have emerged as critical factors for data center (DC) design and operation. A set of advances has been encapsulated in green DCs with the onsite generation of renewable energy and efficient cooling systems. In this paper, we study how to apply Deep Reinforcement Learning (DRL) to optimize green DC operation. Green DC management is typically an infinite-horizon problem with exogenous stochastic input processes. We propose EA, a framework that applies DRL to the typical infinite-horizon problem without discounting. EA approximates the infinite-horizon problem with a finite-horizon one. In this approach, it is important to avoid actions optimized for the end of the finite-horizon problem but inappropriate for the true infinite-horizon one. EA addresses this challenge by combining a stationary policy with the fact that green DC management has repeating patterns (e.g., daily temperature, solar energy generation, and workload). We apply EA to the management of a green DC with onsite solar energy generation and a hybrid cooling system that includes ``free'' cooling. Evaluation results show that EA successfully learns important principles such as delaying deferrable jobs to solar-rich times and gracefully maintaining inside temperature. Further, EA outperforms three state-of-the-art DRL algorithms, realizing the greatest benefits on days with high outside temperature and high solar generation. While we evaluate EA in the specific context of a green DC, we believe that EA is a promising approach for more general system management settings.

We introduce SibylOpt, a system that applies deep reinforcement learning (DRL) to optimize the operation of a green data center (DC). SibylOpt uses an offline reinforcement learning algorithm, Advantage-Weighted Actor-Critic (AWAC), that requires historical data for training but does not depend on a DC simulator, which is effort intensive to build and keep updated as the DC evolves. SibylOpt then augments the offline training with online learning for refinement and adaptation to changes. We apply SibylOpt to the management of a small green DC that has onsite solar energy generation and a hybrid cooling system that includes "free-cooling." Evaluation results (using simulation) show that the offline trained SibylOpt achieves higher rewards trading off job wait time, cooling, and grid electricity consumption compared to two baseline policies. It is also competitive with PPO, a DRL approach that requires an accurate DC simulator for training.

Managing datacenters to maximize efficiency and sustain-ability is a complex and challenging problem. In this work, we explore the use of deep reinforcement learning (RL) to manage "green" datacenters, bringing a robust approach for designing efficient management systems that account for specific workload, datacenter, and environmental characteristics. We design and evaluate GreenDRL, a system that combines a deep RL agent with simple heuristics to manage workload, energy consumption, and cooling in the presence of onsite generation of renewable energy to minimize brown energy consumption and cost. Our design addresses several important challenges, including adaptability, robustness, and effective learning in an environment comprising an enormous state/action space and multiple stochastic processes. Evaluation results (using simulation) show that GreenDRL is able to learn important principles such as delaying deferrable jobs to leverage variable generation of renewable (solar) energy, and avoiding the use of power-intensive cooling settings even at the expense of leaving some renewable energy unused. In an environment where a fraction of the workload is deferrable by up to 12 hours, GreenDRL can reduce grid electricity consumption for days with different solar energy generation and temperature characteristics by 32--54% compared to a FIFO baseline approach. GreenDRL also matches or outperforms a management approach that uses linear programming together with oracular future knowledge to manage workload and server energy consumption, but leaves the management of the cooling system to a separate (and independent) controller. Overall, our work shows that deep RL is a promising technique for building efficient management systems for green datacenters.

Software memory disclosure attacks, such as buffer over-read, often work quietly and would cause secret data leakage. The well-known OpenSSL Heartbleed vulnerability leaked out millions of servers' private keys, which caused most of the Internet services insecure at that time. Existing solutions are either hard to apply to large code bases (e.g., through formal verification [20] or symbolic execution [8] on program code), or too heavyweight (e.g., by involving a hypervisor software [23], [24] or a modified operating system kernel [17]). In this paper, we propose SecretSafe, a lightweight and easy-to-use system which leverages the traditional x86 segmentation mechanism to isolate the application secrets from the remaining data. Software developers could prevent the secrets from being leaked out by simply declaring the secret variables with SECURE keyword. Our customized compiler will automatically separate the secrets from the remaining non-secret data with an isolated memory segment. Any legal instructions that have to access the secrets will be automatically instrumented to enable accesses to the isolated segment. We have implemented a SecretSafe prototype with the open source LLVM compiler framework. The evaluation shows that SecretSafe is both secure and efficient.

We consider the scenario where a cloud service provider (CSP) operates multiple geo-distributed datacenters to provide Internet-scale service. Our objective is to minimize the total electricity and bandwidth cost by jointly optimizing electricity procurement from wholesale markets and geographical load balancing (GLB), i.e., dynamically routing workloads to locations with cheaper electricity. Under the ideal setting where exact values of market prices and workloads are given, this problem reduces to a simple LP and is easy to solve. However, under the realistic setting where only distributions of these variables are available, the problem unfolds into a non-convex infinite-dimensional one and is challenging to solve. Our main contribution is to develop an algorithm that is proven to solve the challenging problem optimally and efficiently, by exploring the full design space of strategic bidding. Trace-driven evaluations corroborate our theoretical results, demonstrate fast convergence of our algorithm, and show that it can reduce the cost for the CSP by up to 20% as compared to baseline alternatives. Our study highlights the intriguing role of uncertainty. While variability in workloads deteriorates the cost-saving performance of joint electricity procurement and GLB, counter-intuitively, variability in market prices can be exploited to achieve a cost reduction even larger than the setting without price variability.

The recent years witnessed rapid emergence and proliferation of cloud computing. To fully utilize the compute resources, some cloud operators provide spot resources. Spot resources allow customers to bid on unused capacity. However, pricing policy of spot resources should be carefully designed and the impact on both present and future should be considered. For the present, the cloud provider can set a higher price to gain extra revenue. For the future, higher price will shift more requests with lower prices to later time and reduce the revenue of future. Meanwhile, the quality of service should be considered either since bad QoS will incur loss of potential users. In this paper, we present a demand curve to model the impact of pricing on the present and future revenue. Then we formulate the revenue maximization problem as a time-average optimization problem. Next, since this basic model fails to provide information of service delay, we extend it to a more generalized one that ensures the worst-case delay of user requests. While the future knowledge of arrival requests is unknown, it is necessary to design online algorithms for the optimization problems. We apply Lyapunov optimization framework and design an efficient online algorithm which dose not require any future knowledge of requests arrival. Evaluations based on real-life datacenter workload and Amazon EC2 Spot Price illustrate efficiency of our algorithms.

The study of today's Cyber-Physical System (CPS) has been an important research area. Internet Data Centers (IDCs) are energy consuming CPSs that support the reliable operations of many important online services. Along with the increasing Internet services and cloud computing in recent years, the power usage associated with IDC operations had been surging significantly. Such mass power consumption has brought extremely heavy burden on IDC operators. Recently there are extensive research on power management for IDCs. While most work only consider about dynamical optimization of IDC under electricity markets, the reaction of IDC toward electricity market has been overlooked. Due to the fact that IDCs are usually large-volume users in the electricity market, they might have market power to affect the electricity price. In this paper, we study how to address the challenge of interactions between IDC operation and electricity market price. To this end, we propose a supply function to model the market power of IDC and formulate a total electricity cost minimization problem as a non-linear programming. In order to design efficient solution method, we transform the optimization problem to a quadratic programming. Extensive performance evaluations demonstrate that the proposed method can effectively minimize the total electricity cost of IDCs by adaptively handling the interaction between IDCs and smart grid.

Complex web applications are usually served by multi-tier web clusters. With the growing cost of energy, the importance of reducing power consumption in server systems is now well-known and has become a major research topic. However, most of previous works focused solely on homogeneous clusters. This paper addresses the challenge of power management in Heterogeneous Multi-tier Web Clusters. We apply Generalized Benders Decomposition (GBD) to decompose the global optimization problem into small sub-problems. This algorithm achieves the optimal solution in an iterative fashion. The simulation results show that our algorithm achieve more energy conservation than the previous works.