The majority of today's power-hungry datacenters are physically co-located with office rooms in mixed-use buildings (MUBs). The heating, ventilation, and air conditioning (HVAC) system within each MUB is often shared or partially-shared between datacenter rooms and office zones, for removing the heat generated by computing equipment and maintaining desired room temperature for building tenants. To effectively reduce the total energy cost of MUBs, it is important to leverage the scheduling flexibility in both the HVAC system and the datacenter workload. In this work, we formulate both HVAC control and datacenter workload scheduling as a Markov decision process (MDP), and propose a deep reinforcement learning (DRL) based algorithm for minimizing the total energy cost while maintaining desired room temperature and meeting datacenter workload deadline constraints. Moreover, we also develop a heuristic DRL-based algorithm to enable interactive workload allocation among geographically distributed MUBs for further energy reduction. The experiment results demonstrate that our regular DRL-based algorithm can achieve up to 26.9 percent cost reduction for a single MUB, when compared with a baseline strategy. Our heuristic DRL-based algorithm can reduce the total energy cost by an additional 5.5 percent, when intelligently allocating interactive workload for multiple geographically distributed MUBs.

Smart buildings in the future are complex cyber-physical-human systems that involve close interactions among embedded platform (for sensing, computation, communication and control), mechanical components, physical environment, building architecture, and occupant activities. The design and operation of such buildings require a new set of methodologies and tools that can address these heterogeneous domains in a holistic, quantitative and automated fashion. In this paper, we will present our design automation methods for improving building energy efficiency and offering comfortable services to occupants at low cost. In particular, we will highlight our work in developing both model-based and data-driven approaches for building automation and control, including methods for co-scheduling heterogeneous energy demands and supplies, for integrating intelligent building energy management with grid optimization through a proactive demand response framework, for optimizing HVAC control with deep reinforcement learning, and for accurately measuring in-building temperature by combining prior modeling information with few sensor measurements based upon Bayesian inference.

Buildings account for nearly 40% of the total energy consumption in the United States, about half of which is used by the HVAC (heating, ventilation, and air conditioning) system. Intelligent scheduling of building HVAC systems has the potential to significantly reduce the energy cost. However, the traditional rule-based and model-based strategies are often inefficient in practice, due to the complexity in building thermal dynamics and heterogeneous environment disturbances. In this work, we develop a data-driven approach that leverages the deep reinforcement learning (DRL) technique, to intelligently learn the effective strategy for operating the building HVAC systems. We evaluate the performance of our DRL algorithm through simulations using the widely-adopted EnergyPlus tool. Experiments demonstrate that our DRL-based algorithm is more effective in energy cost reduction compared with the traditional rule-based approach, while maintaining the room temperature within desired range.

The recent breakthroughs of deep reinforcement learning (DRL) technique in Alpha Go and playing Atari have set a good example in handling large state and actions spaces of complicated control problems. The DRL technique is comprised of (i) an offline deep neural network (DNN) construction phase, which derives the correlation between each state-action pair of the system and its value function, and (ii) an online deep Q-learning phase, which adaptively derives the optimal action and updates value estimates. In this paper, we first present the general DRL framework, which can be widely utilized in many applications with different optimization objectives. This is followed by the introduction of three specific applications: the cloud computing resource allocation problem, the residential smart grid task scheduling problem, and building HVAC system optimal control problem. The effectiveness of the DRL technique in these three cyber-physical applications have been validated. Finally, this paper investigates the stochastic computing-based hardware implementations of the DRL framework, which consumes a significant improvement in area efficiency and power consumption compared with binary-based implementation counterparts.

Buildings account for nearly 40% of energy consumption in the United States. To improve energy efficiency and reduce peak demand, intelligent building management systems can be developed to manage the energy consumption of flexible loads such as heating, ventilation, and air conditioning (HVAC) system an electric vehicle (EV) charging, as well as the usage of energy storage systems such as batteries. In the case where a building cluster is managed by the same institution, coordinating the energy consumption behavior across multiple buildings can provide further benefits in energy efficiency. In this paper, we first invest gate an integrated co-scheduling scheme that uses a joint formulation to optimize the control of HVAC systems, EV charging a d battery storage in multiple buildings for reducing the overall energy cost. Then, we further explore a more efficient heuristic scheme where the shared battery storage and EV charging demand are assigned to each building for separate building-level scheduling. Our experiments demonstrate the effectiveness of our co-scheduling scheme and separate-scheduling heuristic in reducing energy cost for building clusters.

Buildings account for nearly 40 percent of the total energy consumption in the United States. As a critical step toward smart cities, it is essential to intelligently manage and coordinate the building operations to improve the efficiency and reliability of overall energy system. With the advent of smart meters and two-way communication systems, various energy consumptions from smart buildings can now be coordinated across the smart grid together with other energy loads and power plants. In this paper, we propose a comprehensive framework to integrate the operations of smart buildings into the energy scheduling of bulk power system through proactive building demand participation. This new scheme enables buildings to proactively express and communicate their energy consumption preferences to smart grid operators rather than passively receive and react to market signals and instructions such as time varying electricity prices. The proposed scheme is implemented in a simulation environment. The experiment results show that the proactive demand response scheme can achieve up to 10 percent system generation cost reduction and 20 percent building operation cost reduction compared with passive demand response scheme. The results also demonstrate that the system cost savings increase significantly with more flexible load installed and higher percentage of proactive customers participation level in the power network.

Demand response (DR) is an effective mechanism in improving power system efficiency and reducing energy cost for customers. However, DR processes might be vulnerable to cyber attacks from the usage of advanced metering infrastructure and wide-area network to exchange information. In this paper, we study potential attacks for a proactive demand participation scheme we recently proposed and for a conventional passive demand response scheme, particularly focusing on guideline price manipulation attacks. Our experiment results demonstrate that 1) guideline price manipulations may significantly lower the attacker's own electricity consumption cost while increasing other customers' cost, for both proactive and passive schemes; 2) such impact is less severe in the proactive scheme, i.e., the proactive demand participation scheme is more robust with respect to guideline price manipulation than the conventional DR.

As the building stock consumes 40% of the U.S. primary energy consumption, it is critically important to improve building energy efficiency. This involves reducing the total energy consumption of buildings, reducing the peak energy demand, and leveraging renewable energy sources, etc. To achieve such goals, hybrid energy supply has becoming popular, where multiple energy sources such as grid electricity, on-site fuel cell generators, solar, wind, and battery storage are scheduled together to improve energy efficiency.

Building stock consumes 40% of primary energy consumption in the United States. Among various types of energy loads in buildings, HVAC (heating, ventilation, and air conditioning) and EV (electric vehicle) charging are two of the most important ones and have distinct characteristics. HVAC system accounts for 50% of the building energy consumption and typically operates throughout the day, while EV charging is an emerging major energy load that is hard to predict and may cause spikes in energy demand. To maximize building energy efficiency and grid stability, it is important to address both types of energy loads in a holistic framework. Furthermore, on the supply side, the utilization of multiple energy sources such as grid electricity, solar, wind, and battery storage provides more opportunities for energy efficiency, and should be considered together with the scheduling of energy loads. In this paper, we present a novel model predictive control (MPC) based algorithm to co-schedule HVAC control, EV scheduling and battery usage for reducing the total building energy consumption and the peak energy demand, while maintaining the temperature within the comfort zone for building occupants and meeting the deadlines for EV charging. Experiment results demonstrate the effectiveness of our approach under a variety of demand, supply and environment constraints.