This paper develops a distributed reinforcement learning (RL) method to coordinate cooperative microgrids (MGs). The high uncertainty of power loads and renewable energy sources motivate the operator to perform real-time dispatch. On the one hand, the existing online methods usually utilize approximate models that result in intractable constraint violation. A common method is to relax it as a chance constraint, while it is still hard to ensure its satisfaction in practice. On the other hand, some MGs may hope to preserve the private information on their local costs and states. To address these problems, we make the following contributions. First, the coordination problem is reformulated as a constrained multi-agent Markov decision process. Second, the distributed RL algorithm with a theoretical convergence guarantee is developed. Third, to further preserve the local private information and improve the performance, this algorithm is modified by adding a local feature extraction module for each agent. This module could also be regarded as an encryption module for the local state information. Fourth, numerical experiments are carried out to validate the effectiveness of the modified algorithm.

With the growing complexity of modern micro-architectures, processor design must accommodate a wide array of parameters, resulting in vast design spaces. Identifying the optimal trade-offs among various design metrics poses a significant challenge. Performance is inherently difficult to model analytically, while area can be represented by analytical models. Performance evaluation relies on expensive and time-consuming cycle-accurate simulators (CAS), which puts forward strict requirements on the convergence speed and data dependence of the optimization methods. In this paper, based on the characteristics of the design metrics, the processor design problem is modeled as a hybrid black-box and white-box multi-objective discrete optimization problem (BWMO-DOP). In engineering applications, parallel simulation is a common acceleration technology. Therefore, white-box objective, area, is formulated as parallel constraints, while black-box objective, performance, is approximated by order-preserving surrogate objective. And then, BWMO-DOP is simplified into parallel single-objective expensive black-box optimization problems, which are solved by an efficient SOP-MOOA. SOP-MOOA iteratively explores more design points, enhancing the accuracy of the surrogate model while simultaneously updating the Pareto set. Experimental results demonstrate that the proposed algorithm outperforms baseline algorithm in an engineering case and three general numerical cases. In the engineering experiment for performance-area optimization, the proposed algorithm outperforms the baseline algorithm by a factor of more than two when considering the combined effect of performance improvement percentage and area reduction percentage. In numerical tests conducted on three 40-dimensional black-box functions under the same evaluation overhead, the proposed algorithm consistently identified Pareto set of superior quality.

Design Space Exploration (DSE) in processors is an expensive discrete simulation optimization problem. The data requirements of the regular data-driven methods are so large that it is challenging to converge to a satisfactory solution within a limited simulation budget. Based on binary integer linear programming (BILP), an iteratively piecewise linear approximate method (IPLAM) is proposed for this kind of problems to reduce the dependence of simulation data. IPLAM starts from an initial reference point. Each iteration generates a set of trial points based on the most promising reference point by piecewise shift method. After evaluating the trial points, a local surrogate model is constructed for the unit neighborhood of the reference point. The surrogate model is then used to guide the exploration of the next reference point. In theory, IPLAM can converge to the global optimal point under mild assumptions, which is further verified by the numerical experiments. Meanwhile, the numerical experiments demonstrate that IPLAM outperforms the advanced Bayesian optimization and differential evolution methods on high-dimensional discrete black-box test functions. Besides, the practical effectiveness of IPLAM is validated by an industrial case for processor DSE.

To address the power system hardening problem, traditional approaches often adopt robust optimization (RO) that considers a fixed set of concerned contingencies, regardless of the fact that hardening some components actually renders relevant contingencies impractical. In this paper, we directly adopt a dynamic uncertainty set that explicitly incorporates the impact of hardening decisions on the worst-case contingencies, which leads to a decision-dependent uncertainty (DDU) set. Then, a DDU-based robust-stochastic optimization (DDU-RSO) model is proposed to support the hardening decisions on distribution lines and distributed generators (DGs). Also, the randomness of load variations and available storage levels is considered through stochastic programming (SP) in the innermost level problem. Various corrective measures (e.g., the joint scheduling of DGs and energy storage) are included, coupling with a finite support of stochastic scenarios, for resilience enhancement. To relieve the computation burden of this new hardening formulation, an enhanced customization of parametric column-and-constraint generation (P-C&CG) algorithm is developed. By leveraging the network structural information, the enhancement strategies based onresilience importance indicesare designed to improve the convergence performance. Numerical results on 33-bus and 118-bus test distribution networks have demonstrated the effectiveness of DDU-RSO aided hardening scheme. Furthermore, in comparison to existing solution methods, the enhanced P-C&CG has achieved a superior performance by reducing the solution time by a few orders of magnitude.

The efficiency and profitability of cloud data centers are significantly influenced by virtual machine (VM) placement. However, the Non-Uniform Memory Access (NUMA), which has been practically applied to reduce the memory bandwidth competition, is often neglected in the existing research. Actually, the incorporation of NUMA may change the traditional resource allocation mechanism, and demands for a new VM placement model. Hence, considering the multi-NUMA architecture, this paper studies the NUMA-aware VM placement (NAVMP) problem in a cloud computing system, where the resource pool is composed of enormous number of heterogeneous servers with diverse multi-resource remains. The NAVMP problem is analytically formulated as an integer program (IP). Also, for the first time, the incarnations of VM types are introduced to simplify the VM deployment rules originated from complex NUMA architecture. We aim to maximize the VM provision ability (VPA) of the resource pool, and thus propose a novel Value Function to describe servers’ VPA. The resulting formulation, which is a new variant of the multiple-choice multiple multi-dimensional knapsack (MMMK) problem, is of significant computational challenges. So we customize a decomposition approach based on Column Generation (CG) to support the offline optimization. Numerical experiments on a practical dataset demonstrate the validity and scalability of the customized CG-based approach. Our approach outperforms a professional IP solver, i.e., Cbc, and a popular meta-heuristic algorithm, i.e., genetic algorithm (GA), and can efficiently address large-scale NAVMP instances with ten thousands of VM demands and servers.Note to Practitioners—This paper proposes a novel IP model for NAVMP. To cope with the complicated deployment logic associated with the complex multi-NUMA architecture of modern multi-core systems, we present an NAVMP formulation from the perspective of incarnations of VM types. Different from the traditional VM placement problem that aims to minimize the number of activated servers, i.e., the vector bin packing (VBP)-based model, we adopt the objective that maximizes the VPA of a resource pool for further improving the resource utilization. The resulting formulation is an MMMK problem, which is computational very challenging for a practical scale resource pool. Hence, to mitigate the computation burden, we design and implement a CG-based decomposition approach to support the offline optimization for NAVMP. Parallelization scheme and nontrivial heuristic strategies are applied to promote the computation efficiency. According to our numerical experiments, the proposed decomposition approach demonstrates a much superior solution capacity to the Cbc solver and GA. In particular, to achieve a comparable solution precision with Cbc, the computing time can be reduced by orders of magnitude. Also the CG-based approach outperforms GA in both the solution quality and computation time for large-scale instances. Besides, compared to the VBP model, our MMMK-based NAVMP model has improved the VPA up to 44.39%. Practically, the proposed offline approach can be leveraged to guide online VM allocation decisions, and perform efficient results evaluation.

This paper analyzes the evacuation process of people in a building, and constructs a discrete-time evacuation model that can accurately describe the evacuation problem according to evacuation scenarios. In the discrete time framework, the evacuation policy is dynamically adjusted based on the dynamic transfer of people. For the large-scale evacuation problem, this paper proposes a two-stage method to solve the evacuation policy of edge and node separately, which improves the solving efficiency. The results of case study prove the reliability and efficiency of our method. Note to Practitioners—In this paper, considering the influence of effective edge width and crowd density on the moving speed of people, a discrete time based evacuation model is constructed which accurately reflects the evacuation process. A two-stage method is proposed to solve the difficult problem of large-scale evacuation. The evacuation policy obtained by the two-stage method will be used as the evacuation plan, and the evacuation plan of different evacuation scenarios will be counted. When an evacuation event occurs, the distribution of people is matched with the scenario in the database, and the evacuation plan of the closest scenario is selected for evacuation.

The robust unit commitment (UC) is of paramount importance for achieving reliable operations considering the uncertainty of renewable realizations. The typical affine decision rule method and the robust feasible region method may achieve uneconomic dispatches as the dispatch decisions just rely on the current-stage information. Through approximating the future cost-to-go functions, the dual dynamic programming based methods have been shown adaptive to the multistage robust optimization problems, while suffering from high computational complexity. Thus, we propose the robust approximate dynamic programming (RADP) method to promote the computational speed and the economic performance for large-scale robust UC problems. RADP initializes the candidate points for guaranteeing the feasibility of upper bounding the value functions, solves the alternating calculation based bilinear programming to obtain the worst cases, and combines the primal and dual updates for the two-phase robust UC decision-making problem to achieve fast convergence. The finite termination guarantee of the RADP method is verified by the analyses for the multistage robust optimization problems with achieving suboptimal solutions. Numerical tests on 118-bus and 2383-bus transmission systems have demonstrated that RADP can approach the suboptimal economic performance at significantly improved computational efficiency.Note to Practitioners—This paper was motivated by solving the large-scale robust UC problem embedded with the multistage economic dispatch with improved computational and economical performance. Compared to the existing methods, this work suggests a RADP-based approach, which is inspired by the robust dual dynamic programming (RDDP) scheme. The proposed RADP owns lower computational complexity when compared with RDDP. As the problem in this work is formulated in a general form, the proposed RADP can be applied to other robust optimization problems such as the inventory management problem with uncertain demands. To apply the method to large-scale decision-making problems, one needs to solve linear programming problems to obtain the finite upper/lower bounds first, and then initialize the upper-bound points based on the feasible region limits of the decision variables. Conduct the forward pass to generate the candidate points and the backward pass to refine the cost-to-go functions. If the application problems have discrete decision variables as in the two-phase UC, one can solve the nonanticipativity constrained problem to obtain discrete solutions before doing the RADP scheme. The analyses and numerical experiments suggest that this approach can achieve suboptimal solutions. In future research, we will address the accelerated multistage robust decision-making with achieving optimal solutions.

The concept of VES provides a new way that utilizes the existing resources and devices to achieve functions similar to an energy storage system (ESS) without introducing physical energy storage devices. By adjusting the loads of energy-intensive devices or the output of power generation resources in the existing self-supply entity (SSE), VES can realize power transfer functions and provide regulation services to power systems according to the scheduling strategies. In this context, this paper mainly discusses the economic dispatch problem with VES under uncertainties and proposes a novel multi-stage method while guaranteeing nonanticipativity and multi-stage robustness. Firstly, a conceptual single-level multi-stage robust economic dispatch model is proposed subject to the systematic constraints of VES. To make the economic dispatch model tractable with solution feasibility guaranteed, a specific feasible region for dispatch variables is defined based on the constraints’ structure. Based on the feasible region, the original model can be transformed into a mixed integer linear programming (MILP) problem and can be implemented via a rolling horizon. Numerical results in a real area illustrate the effective performance of the proposed method.Note to Practitioners—This article is motivated to realize the function of the storage facility by scheduling the existing self-supply entity, defined as virtual energy storage (VES). And the scheduling method can be applied to deal with other multi-stage robust problems with the same constraints’ structure. One of the main challenges in scheduling systems with VES under uncertainty is to decouple the ramp-rate constraints of thermal units and VES as well as the nonlinear coupling constraints of VES. To make the problem tractable with solution feasibility guaranteed, two groups of nonanticipative and multi-stage robust constraints are introduced based on the constraints’ structure. An efficient method is then established based on the feasible condition and the method performs better than the existing methods. The proposed method allows the decision maker to obtain the economic scheduling decisions of the VES more efficiently.

Estimating set similarity and detecting highly similar sets are fundamental problems in areas such as databases and machine learning. MinHash is a well-known technique for approximating Jaccard similarity of sets and has been successfully used for many applications. Its two compressed versions, b-bit MinHash and Odd Sketch, can significantly reduce the memory usage of the MinHash, especially for estimating high similarities (i.e., similarities around 1). Although MinHash can be applied to static sets as well as streaming sets, of which elements are given in a streaming fashion, unfortunately, b-bit MinHash and Odd Sketch fail to deal with streaming data. To solve this problem, we previously designed a memory-efficient sketch method, MaxLogHash, to accurately estimate Jaccard similarities in streaming sets. Compared with MinHash, our method uses smaller sized registers (each register consists of less than 7 bits) to build a compact sketch for each set. In this paper, we further develop a faster method, MaxLogOPH++. Compared with MaxLogHash, MaxLogOPH++ reduces the time complexity for updating each coming element from O(k) with a small additional memory. We conduct experiments on a variety of datasets, and experimental results demonstrate the efficiency and effectiveness of our methods.

Estimating set similarity and detecting highly similar sets are fundamental problems in areas such as databases, machine learning, and information retrieval. MinHash is a well-known technique for approximating Jaccard similarity of sets and has been successfully used for many applications such as similarity search and large scale learning. Its two compressed versions, b-bit MinHash and Odd Sketch, can significantly reduce the memory usage of the original MinHash method, especially for estimating high similarities (i.e., similarities around 1). Although MinHash can be applied to static sets as well as streaming sets, of which elements are given in a streaming fashion and cardinality is unknown or even infinite, unfortunately, b-bit MinHash and Odd Sketch fail to deal with streaming data. To solve this problem, we design a memory efficient sketch method, MaxLogHash, to accurately estimate Jaccard similarities in streaming sets. Compared to MinHash, our method uses smaller sized registers (each register consists of less than 7 bits) to build a compact sketch for each set. We also provide a simple yet accurate estimator for inferring Jaccard similarity from MaxLogHash sketches. In addition, we derive formulas for bounding the estimation error and determine the smallest necessary memory usage (i.e., the number of registers used for a MaxLogHash sketch) for the desired accuracy. We conduct experiments on a variety of datasets, and experimental results show that our method MaxLogHash is about 5 times more memory efficient than MinHash with the same accuracy and computational cost for estimating high similarities.