For automated vehicle, motion planning especially multi-vehicle cooperative motion planning (MVCMP), is an important and challenging problem, while the high-dimensional coupled nonlinear constraints make it become a highly non-convex nonlinear programming (NLP) problem. In this paper, we propose a parallel computing and convexifying guidance framework based on alternating direction method of multipliers (ADMM) and constraint convexification. By applying ADMM and introducing auxiliary variables, the original problem is decomposed into two types of sub-problems that can be solved in a parallel manner. The first type of sub-problem only contains kinematics constraints and can be tackled by the proposed ‘First Solve Then Regulate Time’ (FSTRT) method, which can eliminate the time coupling and divide this sub-problem into multiple parallel secondary sub-problems for each vehicle. The second type of sub-problem only considers collision avoidance constraints among multiple vehicles, and can also be solved in parallel based on copy variable. In particular, the terminal guided convex feasible set (TG-CFS) algorithm and convexifying guidance strategy are proposed to achieve constraint convexification and efficient warm-start for this sub-problem. The advantage of proposed algorithm lies at effectively resolving constraint coupling and reduce the dimension of original problem, thereby achieving parallel and fast computation of the sub-problems. Simulation and comparison results of different cases indicate that the proposed algorithm can significantly improve the computational efficiency while preserving the optimality.

Online reinforcement learning (RL) typically requires online interaction data to learn a policy for a target task, but collecting such data can be high-stakes. This prompts interest in leveraging historical data to improve sample efficiency. The historical data may come from outdated or related source environments with different dynamics. It remains unclear how to effectively use such data in the target task to provably enhance learning and sample efficiency. To address this, we propose a hybrid transfer RL (HTRL) setting, where an agent learns in a target environment while accessing offline data from a source environment with shifted dynamics. We show that – without information on the dynamics shift – general shifted-dynamics data, even with subtle shifts, does not reduce sample complexity in the target environment. However, focusing on HTRL with prior information on the degree of the dynamics shift, we design HySRL, a transfer algorithm that outperforms pure online RL with problem-dependent sample complexity guarantees. Finally, our experimental results demonstrate that HySRL surpasses the state-of-the-art pure online RL baseline.

We address the problem of characterizing the aggregate flexibility in populations of electric vehicles (EVs) with uncertain charging requirements. Extending upon prior results that provide exact characterizations of aggregate flexibility in populations of electric vehicle (EVs), we adapt the framework to encompass more general charging requirements. In doing so we give a characterization of the exact aggregate flexibility as a generalized polymatroid. Furthermore, this paper advances these aggregation methodologies to address the case in which charging requirements are uncertain. In this extended framework, requirements are instead sampled from a specified distribution. In particular, we construct robust aggregate flexibility sets, sets of aggregate charging profiles over which we can provide probabilistic guarantees that actual realized populations will be able to track. By leveraging measure concentration results that establish powerful finite sample guarantees, we are able to give tight bounds on these robust flexibility sets, even in low sample regimes that are well suited for aggregating small populations of EVs. We detail explicit methods to tractably compute these sets. Finally, we provide numerical results that validate our results and case studies that demonstrate the applicability of the theory developed herein.

Multi-objective reinforcement learning (MORL) excels at handling rapidly changing preferences in tasks that involve multiple criteria, even for unseen preferences. However, previous dominating MORL methods typically generate a fixed policy set or preference-conditioned policy through multiple training iterations exclusively for sampled preference vectors, and cannot ensure the efficient discovery of the Pareto front. Furthermore, integrating preferences into the input of policy or value functions presents scalability challenges, in particular as the dimension of the state and preference space grow, which can complicate the learning process and hinder the algorithm's performance on more complex tasks. To address these issues, we propose a two-stage Pareto front discovery algorithm called Constrained MORL (C-MORL), which serves as a seamless bridge between constrained policy optimization and MORL. Concretely, a set of policies is trained in parallel in the initialization stage, with each optimized towards its individual preference over the multiple objectives. Then, to fill the remaining vacancies in the Pareto front, the constrained optimization steps are employed to maximize one objective while constraining the other objectives to exceed a predefined threshold. Empirically, compared to recent advancements in MORL methods, our algorithm achieves more consistent and superior performances in terms of hypervolume, expected utility, and sparsity on both discrete and continuous control tasks, especially with numerous objectives (up to nine objectives in our experiments).

This article investigates the online operation of distributed data centers equipped with energy battery. We aim to minimize their long-term operational cost by optimally distributing workload among data centers and operating energy battery. However, future spatio-temporally variant uncertainties in both workload and electricity prices have been the main impediment for a performance-guaranteed online data center operation strategy. To address this issue, we develop a fully distributed online algorithm that decouples workload distribution and battery operation across the network and time by introducing well-designed virtual queues for workload and batteries into the framework of Lyapunov optimization. Theoretically, an analytical gap between the long-term operational cost achieved by our algorithm and the theoretical optimum is provided to corroborate the desirable operation strategy. Extensive simulations using the real-world workload and electricity price data demonstrate the cost-delay tradeoff that our algorithm strikes and validate the theoretical results that we obtained.

This paper investigates the online station assignment for (commercial) electric vehicles (EVs) that request battery swapping from a central operator, i.e., in the absence of future information a battery swapping service station has to be assigned instantly to each EV upon its request. Based on EVs’ locations, the availability of fully-charged batteries at service stations in the system, as well as traffic conditions, the assignment aims to minimize cost to EVs and congestion at service stations. Inspired by a polynomial-time offline solution via a bipartite matching approach, we develop an efficient and implementable online station assignment algorithm that provably achieves the tight (optimal) competitive ratio under mild conditions. Monte Carlo experiments on a real transportation network by Baidu Maps show that our algorithm performs reasonably well on realistic inputs, even with a certain amount of estimation error in parameters.

In cyber-physical systems, mobile actuators can enhance system's flexibility and scalability, but at the same time incurs complex couplings in the scheduling and controlling of the actuators. In this paper, we propose a novel event-driven method aiming at satisfying a required level of control accuracy and saving energy consumption of the actuators, while guaranteeing a bounded action delay. We formulate a joint-design problem of both actuator scheduling and output control. To solve this problem, we propose a two-step optimization method. In the first step, the problem of actuator scheduling and action time allocation is decomposed into two subproblems. They are solved iteratively by utilizing the solution of one in the other. The convergence of this iterative algorithm is proved. In the second step, an online method is proposed to estimate the error and adjust the outputs of the actuators accordingly. Through simulations and experiments, we demonstrate the effectiveness of the proposed method.

This paper considers a city with a large fleet of plug-in electric taxis (PETs) and studies the charging coordination problem of the fleet. The goal is to reduce charging cost for each PET, defined as the loss of service income caused by charging, by wisely choosing when and where to charge. Considering the fact that the fleet can contain thousands of autonomous PETs, this problem is approached in a distributed way. In detail, a two-stage decision process is designed for each PET in an online fashion upon receiving real-time information. In the first stage, a thresholding method is proposed to assist a PET driver in choosing a proper time slot for charging, with comprehensive consideration of state of charge of PET, time varying income, and queuing status at charging stations (CSs). In the second stage, a game-theoretical approach is devised for PETs to select CSs, so that the traveling and queuing time of each PET can be reduced with fairness. Extensive numerical simulations illustrate the following threefold benefits of the proposed approach: it can effectively reduce the charging cost for PETs, enhance the utilization ratio for CSs, and also flatten the unevenness of charging request for power grid.

With the increasing penetration of electric vehicles (EVs) and various user preferences, charging stations often provide several different charging modes to satisfy the various requirements of EVs. How to effectively utilize the charging capacity to minimize the service dropping rate is a pressing and open issue for charging stations. Given that EV owners are price-sensitive to the charging modes, we intend to design an optimal pricing scheme to minimize the service dropping rate of the charging station. First, we formulate the operation of a dual-mode charging station as a queuing network with multiple servers and heterogeneous service rates, and analyze the relationship between the service dropping rate of the charging station and the selections of EVs. Then, we formulate a customer attrition minimization problem to minimize the number of EVs that leave the charging station without being charged and propose an optimal pricing approach to guide and coordinate the charging processes of EVs in the charging station. The simulation has been conducted to evaluate the performance of the proposed charging scheduling scheme and show the efficiency of the proposed pricing scheme.

Wireless charging is a promising technology for provisioning dynamic power supply in wireless rechargeable sensor networks (WRSNs). The charging equipment can be carried by some mobile nodes to enhance the charging flexibility. With such mobile chargers (MCs), the charging process should simultaneously address the MC scheduling, the moving and charging time allocation, while saving the total energy consumption of MCs. However, the efficient solutions that jointly solve those challenges are generally lacking in the literature. First, we investigate the multi-MC coordination problem that minimizing the energy expenditure of MCs while guaranteeing the perpetual operation of WRSNs, and formulate this problem as a mixed-integer linear program (MILP). Second, to solve this problem efficiently, we propose a novel decentralized method which is based on Benders decomposition. The multi-MC coordination problem is then decomposed into a master problem (MP) and a slave problem (SP), with the MP for MC scheduling and the SP for MC moving and charging time allocation. The MP is being solved by the base station (BS), while the SP is further decomposed into several sub-SPs and being solved by the MCs in parallel. The BS and MCs coordinate themselves to decide an optimal charging strategy. The convergence of proposed method is analyzed theoretically. Simulation results demonstrate the effectiveness and scalability of the proposed method.