Wildfires have increased in extent and severity, and are posing a growing threat to people’s well-being and the environment. Prescribed burns (burning on purpose parts of the landscape) are one of the key mitigation strategies available to reduce the potential damage of wildfires. However, where to conduct prescribed burns has long been a problem for domain experts. With the advancement of forest science, weather science, and computational modeling, there produced powerful fire simulators that can help inform how wildfires will start and grow. In this paper, we model the problem of selecting where to perform a set of prescribed burns across a large landscape into a multi-objective optimization problem. We build a surrogate objective function from simulation data and solve the multi-objective optimization problem with genetic algorithms. We name our solution as Spatial Multi-Objective for Prescribed Burn (SMO-PB). We also investigate three variants of the approach that further consider spatial fairness. With a case study of Dogrib, Canada, we show that our formulations can successfully provide solutions capable of real world deployment, and showed how fairness can be reached without diminishing the performance a lot.

Wildlife poaching is threatening key species that play important roles in the ecosystem. With historical ranger patrol records, it is possible to provide data-driven predictions of poaching threats and plan patrols to combat poaching. However, the patrollers often patrol in a multimodal way, which combines driving and walking. It is a tedious task for the domain experts to manually plan such a patrol and as a result, the planned patrol routes are often far from optimal. In this paper, we propose a data-driven approach for multimodal patrol planning. We first use machine learning models to predict the poaching threats and then use a novel mixed-integer linear programming-based algorithm to plan the patrol route. In a field test focusing on the machine learning prediction result at Jilin Huangnihe National Nature Reserve (HNHR) in December 2019, the rangers found 42 snares, which is significantly higher than the historical record. Our offline experiments show that the resulting multimodal patrol routes can improve the efficiency of patrol and thus they can serve as the basis for future deployment in the field.

One practical requirement in solving dynamic games is to ensure that the players play well from any decision point onward. To satisfy this requirement, existing efforts focus on equilibrium refinement, but the scalability and applicability of existing techniques are limited. In this paper, we propose Temporal-Induced Self-Play (TISP), a novel reinforcement learning-based framework to find strategies with decent performances from any decision point onward. TISP uses belief-space representation, backward induction, policy learning, and non-parametric approximation. Building upon TISP, we design a policy-gradient-based algorithm TISP-PG. We prove that TISP-based algorithms can find approximate Perfect Bayesian Equilibrium in zero-sum one-sided stochastic Bayesian games with finite horizon. We test TISP-based algorithms in various games, including finitely repeated security games and a grid-world game. The results show that TISP-PG is more scalable than existing mathematical programming-based methods and significantly outperforms other learning-based methods.

Coordination is one of the essential problems in multi-agent systems. Typically multi-agent reinforcement learning (MARL) methods treat agents equally and the goal is to solve the Markov game to an arbitrary Nash equilibrium (NE) when multiple equilibra exist, thus lacking a solution for NE selection. In this paper, we treat agents unequally and consider Stackelberg equilibrium as a potentially better convergence point than Nash equilibrium in terms of Pareto superiority, especially in cooperative environments. Under Markov games, we formally define the bi-level reinforcement learning problem in finding Stackelberg equilibrium. We propose a novel bi-level actor-critic learning method that allows agents to have different knowledge base (thus intelligent), while their actions still can be executed simultaneously and distributedly. The convergence proof is given, while the resulting learning algorithm is tested against the state of the arts. We found that the proposed bi-level actor-critic algorithm successfully converged to the Stackelberg equilibria in matrix games and find a asymmetric solution in a highway merge environment.

Although security games have attracted intensive research attention over the past years, few existing works consider how information from local communities would affect the game. In this paper, we introduce a new player -- a strategic informant, who can observe and report upcoming attacks -- to the defender-attacker security game setting. Characterized by a private type, the informant has his utility structure that leads to his strategic behaviors. We model the game as a 3-player extensive-form game and propose a novel solution concept of Strong Stackelberg-perfect Bayesian equilibrium. To compute the optimal defender strategy, we first show that although the informant can have infinitely many types in general, the optimal defense plan can only include a finite (exponential) number of different patrol strategies. We then prove that there exists a defense plan with only a linear number of patrol strategies that achieve the optimal defender's utility, which significantly reduces the computational burden and allows us to solve the game in polynomial time using linear programming. Finally, we conduct extensive experiments to show the effect of the strategic informant and demonstrate the effectiveness of our algorithm.