Public health programs often provide interventions to encourage program adherence, and effectively allocating interventions is vital for producing the greatest overall health outcomes, especially in underserved communities where resources are limited. Such resource allocation problems are often modeled as restless multi-armed bandits (RMABs) with unknown underlying transition dynamics, hence requiring online reinforcement learning (RL). We present Bayesian Learning for Contextual RMABs (BCoR), an online RL approach for RMABs that novelly combines techniques in Bayesian modeling with Thompson sampling to flexibly model the complex RMAB settings present in public health program adherence problems, namely context and non-stationarity. BCoR's key strength is the ability to leverage shared information within and between arms to learn the unknown RMAB transition dynamics quickly in intervention-scarce settings with relatively short time horizons, which is common in public health applications. Empirically, BCoR achieves substantially higher finite-sample performance over a range of experimental settings, including a setting using real-world adherence data that was developed in collaboration with ARMMAN, an NGO in India which runs a large-scale maternal mHealth program, showcasing BCoR practical utility and potential for real-world deployment.

Mosquitoes are important vectors for pathogens that infect humans and other vertebrate animals. Some aspects of adult mosquito behavior and mosquito ecology play an important role in determining the capacity of vector populations to transmit pathogens. Here, we re-examine factors affecting the transmission of pathogens by mosquitoes using a new approach. Unlike most previous models, this framework considers the behavioral states and state transitions of adult mosquitoes through a sequence of activity bouts. We developed a new framework for individual-based simulation models called MBITES (Mosquito Bout-based and Individual-based Transmission Ecology Simulator). In MBITES, it is possible to build models that simulate the behavior and ecology of adult mosquitoes in exquisite detail on complex resource landscapes generated by spatial point processes. We also developed an ordinary differential equation model which is the Kolmogorov forward equations for models developed in MBITES under a specific set of simplifying assumptions. While mosquito infection and pathogen development are one possible part of a mosquito's state, that is not our main focus. Using extensive simulation using some models developed in MBITES, we show that vectorial capacity can be understood as an emergent property of simple behavioral algorithms interacting with complex resource landscapes, and that relative density or sparsity of resources and the need to search can have profound consequences for mosquito populations' capacity to transmit pathogens.