We propose and analyze a new policy gradient algorithm for reinforcement learning (RL), relative Monte Carlo (rMC). The method estimates policy gradients using relative returns between a root sample path and counterfactual simulated paths, instantiated by taking a different action from the root. The resulting gradient estimate is both unbiased and has low variance. rMC is compatible with any differentiable policy, including neural networks, and is guaranteed to converge even for infinite horizon tasks. The method utilizes common random number coupling of the simulated paths to reduce variance and increase the likelihood that paths merge, thereby reducing simulation complexity. It is particularly well suited to discrete event control problems where actions have a "local" effect, such as queueing, supply chain, or ride-hailing problems. Indeed, we show that it has provably low complexity for a family of inventory control problems. Numerical tests on a challenging inventory and fulfillment problem show that compared to traditional RL approaches, rMC converges in far fewer iterations (lower variance), has better policy performance (unbiased), and requires minimal hyperparameter tuning.

We build a model to study the implications of utilization-based minimum earning regulations of the kind recently enacted by New York City for its ride-hailing providers. We identify the precise conditions under which a utilization-based minimum earnings rule causes marketplace instability, where stability is defined as the ability of platforms to keep wages bounded while maintaining the current flexible (free-entry) work model. We also calibrate our model using publicly available data, showing the limited power of the law to increase earnings within an open marketplace. We argue that affected ride-hailing companies might respond to the law by reducing driver flexibility.

In this paper, the presence of delay in a job shop is addressed. We show that delay is an important consideration in many manufacturing systems that are modeled as continuous flow processes. A scheduling policy for a job shop with delays is then derived using theoretical arguments and heuristics.