MOTIVATION: The goal of oligonucleotide (oligo) design is to select oligos that optimize a set of design criteria. Oligo design problems are combinatorial in nature and require computationally intensive models to evaluate design criteria. Even relatively small problems can be intractable for brute-force approaches that test every possible combination of oligos, so heuristic approaches must be used to find near-optimal solutions. RESULTS: We present a general reinforcement learning (RL) framework, called OligoRL, to solve oligo design problems with complex constraints. OligoRL allows 'black-box' design criteria and can be adapted to solve many oligo design problems. We highlight the flexibility of OligoRL by building tools to solve three distinct design problems: (i) finding pools of random DNA barcodes that lack restriction enzyme recognition sequences (CutFreeRL); (ii) compressing large, non-degenerate oligo pools into smaller degenerate ones (OligoCompressor) and (iii) finding Not-So-Random hexamer primer pools that avoid rRNA and other unwanted transcripts during RNA-seq library preparation (NSR-RL). OligoRL demonstrates how RL offers a general solution for complex oligo design problems. AVAILABILITY AND IMPLEMENTATION: OligoRL and all simulation codes are available as a Julia package at http://jensenlab.net/tools and archived at https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://github.com/bmdavid2/OligoRL. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.