Self-recovery is a critical capability for robust, agile robots operating in the real world. Given truly challenging terrain, it is nearly inevitable that, at some point, the robot will fail and subsequently need to recover if it is to continue its task. One critical subset of recovery is standing back up after falling down (aka "self-righting"), an essential early milestone for babies learning to walk, and an existential capability for animals. While some robots can be designed with multiple orientations for mobility, most seeking to affect the world would significantly benefit from planners/policies that facilitate self-righting whenever possible.In this work, we present a series of challenges that outline why recovery in the wild is difficult. We then present a set of benchmark policies trained in simulation using deep reinforcement learning (RL) and the Student-Teacher approach. Finally, we evaluate the performance of these policies on a set of benchmark contexts in simulation, and provide baseline validation on a physical robot.