Multiplexing Enhanced Mobile Broadband (eMBB) and Ultra-Reliable Low Latency Communications (URLLC) traffic on the same 5G New Radio (NR) air interface poses significant challenges due to extreme latency requirement of URLLC packets. This paper investigates the direct puncturing of URLLC traffic over eMBB transmissions, a method that, while guaranteeing immediate URLLC packet delivery, can severely degrade eMBB performance. To alleviate the adverse impact on eMBB, we present Cyrus—a deep reinforcement learning (DRL)-based puncturing solution for eMBB and URLLC multiplexing. Cyrus is tailored for the Open RAN (O-RAN) architecture and unifies the three control loops of O-RAN synergistically in its design of DRL-based solution. Not only does Cyrus meet the real-time requirements for URLLC but also it continuously updates and improves its scheduling policy based on changing network conditions. The effectiveness of Cyrus is demonstrated through link-level simulations for 5G NR, showing significant improvement in eMBB performance over the state-of-the-art, particularly as URLLC traffic increases.