Percutaneous coronary intervention (PCI) involves the insertion of a catheter or guidewire into a blood vessel of a patient, which poses a problem as a doctor is exposed to radiation during the procedure. The use of assistive robots has been proposed to address this issue. Furthermore, recent research is progressing toward complete autonomous navigation using deep reinforcement learning (DRL). Nevertheless, existing algorithms face limitations when operating in numerous unseen environments close to real PCI. This study proposes a robust DRL framework for image-based guidewire navigation to overcome the limitation. We introduce a subtasks strategy and domain randomization to improve robustness in various environments. The subtasks strategy consistently addresses complex global tasks by breaking them into subtasks designed using local maps, allowing them to be robustly solved by a single agent. Domain randomization is applied to handle real PCI issues, including variations in vessel geometry, guidewire deformation, and camera settings. By integrating the two novel methods, our DRL algorithm demonstrates superior performance compared to existing methods across various challenging simulation and phantom environments, validating its effectiveness in real-world scenarios. A video of our experiment is available at https://youtu.be/93Q88gESzOY.