Safety in Human-Robot Collaboration (HRC) is a bottleneck to HRC-productivity in industry. With robots being the main source of hazards, safety engineers use over-emphasized safety measures, and carry out lengthy and expensive risk assessment processes on each HRC-layout reconfiguration. Recent advances in deep Reinforcement Learning (RL) offer solutions to add intelligence and comprehensibility of the environment to robots. In this paper, we propose a framework that uses deep RL as an enabling technology to enhance intelligence and safety of the robots in HRC scenarios and, thus, reduce hazards incurred by the robots. The framework offers a systematic methodology to encode the task and safety requirements and context of applicability into RL settings. The framework also considers core components, such as behavior explainer and verifier, which aim for transferring learned behaviors from research labs to industry. In the evaluations, the proposed framework shows the capability of deep RL agents learning collision-free point-to-point motion on different robots inside simulation, as shown in the supplementary video.