This paper is focused on safe mapless navigation of mobile robots in unknown and possibly complex environments containing both internal and dynamic obstacles. We present a novel modular approach that combines the strengths of artificial potential functions (APF) with deep reinforcement learning. Differing from related work, the robot learns how to adjust the two input parameters of the APF controller as necessary through soft actor-critic algorithm. Environmental complexity measures are introduced in order to ensure that the robot's training covers a range of learning scenarios that vary in regard to maneuvering difficulty. Our experimental results show that differing from the classical navigation methods and end-to-end models, the robot can navigate successfully on its own even in complex scenarios with moving entities without requiring any maps.