Autonomous driving vehicles can reduce congestion and improve safety while increasing traffic efficiency. To reflect the quality of driving more comprehensively, the driving safety, efficiency, and occupant comfort should be jointly optimized for autonomous vehicles. Furthermore, in order to cope with complicated traffic environments and achieve satisfactory driving performance, a powerful behavior decision-making module is indispensable for autonomous vehicles. Toward this end, we study a reinforcement-learning (RL)-based method to intelligently make the behavior decision in this article. A Markov decision process (MDP) model is first formulated with a comprehensive reward function, including the effects of driving safety, efficiency, and comfort. The knowledge of the surrounding vehicles is also leveraged to exploit the behavior prediction of the target vehicle. We then propose a behavior decision strategy based on the actor–critic (AC) mechanism, which can efficiently learn both a Gaussian policy function and a linear value function. Finally, the real traffic data are used to build up the simulations for evaluating the performances of the proposed method thoroughly. Simulation results show that our proposed method can significantly reduce the collision rate for autonomous vehicles.