Thanks to the adoption of more sensors in the automotive industry, context-aware Advanced Driver Assistance Systems (ADAS) become possible. On one side, a common thread in ADAS applications is to focus entirely on the context of the vehicle and its surrounding vehicles leaving the human (driver) context out of consideration. On the other side, and due to the increasing sensing capabilities in mobile phones and wearable technologies, monitoring complex human context becomes feasible which paves the way to develop driver-in-the-loop context-aware ADAS that provide personalized driving experience. In this paper, we propose Sentio1; a Reinforcement Learning based algorithm to enhance the Forward Collision Warning (FCW) system leading to Driver-in-the-Loop FCW system. Since the human driving preference is unknown a priori, varies between different drivers, and moreover, varies across time for the same driver, the proposed Sentio algorithm needs to take into account all these variabilities which are not handled by the standard reinforcement learning algorithms. We verified the proposed algorithm against several human drivers. Our evaluation, across distracted human drivers, shows a significant enhancement in driver experience---compared to standard FCW systems---reflected by an increase in the driver safety by 94.28%, an improvement in the driving experience by 20.97%, a decrease in the false negatives from 55.90% down to 3.26%, while adding less than 130 ms runtime execution overhead.