Password-based authentication system has been praised for its user-friendly, cost-effective, and easily deployable features. It is arguably the most commonly used security mechanism for various resources, services, and applications. On the other hand, it has well-known security flaws, including vulnerability to guessing attacks. Present state-of-the-art approaches have high overheads, as well as difficulties and unreliability during training, resulting in a poor user experience and a high false positive rate. As a result, a lightweight authentication compromise detection model that can make accurate detection with a low false positive rate is required.In this paper we propose – LOG-OFF – a behavior-based authentication compromise detection model. LOG-OFF is a lightweight model that can be deployed efficiently in practice because it does not include a labeled dataset. Based on the assumption that the behavioral pattern of a specific user does not suddenly change, we study the real-world authentication traffic data. The dataset contains more than 4 million records. We use two features to model the user behaviors, i.e., consecutive failures and login time, and develop a novel approach. LOG-OFF learns from the historical user behaviors to construct user profiles and makes probabilistic predictions of future login attempts for authentication compromise detection. LOG-OFF has a low false positive rate and latency, making it suitable for real-world deployment. In addition, it can also evolve with time and make more accurate detection as more data is being collected.