The broad applicability of Internet of Things (IoT) would truly enable the pervasiveness of smart devices for sensing data. In this context, achieving service personalization requires collecting sensitive data about users. That yields to privacy concerns due to the possibility of abusing the data through unauthorized access. Moreover, IoT devices have limited computing resources, making them difficult to perform heavy protection mechanisms. Despite several existing solutions for privacy protection, they were not designed to run on limited resources in large scale environment. In addition, existing access control solutions, including XACML, are heavy to run on resource constraint devices and lack behavior-based customization of user privacy where users have little to no control over their private data. In this regard, we address the aforementioned problems by proposing LP-SBA-XACML, which embeds an efficient and lightweight semantics-based scheme targeting user privacy and providing efficient policy evaluation. LP-SBA-XACML is a scalable and lightweight solution suitable for the IoT context while preserving the assumptions of XACML. Moreover, an intelligent model for real-time behavior/activity prediction is integrated to systematically customize user’s privacy and services. Experiments conducted on synthetic and real-life scenarios demonstrate the feasibility and relevance of our proposed framework within a mobile IoT resource-constrained environment.