Predicting users’ privacy concerns is challenging due to privacy’s subjective and complex nature. Previous research demonstrated that generic attitudes, such as those captured by Westin’s Privacy Segmentation Index, are inadequate predictors of context-specific attitudes. We introduce ContextLabel, a method enabling practitioners to capture users’ privacy profiles across domains and predict their privacy concerns towards unseen data practices. ContextLabel’s key innovations are (1) using non-mutually exclusive labels to capture more nuances of data practices, and (2) capturing users’ privacy profiles by asking them to express privacy concerns to a few data practices. To explore the feasibility of ContextLabel, we asked 38 participants to express their thoughts in free text towards 13 distinct data practices across five days. Our mixed-methods analysis shows that a preliminary version of ContextLabel can predict users’ privacy concerns towards unseen data practices with an accuracy (73%) surpassing Privacy Segmentation Index (56%) and methods using categorical factors (59%).