Personalization of LLMs by sociodemographic subgroup often improves user experience, but can also introduce or amplify biases and unfair outcomes across groups.Prior work has employed so-called personas, sociodemographic user attributes conveyed to a model, to study bias in LLMs by relying on a single cue to prompt a persona, such as user names or explicit attribute mentions.This disregards LLM sensitivity to prompt variation and the rarity of some cues in real interactions (external validity).We compare six commonly used persona cues across seven open and proprietary LLMs on four writing and advice tasks.While cues are overall highly correlated, they produce substantial variance in responses across personas that can change findings on persona-induced differences and bias.We therefore caution against claims based on single persona cues, especially when they are overly explicit and have low external validity.

Generative Large Language Models (LLMs) infer user's demographic information from subtle cues in the conversation -a phenomenon called implicit personalization.Prior work has shown that such inferences can lead to lower quality responses for users assumed to be from minority groups, even when no demographic information is explicitly provided.In this work, we systematically explore how LLMs respond to stereotypical cues using controlled synthetic conversations, by analyzing the models' latent user representations through both model internals and generated answers to targeted user questions.Our findings reveal that LLMs do infer demographic attributes based on these stereotypical signals, which for a number of groups even persists when the user explicitly identifies with a different demographic group.Finally, we show that this form of stereotypedriven implicit personalization can be effectively mitigated by intervening on the model's internal representations using a trained linear probe to steer them toward the explicitly stated identity.Our results highlight the need for greater transparency and control in how LLMs represent user identity.