Authorship models have historically generalized poorly to new domains because of the wide distribution of author-identifying signals across domains. In particular, the effects of topic and genre are highly domain-dependent and impact authorship analysis performance greatly. This paper addresses the existing data gap in authorship for these resources by introducing CROSSNEWS, a novel cross-genre dataset that connects formal journalistic articles and casual social media posts. CROSSNEWS is the largest authorship dataset of its kind for supporting both verification and attribution tasks, with comprehensive topic and genre annotations. We use CROSSNEWS to demonstrate that current models exhibit poor performance in genre transfer scenarios, underscoring the need for authorship models robust to genre-specific effects. We also explore SELMA, a new LLM embedding approach for large-scale authorship setups that outperforms existing models in both same-genre and cross-genre settings.

Large Language Models are often trained on data reflecting neurotypical norms, yet are increasingly deployed to support neurodivergent users in sensitive domains like job-seeking.We examine interactions between neurodivergent job-seekers and a GPT-4powered career support chatbot through the lens of misalignment.Through analysis of over 300 chat logs and interviews with 15 neurodivergent participants, we found that the chatbot frequently misrepresented users' skills, imposed neurotypical language and expectations, and provided generic or inappropriate advice-even when relevant user data was available.Participants expected the chatbot to interpret implicit insights from their data, however, they sometimes lacked the clarity or confidence to correct the system when it did not, revealing gaps in both AI design and user understanding of system function.Our findings underscore the need for bi-directional alignment between neurodivergent users and LLMs, and call for design strategies that integrate neurodivergent perspectives and preferences to ensure more authentic, personalized, and human-centered AI support.

Modeling complex subjective tasks in Natural Language Processing, such as recognizing emotion and morality, is considerably challenging due to significant variation in human annotations.This variation often reflects reasonable differences in semantic interpretations rather than mere noise, necessitating methods to distinguish between legitimate subjectivity and error.We address this challenge by exploring label verification in these contexts using Large Language Models (LLMs).First, we propose a simple In-Context Learning binary filtering baseline that estimates the reasonableness of a document-label pair.We then introduce the Label-in-a-Haystack setting: the query and its label(s) are included in the demonstrations shown to LLMs, which are prompted to predict the label(s) again, while receiving task-specific instructions (e.g., emotion recognition) rather than label copying.We show how the failure to copy the label(s) to the output of the LLM are task-relevant and informative.Building on this, we propose the Label-in-a-Haystack Rectification (LiaHR) framework for subjective label correction: when the model outputs diverge from the reference gold labels, we assign the generated labels to the example instead of discarding it.This approach can be integrated into annotation pipelines to enhance signal-to-noise ratios.Comprehensive analyses, human evaluations, and ecological validity studies verify the utility of LiaHR for label correction.

Multi-label classification is prevalent in realworld settings, but the behavior of Large Language Models (LLMs) in this setting is understudied.We investigate how autoregressive LLMs perform multi-label classification, focusing on subjective tasks, by analyzing the output distributions of the models at each label generation step.We find that the initial probability distribution for the first label often does not reflect the eventual final output, even in terms of relative order and find LLMs tend to suppress all but one label at each generation step.We further observe that as model scale increases, their token distributions exhibit lower entropy and higher single-label confidence, but the internal relative ranking of the labels improves.Finetuning methods such as supervised finetuning and reinforcement learning amplify this phenomenon.We introduce the task of distribution alignment for multi-label settings: aligning LLM-derived label distributions with empirical distributions estimated from annotator responses in subjective tasks.We propose both zero-shot and supervised methods which improve both alignment and predictive performance over existing approaches.We find one method -taking the max probability over all label generation distributions instead of just using the initial probability distribution -improves both distribution alignment and overall F1 classification without adding any additional computation.

While birth control pills are one of the most common forms of contraception, their usage has several emotional and physical costs, such as taking the pill daily and experiencing hormonal side effects. The burden of these tasks in relationships generally falls on the pill user with minimal involvement from their partner. In this study, we conducted semi-structured interviews with pill users and their partners to investigate the differences between their perceived current and ideal divisions of birth control responsibility. During the interview, we presented a collaborative birth control tracking app prototype to examine how such technology can overcome these discrepancies. We found that pill users were unsatisfied with their partners' engagement in contraceptive tasks but did not communicate this well. Meanwhile, partners wanted to contribute more to pregnancy prevention but did not know how. When presented with our app prototype, users and partners stated that our design could address these issues by improving communication between users and partners. In particular, users appreciated how technology could increase engagement and support from their partner, and partners liked that our app presented several concrete ways to become more involved and show emotional support. However, privacy issues exist given the sensitive nature of contraception. We highlight design considerations that should be kept in mind about privacy while recognizing pill users' efforts and promoting partners' involvement.