Large Language Models (LLMs) are typically evaluated through general or domain-specific benchmarks testing capabilities that often lack grounding in the lived realities of end users. Critical domains such as healthcare require evaluations that extend beyond artificial or simulated tasks to reflect the everyday needs, cultural practices, and nuanced contexts of communities. We propose Samiksha, a community-driven evaluation pipeline co-created with civil-society organizations (CSOs) and community members. Our approach enables scalable, automated benchmarking through a culturally aware, community-driven pipeline in which community feedback informs what to evaluate, how the benchmark is built, and how outputs are scored. We demonstrate this approach in the health domain in India. Our analysis highlights how current multilingual LLMs address nuanced community health queries, while also offering a scalable pathway for contextually grounded and inclusive LLM evaluation.

The reference to assumptions in how practitioners use or interact with machine learning (ML) systems is ubiquitous in HCI and responsible ML discourse.However, what remains unclear from prior works is the conceptualization of assumptions and how practitioners identify and handle assumptions throughout their workflows.This leads to confusion about what assumptions are and what needs to be done with them.We use the concept of an argument from Informal Logic, a branch of Philosophy, to offer a new perspective to understand and explicate the confusions surrounding assumptions.Through semi-structured interviews with 22 ML practitioners, we find what contributes most to these confusions is how independently assumptions are constructed, how reactively and reflectively they are handled, and how nebulously they are recorded.Our study brings the peripheral discussion of assumptions in ML to the center and presents recommendations for practitioners to better think about and work with assumptions.

Celebrity influencers are increasingly central to political discourse as they engage in, and get engaged with, on matters of electoral importance. In this paper, using Twitter data from 1432 sportspersons and entertainers and their engagement with the 1000 of the most followed ruling party and opposition politicians from India, we propose a new method to measure partisanship of celebrities along different modes of engagement. Our examination of polarization, through topical and retweet analyses, shows patterns related to both party incumbency and the level of internal organization. We find that the ruling BJP has been more effective than the opposition, the INC, in organized outreach to celebrities, by eschewing explicit party-based partisanship, and instead employing non-partisan narrative techniques, such as maintaining nationalism as the central theme in tweets. We find that while entertainers are equally engaged by both the ruling and opposition parties, sportspersons, who often enjoy a nationalist appeal by virtue of representing the country, tend to have a much more partisan relationship with the incumbent party.

Significant research in HCI and beyond has sought to understand end-user needs in formal and informal technology-mediated mental health support (TMMHS) systems. However, little work has been done to understand the experiences and needs of the individuals who power or support these systems, particularly in the Global South. We present a qualitative study of one of the most accessible forms of mental health care in India — helplines. Through in-depth interviews conducted with 12 helpline volunteers, we research the human infrastructure responsible for the functioning of helplines. We foreground the often invisible labor involved in erecting and maintaining the institutional, interpersonal, and individual boundaries that are critical to realizing the goals of these helplines. Finally, we discuss the implications of our research for future work examining human infrastructures, particularly in mental health settings, and for the design of future TMMHS systems that deliver on-demand care to diverse, underserved, and stigmatized populations.

In this paper, we present the design of a low-cost IoT based approach to monitor the amount of water dispensed at communal clean water collection nodes called Water Filtration Plants. The design of our system caters to the limitations of low-resource settings, such as brown-outs, power surges, data connectivity issues, while our data processing methodology caters to the limitations inherent in the use of low-cost hardware installed in our deployment. Our actionable insights help the water utility of a dense Urban city in Pakistan improve the quality of service of providing clean drinking water to its residents.