Large Language Models (LLMs) are increasingly utilized for domain-specific tasks, yet evaluating their outputs remains challenging. A common strategy is to apply evaluation criteria to assess alignment with domain-specific standards, yet little is understood about how criteria differ across sources or where each type is most useful in the evaluation process. This study investigates criteria developed by domain experts, lay users, and LLMs to identify their complementary roles within an evaluation workflow. Results show that experts produce fact-based criteria with long-term value, lay users emphasize usability with a shorter-term focus, and LLMs target procedural checks for immediate task requirements. We also examine how criteria evolve between a priori and a posteriori phases, noting drift across stages as well as convergence in the a posteriori phase. Based on our observations, we propose design guidelines for a staged evaluation workflow combining the complementary strengths of these sources to balance quality, cost, and scalability.

Technology offers new opportunities to support healthier food choices, particularly for individuals in low-income communities who face systemic barriers to obtaining nutritious, affordable groceries. We introduce a novel conceptual model of grocery planning that frames food purchasing as a multi-objective optimization problem that considers cost, nutrition components, and a consumer’s personal dietary goals. Guided by Zimmerman’s model of Self-Regulated Learning and prior research on food agency, we designed the Food Information System, a planning tool that provides optimized product recommendations aligned with users’ goals by integrating store inventory, prices, and nutritional data. We evaluated our system in an eight-week within-subjects intervention with 55 participants from a food-insecure community, followed by focus group sessions. While overall Healthy Eating Index scores remained largely stable, participants reported improved nutritional awareness and greater perceived agency in planning and purchasing groceries. We discuss design implications to support food agency by promoting long-term food literacy and by enhancing autonomy in making food choices.

Large Language Models (LLMs) are increasingly developed for use in complex professional domains, yet little is known about how teams design and evaluate these systems in practice. This paper examines the challenges and trade-offs in LLM development through a 12-week ethnographic study of a team building a pedagogical chatbot. The researcher observed design and evaluation activities and conducted interviews with both developers and domain experts. Analysis revealed four key practices: creating workarounds for data collection, turning to augmentation when expert input was limited, co-developing evaluation criteria with experts, and adopting hybrid expert–developer–LLM evaluation strategies. These practices show how teams made strategic decisions under constraints and demonstrate the central role of domain expertise in shaping the system. Challenges included expert motivation and trust, difficulties structuring participatory design, and questions around ownership and integration of expert knowledge. We propose design opportunities for future LLM development workflows that emphasize AI literacy, transparent consent, and frameworks recognizing evolving expert roles.

Large Language Models (LLMs) have the potential to contribute to the fields of nutrition and dietetics in generating food product explanations that facilitate informed food selections. However, the extent to which these models offer effective and accurate information remains unverified. In collaboration with registered dietitians (RDs), we evaluate the strengths and weaknesses of LLMs in providing accurate and personalized nutrition information. Through a mixed-methods approach, RDs validated GPT-4 outputs at various levels of prompt specificity, which led to the development of design guidelines used to prompt LLMs for nutrition information. We tested these guidelines by creating a GPT prototype, The Food Product Nutrition Assistant, tailored for food product explanations. This prototype was refined and evaluated in focus groups with RDs. We find that the implementation of these dietitian-reviewed template instructions enhance the generation of detailed food product descriptions and tailored nutrition information.

Understanding nutrition labels remains challenging for consumers; however, digital shopping environments offer opportunities to explore how interactive nutrition labels may be used to enhance comprehension. We conducted an A/B study with 24 participants, comparing their ability to interpret and apply nutrition information using conventional, static labels versus interactive labels. We evaluated interactive nutrition labels’ impact through quantitative metrics and qualitative insights from interviews and think-aloud sessions. Our findings reveal a statistically significant improvement in assessing nutrient amounts and interpreting numerical information when users engage with interactive labels. These results underscore the potential interactivity has on promoting public understanding of nutritional content and highlight opportunities for refinement. Based on our findings, we propose new design directions and discuss technology’s role in making nutrition labels more effective for decision-making and nutrition education.