Technology offers new opportunities to support healthier food choices, particularly for individuals in low-income communities who face systemic barriers to obtaining nutritious, affordable groceries. We introduce a novel conceptual model of grocery planning that frames food purchasing as a multi-objective optimization problem that considers cost, nutrition components, and a consumer’s personal dietary goals. Guided by Zimmerman’s model of Self-Regulated Learning and prior research on food agency, we designed the Food Information System, a planning tool that provides optimized product recommendations aligned with users’ goals by integrating store inventory, prices, and nutritional data. We evaluated our system in an eight-week within-subjects intervention with 55 participants from a food-insecure community, followed by focus group sessions. While overall Healthy Eating Index scores remained largely stable, participants reported improved nutritional awareness and greater perceived agency in planning and purchasing groceries. We discuss design implications to support food agency by promoting long-term food literacy and by enhancing autonomy in making food choices.

Deep learning-based food recognition has made significant progress in predicting food types from eating occasion images. However, two key challenges hinder real-world deployment: (1) continuously learning new food classes without forgetting previously learned ones, and (2) handling the long-tailed distribution of food images, where a few common classes and many more rare classes. To address these, food recognition methods should focus on long-tailed continual learning. In this work, We introduce a dataset that encompasses 186 American foods along with comprehensive annotations. We also introduce three new benchmark datasets, VFN186-LT, VFN186-INSULIN and VFN186-T2D, which reflect real-world food consumption for healthy populations, insulin takers and individuals with type 2 diabetes without taking insulin. We propose a novel end-to-end framework that improves the generalization ability for instance-rare food classes using a knowledge distillation-based predictor to avoid misalignment of representation during continual learning. Additionally, we introduce an augmentation technique by integrating class-activation-map (CAM) and CutMix to improve generalization on instance-rare food classes. Our method, evaluated on Food101-LT, VFN-LT, VFN186-LT, VFN186-INSULIN, and VFN186-T2DM, shows significant improvements over existing methods. An ablation study highlights further performance enhancements, demonstrating its potential for real-world food recognition applications.

Large Language Models (LLMs) have the potential to contribute to the fields of nutrition and dietetics in generating food product explanations that facilitate informed food selections. However, the extent to which these models offer effective and accurate information remains unverified. In collaboration with registered dietitians (RDs), we evaluate the strengths and weaknesses of LLMs in providing accurate and personalized nutrition information. Through a mixed-methods approach, RDs validated GPT-4 outputs at various levels of prompt specificity, which led to the development of design guidelines used to prompt LLMs for nutrition information. We tested these guidelines by creating a GPT prototype, The Food Product Nutrition Assistant, tailored for food product explanations. This prototype was refined and evaluated in focus groups with RDs. We find that the implementation of these dietitian-reviewed template instructions enhance the generation of detailed food product descriptions and tailored nutrition information.

To identify technological opportunities to better support nutrition security and equality among those living in low-socioeconomic situations, we conducted 33 semi-structured interviews and seven in-home visits of lower- to middle-income households from a mid-sized city in northern Indiana. Inspired by assets-based approaches to public health, we investigated technology’s role in supporting how participants selected and purchased food, planned meals, and worked through logistical barriers to obtain food. Technology helped participants identify sales and coupons, search for recipes and health-related insights to address diet and health concerns, and share information. We contribute design implications (e.g., amplifying optimization behaviors and social engagement, leveraging substitutions) in support of food agency. We further contribute three emergent archetypes to convey central shopping tendencies (i.e., inventory shoppers, menu planners, and adaptive shoppers) and identify corresponding design implications. We situate our results into nutrition decision-making and education, social psychology, food consumer studies, and HCI literature.

Image-based dietary assessment refers to the process of determining what someone eats and how much energy and nutrients are consumed from visual data. Food classification is the first and most crucial step. Existing methods focus on improving accuracy measured by the rate of correct classification based on visual information alone, which is very challenging due to the high complexity and inter-class similarity of foods. Further, accuracy in food classification is conceptual as description of a food can always be improved. In this work, we introduce a new food classification framework to improve the quality of predictions by integrating the information from multiple domains while maintaining the classification accuracy. We apply a multi-task network based on a hierarchical structure that uses both visual and nutrition domain specific information to cluster similar foods. Our method is validated on the modified VIPER-FoodNet (VFN) food image dataset by including associated energy and nutrient information. We achieve comparable classification accuracy with existing methods that use visual information only, but with less error in terms of energy and nutrient values for the wrong predictions.

Chronic diseases, such as heart disease, diabetes, and obesity, have been linked with diet. Nutrient intake is also associated with diet. However, much of the research completed to elucidate these associations has not incorporated the concept of time. This paper introduces the concept of temporal dietary patterns and demonstrates a novel construct of 24-hour temporal dietary patterns for energy intake, present in a sample of the adult U.S. population 20 years and older (NHANES 1999-2004 dataset). An appropriate distance metric is proposed for comparing 24-hour diet records and is used with kernel k-means clustering to identify the temporal dietary patterns.