We consider reinforcement learning (RL) for a class of problems with bagged decision times. A bag contains a finite sequence of consecutive decision times. The transition dynamics are non-Markovian and non-stationary within a bag. All actions within a bag jointly impact a single reward, observed at the end of the bag. For example, in mobile health, multiple activity suggestions in a day collectively affect a user’s daily commitment to being active. Our goal is to develop an online RL algorithm to maximize the discounted sum of the bag-specific rewards. To handle non-Markovian transitions within a bag, we utilize an expert-provided causal directed acyclic graph (DAG). Based on the DAG, we construct states as a dynamical Bayesian sufficient statistic of the observed history, which results in Markov state transitions within and across bags. We then formulate this problem as a periodic Markov decision process (MDP) that allows non-stationarity within a period. An online RL algorithm based on Bellman equations for stationary MDPs is generalized to handle periodic MDPs. We show that our constructed state achieves the maximal optimal value function among all state constructions for a periodic MDP. Finally, we evaluate the proposed method on testbed variants built from real data in a mobile health clinical trial.

Parental emotion coaching approaches that advocate for noticing and validating child emotions can greatly impact children's regulatory abilities. However, in daily life, parents often struggle to apply emotion coaching strategies that they access through parenting programmes or online help, suggesting a need for in situ support. This paper explores a potential new avenue for providing such support. We undertook conceptual work to develop a set of emotion-focused reflective questions that could increase parents’ attention to child emotions and delivered these as daily ecological momentary assessments (EMAs). We investigated the perceived impact of the approach through a 2-week online trial (n=33) and then co-designed child-facing component with parents through a 4-week asynchronous remote community study (n=15). Our paper contributes (1) conceptual insights on designing a potential novel intervention approach, (2) empirical insights on its acceptability and perceived impacts for parents, and (3) design implications for applying the approach to wider psychological constructs.

Engaging with people's lived experiences is foundational for HCI research and design. This paper introduces a novel narrative elicitation method to empower people to easily articulate 'micro-narratives' emerging from their lived experiences, irrespective of their writing ability or background. Our approach aims to enable at-scale collection of rich, co-created datasets that highlight target populations' voices with minimal participant burden, while precisely addressing specific research questions. To pilot this idea, and test its feasibility, we: (i) developed an AI-powered prototype, which leverages LLM-chaining to scaffold the cognitive steps necessary for users' narrative articulation; (ii) deployed it in three mixed-methods studies involving over 380 users; and (iii) consulted with established academics as well as C-level staff at (inter)national non-profits to map out potential applications. Both qualitative and quantitative findings show the acceptability and promise of the micro-narrative method, while also identifying the ethical and safeguarding considerations necessary for any at-scale deployments.

This paper explores the integration of causal pathway diagrams (CPD) into human-centered design (HCD), investigating how these diagrams can enhance the early stages of the design process. A dedicated CPD plugin for the online collaborative whiteboard platform Miro was developed to streamline diagram creation and offer real-time AI-driven guidance. Through a user study with designers (N = 20), we found that CPD’s branching and its emphasis on causal connections supported both divergent and convergent processes during design. CPD can also facilitate communication among stakeholders. Additionally, we found our plugin significantly reduces designers’ cognitive workload and increases their creativity during brainstorming, highlighting the implications of AI-assisted tools in supporting creative work and evidence-based designs.

OBJECTIVE: Digital behavior change interventions (DBCIs) are feasibly effective tools for addressing physical activity. However, in-depth understanding of participants' long-term engagement with DBCIs remains sparse. Since the effectiveness of DBCIs to impact behavior change depends, in part, upon participant engagement, there is a need to better understand engagement as a dynamic process in response to an individual's ever-changing biological, psychological, social, and environmental context. METHODS: The year-long micro-randomized trial (MRT) HeartSteps II provides an unprecedented opportunity to investigate DBCI engagement among ethnically diverse participants. We combined data streams from wearable sensors (Fitbit Versa, i.e., walking behavior), the HeartSteps II app (i.e. page views), and ecological momentary assessments (EMAs, i.e. perceived intrinsic and extrinsic motivation) to build the idiographic models. A system identification approach and a fluid analogy model were used to conduct autoregressive with exogenous input (ARX) analyses that tested hypothesized relationships between these variables inspired by Self-Determination Theory (SDT) with DBCI engagement through time. RESULTS: Data from 11 HeartSteps II participants was used to test aspects of the hypothesized SDT dynamic model. The average age was 46.33 (SD=7.4) years, and the average steps per day at baseline was 5,507 steps (SD=6,239). The hypothesized 5-input SDT-inspired ARX model for app engagement resulted in a 31.75 % weighted RMSEA (31.50 % on validation and 31.91 % on estimation), indicating that the model predicted app page views almost 32 % better relative to the mean of the data. Among Hispanic/Latino participants, the average overall model fit across inventories of the SDT fluid analogy was 34.22 % (SD=10.53) compared to 22.39 % (SD=6.36) among non-Hispanic/Latino Whites, a difference of 11.83 %. Across individuals, the number of daily notification prompts received by the participant was positively associated with increased app page views. The weekend/weekday indicator and perceived daily busyness were also found to be key predictors of the number of daily application page views. CONCLUSIONS: This novel approach has significant implications for both personalized and adaptive DBCIs by identifying factors that foster or undermine engagement in an individual's respective context. Once identified, these factors can be tailored to promote engagement and support sustained behavior change over time.

Policy learning is an important component of many real-world learning systems. A major challenge in policy learning is how to adapt efficiently to unseen environments or tasks. Recently, it has been suggested to exploit invariant conditional distributions to learn models that generalize better to unseen environments. However, assuming invariance of entire conditional distributions (which we call full invariance) may be too strong of an assumption in practice. In this paper, we introduce a relaxation of full invariance called effect-invariance (e-invariance for short) and prove that it is sufficient, under suitable assumptions, for zero-shot policy generalization. We also discuss an extension that exploits e-invariance when we have a small sample from the test environment, enabling few-shot policy generalization. Our work does not assume an underlying causal graph or that the data are generated by a structural causal model; instead, we develop testing procedures to test e-invariance directly from data. We present empirical results using simulated data and a mobile health intervention dataset to demonstrate the effectiveness of our approach.

Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right type and amount of support by iteratively selecting a sequence of intervention options from a pre-defined set of components in response to each individual’s time varying state. In this work, we explore the application of reinforcement learning methods to the problem of learning intervention option selection policies. We study the effect of context inference error and partial observability on the ability to learn effective policies. Our results show that the propagation of uncertainty from context inferences is critical to improving intervention efficacy as context uncertainty increases, while policy gradient algorithms can provide remarkable robustness to partially observed behavioral state information.

Self-tracking can help people understand their medical condition and the factors that influence their symptoms. However, it is unclear how tracking technologies should be tailored to help people cope with the progression of a degenerative disease. To understand how smartphone apps and other tracking technologies can support people in coping with an incurable illness, we interviewed both people with Parkinson's Disease (n=17) and care partners (n=6) who help people with Parkinson's manage their lives. We describe how symptom trackers can help people identify and solve problems to improve their quality of life, the role symptom trackers can play in helping people combat their own tendencies towards avoidance and denial, and the complex role of care partners in defining and tracking ambiguous symptoms. Our findings yield insights that can guide the design of tracking technologies to help people with Parkinson's Disease accept and plan for their condition.

Over the last ten years, HCI researchers have introduced a range of novel ways to support health behavior change, from glanceable displays to sophisticated game dynamics. Yet, this research has not had as much impact as its originality warrants. A key reason for this is that common forms of evaluation used in HCI make it difficult to effectively accumulate-and use-knowledge across research projects. This paper proposes a strategy for HCI research on behavior change that retains the field's focus on novel technical contributions while enabling accumulation of evidence that can increase impact of individual research projects both in HCI and the broader behavior-change science. The core of this strategy is an emphasis on the discovery of causal effects of individual components of behavior-change technologies and the precise ways in which those effects vary with individual differences, design choices, and contexts in which those technologies are used.

Contextual bandits have become popular as they offer a middle ground between very simple approaches based on multi-armed bandits and very complex approaches using the full power of reinforcement learning. They have demonstrated success in web applications and have a rich body of associated theoretical guarantees. Linear models are well understood theoretically and preferred by practitioners because they are not only easily interpretable but also simple to implement and debug. Furthermore, if the linear model is true, we get very strong performance guarantees. Unfortunately, in emerging applications in mobile health, the time-invariant linear model assumption is untenable. We provide an extension of the linear model for contextual bandits that has two parts: baseline reward and treatment effect. We allow the former to be complex but keep the latter simple. We argue that this model is plausible for mobile health applications. At the same time, it leads to algorithms with strong performance guarantees as in the linear model setting, while still allowing for complex nonlinear baseline modeling. Our theory is supported by experiments on data gathered in a recently concluded mobile health study.

This study examined the use of time-based objects by patients and their families to manage chronic illnesses at home. Calendar systems and medication containers, the main types of time-based objects studied, were used as part of two family-based collaborative work practices: 1) prompting health management activities, and 2) safeguarding these activities. Additionally, these artifacts were part of two social interaction patterns that managed emotional intimacy: 1) expressing support, and 2) hiding and disguising illness. Accordingly, home-based illness management may be more collaborative than previously recognized. Moreover, through their interactive incorporation into family life, time-based objects are laden with psychosocial significance. Breakdowns in temporal support were also evident, and were accompanied by: missed medication events; rationing of medications; medication errors; and difficulties with preparation for medical appointments. We propose novel artifact designs to better support patients and their families in managing the temporal aspects of chronic illness together.

Researchers in HCI and behavioral science are increasingly exploring the use of technology to support behavior change in domains such as health and sustainability. This work, however, remain largely siloed within the two communities. We begin to address this silo problem by attempting to build a bridge between the two disciplines at the level of behavioral theory. Specifically, we define core theoretical terms to create shared understanding about what theory is, discuss ways in which behavioral theory can be used to inform research on behavior change technologies, identify shortcomings in current behavioral theories, and outline ways in which HCI researchers can not only interpret and utilize behavioral science theories but also contribute to improving them.

New technologies for encouraging physical activity, healthy diet, and other types of health behavior change now frequently appear in the HCI literature. Yet, how such technologies should be evaluated within the context of HCI research remains unclear. In this paper, we argue that the obvious answer to this question - that evaluations should assess whether a technology brought about the intended change in behavior - is too limited. We propose that demonstrating behavior change is often infeasible as well as unnecessary for a meaningful contribution to HCI research, especially when in the early stages of design or when evaluating novel technologies. As an alternative, we suggest that HCI contributions should focus on efficacy evaluations that are tailored to the specific behavior-change intervention strategies (e.g., self-monitoring, conditioning) embodied in the system and studies that help gain a deep understanding of people's experiences with the technology.

Patients do considerable information work. Technologies that help patients manage health information so they can play active roles in their health-care, such as personal health records, provide patients with effective support for focused and sustained personal health tasks. Yet, little attention has been paid to patients' needs for information management support while on the go and away from their personal health information collections. Through a qualitative field study, we investigated the information work that breast cancer patients do in such 'unanchored settings'. We report on the types of unanchored information work that patients do over the course of cancer treatment, reasons this work is challenging, and strategies used by patients to overcome those challenges. Our description of unanchored patient information work expands our understanding of patients' information practices and points to valuable design directions for supporting critical but unmet needs.

The greatest contributor of CO2 emissions in the average American household is personal transportation. Because transportation is inherently a mobile activity, mobile devices are well suited to sense and provide feedback about these activities. In this paper, we explore the use of personal ambient displays on mobile phones to give users feedback about sensed and self-reported transportation behaviors. We first present results from a set of formative studies exploring our respondents' existing transportation routines, willingness to engage in and maintain green transportation behavior, and reactions to early mobile phone "green" application design concepts. We then describe the results of a 3-week field study (N=13) of the UbiGreen Transportation Display prototype, a mobile phone application that semi-automatically senses and reveals information about transportation behavior. Our contributions include a working system for semi-automatically tracking transit activity, a visual design capable of engaging users in the goal of increasing green transportation, and the results of our studies, which have implications for the design of future green applications.

Increasingly, users access online services such as email, e-commerce, and social networking sites via 802.11-based wireless networks. As they do so, they expose a range of personal information such as their names, email addresses, and ZIP codes to anyone within broadcast range of the network. This paper presents results from an exploratory study that examined how users from the general public understand Wi-Fi, what their concerns are related to Wi-Fi use, and which practices they follow to counter perceived threats. Our results reveal that while users understand the practical details of Wi-Fi use reasonably well, they lack understanding of important privacy risks. In addition, users employ incomplete protective practices which results in a false sense of security and lack of concern while on Wi-Fi. Based on our results, we outline opportunities for technology to help address these problems.

Goal-setting has been shown to be an effective strategy for changing behavior; therefore employing goal-setting in persuasive technologies could be an effective way to encourage behavior change. In our work, we are developing persuasive technologies to encourage individuals to live healthy lifestyles with a focus on being physically active. As part of our investigations, we have explored individuals' reactions to goal-setting, specifically goal sources (i.e., who should set the individual's goal) and goal timeframes (i.e., over what time period should an individual have to achieve the goal). In this paper, we present our findings related to various approaches for implementing goal-setting in a persuasive technology to encourage physical activity.

The design, development, and deployment of interactive systems can substantively impact individuals, society, and the natural environment, now and potentially well into the future. Yet, a scarcity of methods exists to support long-term, emergent, systemic thinking in interactive design practice. Toward addressing this gap, we propose four envisioning criteria --- stakeholders, time, values, and pervasiveness -- distilled from prior work in urban planning, design noir, and Value Sensitive Design. We characterize how the criteria can support systemic thinking, illustrate the integration of the envisioning criteria into established design practice (scenariobased design), and provide strategic activities to serve as generative envisioning tools. We conclude with suggestions for use and future work. Key contributions include: 1) four envisioning criteria to support systemic thinking, 2) value scenarios (extending scenario-based design), and 3) strategic activities for engaging the envisioning criteria in interactive system design practice.

Recent advances in small inexpensive sensors, low-power processing, and activity modeling have enabled applications that use on-body sensing and machine learning to infer people's activities throughout everyday life. To address the growing rate of sedentary lifestyles, we have developed a system, UbiFit Garden, which uses these technologies and a personal, mobile display to encourage physical activity. We conducted a 3-week field trial in which 12 participants used the system and report findings focusing on their experiences with the sensing and activity inference. We discuss key implications for systems that use on-body sensing and activity inference to encourage physical activity.

Prototyping and evaluation combine to explore ways that an effective, integrative organization of project-related information might emerge as a by-product of a person's efforts to plan a project. The Personal Project Planner works as an extension to the file manager -- providing people with rich-text overlays to their information. Document-like project plans provide a context in which to create or reference documents, email messages, web pages, etc. that are needed to complete the plan. The user can later locate an information item such as an email message with reference to the plan (e.g., as an alternative to searching through the inbox or sent mail). Results of an interim evaluation of the Planner are very promising and suggest special directions of focus for limited available prototyping resources.

Personal, mobile displays, such as those on mobile phones, are ubiquitous, yet for the most part, underutilized. We present results from a field experiment that investigated the effectiveness of these displays as a means for improving awareness of daily life (in our case, self-monitoring of physical activity). Twenty-eight participants in three experimental conditions used our UbiFit system for a period of three months in their day-to-day lives over the winter holiday season. Our results show, for example, that participants who had an awareness display were able to maintain their physical activity level (even during the holidays), while the level of physical activity for participants who did not have an awareness display dropped significantly. We discuss our results and their general implications for the use of everyday mobile devices as awareness displays.

Users’ ability to accurately recall frequent, habitual activities is fundamental to a number of disciplines, from health sciences to machine learning. However, few, if any, studies exist that have assessed optimal sampling strategies for in situ self-reports. In addition, few technologies exist that facilitate benchmarking self-report accuracy for routine activities. We report on a study investigating the effect of sampling frequency of self-reports of two routine activities (sitting and walking) on recall accuracy and annoyance. We used a novel wearable sensor platform that runs a real time activity inference engine to collect in situ ground truth. Our results suggest that a sampling frequency of five to eight times per day may yield an optimal balance of recall and annoyance. Additionally, requesting self-reports at regular, predetermined times increases accuracy while minimizing perceived annoyance since it allows participants to anticipate these requests. We discuss our results and their implications for future studies.