OBJECTIVE: This study examined whether attendance at online digital health research appointments in the American Women Assessing Risk Epidemiologically (AWARE) study was associated with (1) participant age, (2) scheduling factors (time of day, day of week, month), (3) appointment confirmation, and (4) HIV behavioral risk factors. MATERIALS AND METHODS: We analyzed scheduling and eligibility screening data from AWARE, a 24-month U.S.-based longitudinal digital cohort of cisgender women at elevated likelihood of HIV seroconversion. Participant demographic and behavioral data were merged with the study team's Outlook calendar. Chi-square tests and logistic regression models assessed associations between appointment attendance and participant characteristics and scheduling factors. RESULTS: Women aged ≥50 years had higher odds of missing baseline visits compared to those aged 20-29 years (44.7% vs 32.3%). Appointments scheduled at 2:00 pm (45.7%), 4:00 pm (45.2%), and 8:00 am (40.2%) had higher no-show rates than other times. No-show rates were lowest on Fridays (30.2%) and during March (27.7%) and June (25.2%). Confirming appointments 24 hours in advance significantly reduced no-shows compared to no confirmation (19.0% vs 51.6%). Histories of having been physically hurt (44.2% vs 32.1%), forced to have sexual activities (41.8% vs 34.1%) and incarcerated (39.3% vs 33.4%) were also associated with higher no-show rates. Similar patterns were observed for rescheduled visits. CONCLUSION: Attendance in digital research was influenced by age, scheduling, and structural vulnerabilities. Incorporating digital access support into study design and grant budgets may reduce disparities, improve retention, and enhance efficiency.

OBJECTIVE: To identify demographic, social, and clinical factors associated with HIV self-management and evaluate whether the CHAMPS intervention is associated with changes in an individual's HIV self-management. METHOD: This study was a secondary data analysis from a randomized controlled trial evaluating the effects of the CHAMPS, a mHealth intervention with community health worker sessions, on HIV self-management in New York City (NYC) and Birmingham. Group comparisons and linear regression analyses identified demographic, social, and clinical factors associated with HIV self-management. We calculated interactions between groups (CHAMPS intervention and standard of care) over time (6 and 12 months) following the baseline observation, indicating a difference in the outcome scores from baseline to each time across groups. RESULTS: Our findings indicate that missing medical appointments, uncertainty about accessing care, and lack of adherence to antiretroviral therapy are associated with lower HIV self-management. For the NYC site, the CHAMPS showed a statistically significant positive effect on daily HIV self-management (estimate = 0.149, SE = 0.069, 95% CI [0.018 to 0.289]). However, no significant effects were observed for social support or the chronic nature of HIV self-management. At the Birmingham site, the CHAMPS did not yield statistically significant effects on HIV self-management outcomes. DISCUSSION: Our study suggests that CHAMPS intervention enhances daily self-management activities for people with HIV at the NYC site, indicating a promising improvement in routine HIV care. CONCLUSION: Further research is necessary to explore how various factors influence HIV self-management over time across different regions.

OBJECTIVE: To determine the efficacy of the mLab App, a mobile-delivered HIV prevention intervention to increase HIV self-testing in MSM and TGW. MATERIALS AND METHODS: This was a randomized (2:2:1) clinical trial of the efficacy the mLab App as compared to standard of care vs mailed home HIV test arm among 525 MSM and TGW aged 18-29 years to increase HIV testing. RESULTS: The mLab App arm participants demonstrated an increase from 35.1% reporting HIV testing in the prior 6 months compared to 88.5% at 6 months. In contrast, 28.8% of control participants reported an HIV test at baseline, which only increased to 65.1% at 6 months. In a generalized linear mixed model estimating this change and controlling for multiple observations of participants, this equated to control participants reporting a 61.2% smaller increase in HIV testing relative to mLab participants (P = .001) at 6 months. This difference was maintained at 12 months with control participants reporting an 82.6% smaller increase relative to mLab App participants (P < .001) from baseline to 12 months. DISCUSSION AND CONCLUSION: Findings suggest that the mLab App is well-supported, evidence-based, behavioral risk-reduction intervention for increasing HIV testing rates as compared to the standard of care, suggesting that this may be a useful behavioral risk-reduction intervention for increasing HIV testing among young MSM. TRIAL REGISTRATION: This trial was registered with Clinicaltrials.gov NCT03803683.

IMPORTANCE: Due to insufficient smoking cessation apps for persons living with HIV, our study focused on designing and testing the Sense2Quit app, a patient-facing mHealth tool which integrated visualizations of patient information, specifically smoking use. OBJECTIVES: The purpose of this paper is to detail rigorous human-centered design methods to develop and refine visualizations of smoking data and the contents and user interface of the Sense2Quit app. The Sense2Quit app was created to support tobacco cessation and relapse prevention for people living with HIV. MATERIALS AND METHODS: Twenty people living with HIV who are current or former smokers and 5 informaticians trained in human-computer interaction participated in 5 rounds of usability testing. Participants tested the Sense2Quit app with use cases and provided feedback and then completed a survey. RESULTS: Visualization of smoking behaviors was refined through each round of usability testing. Further, additional features such as daily tips, games, and a homescreen were added to improve the usability of the app. A total of 66 changes were made to the Sense2Quit app based on end-user and expert recommendations. DISCUSSION: While many themes overlapped between usability testing with end-users and heuristic evaluations, there were also discrepancies. End-users and experts approached the app evaluation from different perspectives which ultimately allowed us to fill knowledge gaps and make improvements to the app. CONCLUSION: Findings from our study illustrate the best practices for usability testing for development and refinement of an mHealth-delivered consumer informatics tool for improving tobacco cessation yet further research is needed to fully evaluate how tools informed by target user needs improve health outcomes.

BACKGROUND: Alzheimer's disease and related dementias (ADRD) affect over 55 million globally. Current clinical trials suffer from low recruitment rates, a challenge potentially addressable via natural language processing (NLP) technologies for researchers to effectively identify eligible clinical trial participants. OBJECTIVE: This study investigates the sociotechnical feasibility of NLP-driven tools for ADRD research prescreening and analyzes the tools' cognitive complexity's effect on usability to identify cognitive support strategies. METHODS: A randomized experiment was conducted with 60 clinical research staff using three prescreening tools (Criteria2Query, Informatics for Integrating Biology and the Bedside [i2b2], and Leaf). Cognitive task analysis was employed to analyze the usability of each tool using the Health Information Technology Usability Evaluation Scale. Data analysis involved calculating descriptive statistics, interrater agreement via intraclass correlation coefficient, cognitive complexity, and Generalized Estimating Equations models. RESULTS: Leaf scored highest for usability followed by Criteria2Query and i2b2. Cognitive complexity was found to be affected by age, computer literacy, and number of criteria, but was not significantly associated with usability. DISCUSSION: Adopting NLP for ADRD prescreening demands careful task delegation, comprehensive training, precise translation of eligibility criteria, and increased research accessibility. The study highlights the relevance of these factors in enhancing NLP-driven tools' usability and efficacy in clinical research prescreening. CONCLUSION: User-modifiable NLP-driven prescreening tools were favorably received, with system type, evaluation sequence, and user's computer literacy influencing usability more than cognitive complexity. The study emphasizes NLP's potential in improving recruitment for clinical trials, endorsing a mixed-methods approach for future system evaluation and enhancements.

OBJECTIVE: To pilot test an infographic-based health communication intervention that our team rigorously designed and explore whether its implementation leads to better health outcomes among Latino persons with HIV (PWH). MATERIALS AND METHODS: Latino PWH (N = 30) living in New York City received the intervention during health education sessions at 3 study visits that occurred approximately 3 months apart. At each visit, participants completed baseline or follow-up assessments and laboratory data were extracted from patient charts. We assessed 6 outcomes (HIV-related knowledge, self-efficacy to manage HIV, adherence to antiretroviral therapy, CD4 count, viral load, and current and overall health status) selected according to a conceptual model that describes pathways through which communication influences health outcomes. We assessed changes in outcomes over time using quantile and generalized linear regression models controlling for the coronavirus disease 2019 (COVID-19) research pause and new patient status (new/established) at the time of enrollment. RESULTS: Most participants were male (60%) and Spanish-speaking (60%); 40% of participants identified as Mixed Race/Mestizo, 13.3% as Black, 13.3% as White, and 33.3% as "other" race. Outcome measures generally improved after the second intervention exposure. Following the third intervention exposure (after the COVID-19 research pause), only the improvements in HIV-related knowledge and current health status were statistically significant. DISCUSSION AND CONCLUSION: Our infographic-based health communication intervention may lead to better health outcomes among Latino PWH, but larger trials are needed to establish efficacy. From this work, we contribute suggestions for effective infographic use for patient-provider communication to enhance patient education in clinical settings.

IMPORTANCE: Progression of HIV disease, the transmission of the disease, and premature deaths among persons living with HIV (PLWH) have been attributed foremost to poor adherence to HIV medications. mHealth tools can be used to improve antiretroviral therapy (ART) adherence in PLWH and have the potential to improve therapeutic success. OBJECTIVE: To determine the efficacy of WiseApp, a user-centered design mHealth intervention to improve ART adherence and viral suppression in PLWH. DESIGN, SETTING, AND PARTICIPANTS: A randomized (1:1) controlled efficacy trial of the WiseApp intervention arm (n = 99) versus an attention control intervention arm (n = 101) among persons living with HIV who reported poor adherence to their treatment regimen and living in New York City. INTERVENTIONS: The WiseApp intervention includes the following components: testimonials of lived experiences, push-notification reminders, medication trackers, health surveys, chat rooms, and a "To-Do" list outlining tasks for the day. Both study arms also received the CleverCap pill bottle, with only the intervention group linking the pill bottle to WiseApp. RESULTS: We found a significant improvement in ART adherence in the intervention arm compared to the attention control arm from day 1 (69.7% vs 48.3%, OR = 2.5, 95% CI 1.4-3.5, P = .002) to day 59 (51.2% vs 37.2%, OR = 1.77, 95% CI 1.0-1.6, P = .05) of the study period. From day 60 to 120, the intervention arm had higher adherence rates, but the difference was not significant. In the secondary analyses, no difference in change from baseline to 3 or 6 months between the 2 arms was observed for all secondary outcomes. CONCLUSIONS: The WiseApp intervention initially improved ART adherence but did not have a sustained effect on outcomes.

Smoking cessation is a significant challenge for many people addicted to cigarettes and tobacco. Mobile health-related research into smoking cessation is primarily focused on mobile phone data collection either using self-reporting or sensor monitoring techniques. In the past 5 years with the increased popularity of smartwatch devices, research has been conducted to predict smoking movements associated with smoking behaviors based on accelerometer data analyzed from the internal sensors in a user's smartwatch. Previous smoking detection methods focused on classifying current user smoking behavior. For many users who are trying to quit smoking, this form of detection may be insufficient as the user has already relapsed. In this paper, we present a smoking cessation system utilizing a smartwatch and finger sensor that is capable of detecting pre-smoking activities to discourage users from future smoking behavior. Pre-smoking activities include grabbing a pack of cigarettes or lighting a cigarette and these activities are often immediately succeeded by smoking. Therefore, through accurate detection of pre-smoking activities, we can alert the user before they have relapsed. Our smoking cessation system combines data from a smartwatch for gross accelerometer and gyroscope information and a wearable finger sensor for detailed finger bend-angle information. We compare the results of a smartwatch-only system with a combined smartwatch and finger sensor system to illustrate the accuracy of each system. The combined smartwatch and finger sensor system performed at an 80.6% accuracy for the classification of pre-smoking activities compared to 47.0% accuracy of the smartwatch-only system.

OBJECTIVE: To combine machine efficiency and human intelligence for converting complex clinical trial eligibility criteria text into cohort queries. MATERIALS AND METHODS: Criteria2Query (C2Q) 2.0 was developed to enable real-time user intervention for criteria selection and simplification, parsing error correction, and concept mapping. The accuracy, precision, recall, and F1 score of enhanced modules for negation scope detection, temporal and value normalization were evaluated using a previously curated gold standard, the annotated eligibility criteria of 1010 COVID-19 clinical trials. The usability and usefulness were evaluated by 10 research coordinators in a task-oriented usability evaluation using 5 Alzheimer's disease trials. Data were collected by user interaction logging, a demographic questionnaire, the Health Information Technology Usability Evaluation Scale (Health-ITUES), and a feature-specific questionnaire. RESULTS: The accuracies of negation scope detection, temporal and value normalization were 0.924, 0.916, and 0.966, respectively. C2Q 2.0 achieved a moderate usability score (3.84 out of 5) and a high learnability score (4.54 out of 5). On average, 9.9 modifications were made for a clinical study. Experienced researchers made more modifications than novice researchers. The most frequent modification was deletion (5.35 per study). Furthermore, the evaluators favored cohort queries resulting from modifications (score 4.1 out of 5) and the user engagement features (score 4.3 out of 5). DISCUSSION AND CONCLUSION: Features to engage domain experts and to overcome the limitations in automated machine output are shown to be useful and user-friendly. We concluded that human-computer collaboration is key to improving the adoption and user-friendliness of natural language processing.

OBJECTIVE: We conducted a systematic review to assess the effect of natural language processing (NLP) systems in improving the accuracy and efficiency of eligibility prescreening during the clinical research recruitment process. MATERIALS AND METHODS: Guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards of quality for reporting systematic reviews, a protocol for study eligibility was developed a priori and registered in the PROSPERO database. Using predetermined inclusion criteria, studies published from database inception through February 2021 were identified from 5 databases. The Joanna Briggs Institute Critical Appraisal Checklist for Quasi-experimental Studies was adapted to determine the study quality and the risk of bias of the included articles. RESULTS: Eleven studies representing 8 unique NLP systems met the inclusion criteria. These studies demonstrated moderate study quality and exhibited heterogeneity in the study design, setting, and intervention type. All 11 studies evaluated the NLP system's performance for identifying eligible participants; 7 studies evaluated the system's impact on time efficiency; 4 studies evaluated the system's impact on workload; and 2 studies evaluated the system's impact on recruitment. DISCUSSION: NLP systems in clinical research eligibility prescreening are an understudied but promising field that requires further research to assess its impact on real-world adoption. Future studies should be centered on continuing to develop and evaluate relevant NLP systems to improve enrollment into clinical studies. CONCLUSION: Understanding the role of NLP systems in improving eligibility prescreening is critical to the advancement of clinical research recruitment.

OBJECTIVE: The study sought to design symptom reports of longitudinal patient-reported outcomes data that are understandable and meaningful to end users. MATERIALS AND METHODS: We completed a 2-phase iterative design and evaluation process. In phase I, we developed symptom reports and refined them according to expert input. End users then completed a survey containing demographics, a measure of health literacy, and items to assess visualization preferences and comprehension of reports. We then collected participants' perspectives on reports through semistructured interviews and modified them accordingly. In phase II, refined reports were evaluated in a survey that included demographics, validated measures of health and graph literacy, and items to assess preferences and comprehension of reports. Surveys were administered using a think-aloud protocol. RESULTS: Fifty-five English- and Spanish-speaking end users, 89.1% of whom had limited health literacy, participated. In phase I, experts recommended improvements and 20 end users evaluated reports. From the feedback received, we added emojis, changed date and font formats, and simplified the y-axis scale of reports. In phase II, 35 end users evaluated refined designs, of whom 94.3% preferred reports with emojis, the favorite being a bar graph combined with emojis, which also promoted comprehension. In both phases, participants literally interpreted reports and provided suggestions for future visualizations. CONCLUSIONS: A bar graph combined with emojis was participants' preferred format and the one that promoted comprehension. Target end users must be included in visualization design to identify literal interpretations of images and ensure final products are meaningful.

OBJECTIVE: The goals of this study were to explore the information needs of case managers who provide services to persons living with HIV (PLWH) and to assess the applicability of the Information Needs Event Taxonomy in a new population. DESIGN: The study design was observational with data collection via an online survey. MEASUREMENTS: Responses to open-ended survey questions about the information needs of case managers (n=94) related to PLWH of three levels of care complexity were categorized using the Information Needs Event Taxonomy. RESULTS: The most frequently identified needs were related to patient education resources (33%), patient data (23%), and referral resources (22%) accounting for 79% of all (N=282) information needs. LIMITATIONS: Study limitations include selection bias, recall bias, and a relatively narrow focus of the study on case-manager information needs in the context of caring for PLWH. CONCLUSION: The study findings contribute to the evidence base regarding information needs in the context of patient interactions by: (1) supporting the applicability of the Information Needs Event Taxonomy and extending it through addition of a new generic question; (2) providing a foundation for the addition of context-specific links to external information resources within information systems; (3) applying a new approach for elicitation of information needs; and (4) expanding the literature regarding addressing information needs in community-based settings for HIV services.