OBJECTIVE: Although biomedical informatics has multiple roles to play in addressing the climate crisis, collaborative action and research agendas have yet to be developed. As a first step, AMIA's new Climate, Health, and Informatics Working Group held a mini-summit entitled Climate and health: How can informatics help? during the AMIA 2023 Fall Symposium to define an initial set of areas of interest and begin mobilizing informaticians to confront the urgent challenges of climate change. MATERIALS AND METHODS: The AMIA Climate, Health, and Informatics Working Group (at the time, an AMIA Discussion Forum), the International Medical Informatics Association (IMIA), the International Academy of Health Sciences Informatics (IAHSI), and the Regenstrief Institute hosted a mini-summit entitled Climate and health: How can informatics help? on November 11, 2023, during the AMIA 2023 Annual Symposium (New Orleans, LA, USA). Using an affinity diagramming approach, the mini-summit organizers posed 2 questions to ∼50 attendees (40 in-person, 10 virtual). RESULTS: Participants expressed a broad array of viewpoints on actions that can be undertaken now and areas needing research to support future actions. Areas of current action ranged from enhanced education to expanded telemedicine to assessment of community vulnerability. Areas of research ranged from emergency preparedness to climate-specific clinical coding to risk prediction models. DISCUSSION: The mini-summit was intended as a first step in helping the informatics community at large set application and research priorities for climate, health, and informatics. CONCLUSION: The working group will use these perspectives as it seeks further input, and begins to establish priorities for climate-related biomedical informatics actions and research.

The ability of large language models (LLMs) to follow natural language instructions with human-level fluency suggests many opportunities in healthcare to reduce administrative burden and improve quality of care. However, evaluating LLMs on realistic text generation tasks for healthcare remains challenging. Existing question answering datasets for electronic health record (EHR) data fail to capture the complexity of information needs and documentation burdens experienced by clinicians. To address these challenges, we introduce MedAlign, a benchmark dataset of 983 natural language instructions for EHR data. MedAlign is curated by 15 clinicians (7 specialities), includes clinician-written reference responses for 303 instructions, and provides 276 longitudinal EHRs for grounding instruction-response pairs. We used MedAlign to evaluate 6 general domain LLMs, having clinicians rank the accuracy and quality of each LLM response. We found high error rates, ranging from 35% (GPT-4) to 68% (MPT-7B-Instruct), and 8.3% drop in accuracy moving from 32k to 2k context lengths for GPT-4. Finally, we report correlations between clinician rankings and automated natural language generation metrics as a way to rank LLMs without human review. MedAlign is provided under a research data use agreement to enable LLM evaluations on tasks aligned with clinician needs and preferences.

OBJECTIVE: Establish a baseline of informatics professionals' perspectives on climate change and health. MATERIALS AND METHODS: Anonymized survey sent to 9 informatics listservs March 31, 2022 to April 15, 2022. RESULTS: N = 85 participants completed part or all of survey. Majority of participants worked at hospitals with 1000+ employees (73%) in urban areas (60%) in the United States. Respondents broadly reported general understanding of climate change and health (51%), but 71% reported unfamiliarity with technologies that could help clinicians and informaticians address the impacts of climate change. Seventy-one percent of surveyed wanted climate-driven environmental health information included in EHRs. Seventy-six percent of respondents reported that informaticians should be involved in institutional decarbonization. Seventy-eight percent of respondents felt that it was extremely, very, or moderately important to receive education on climate change. DISCUSSION: General consensus on need to engage informaticians in climate change response, but gaps identified in knowledge dissemination and tools for adaptation and mitigation. CONCLUSION: Informaticians broadly concerned about climate change and want to be engaged in efforts to combat it, but further education and tool development needed.