Effective patient-provider communication is often hindered by disparities in medical knowledge and the use of technical jargon. Although analogies and metaphors can help bridge this gap, physicians struggle to generate them under clinical time pressure, highlighting a need for supportive design. Through formative interviews with patients and physicians, we identified requirements for explanatory tools that are clear, accurate, and context-sensitive. In response, we designed MediMate, an interactive system that allows physicians to rehearse and iteratively refine patient-friendly explanations using LLM-generated analogies in a low-stakes setting. The interface of MediMate is designed to scaffold the creative process, helping physicians balance clarity with medical accuracy. In a user study involving both physicians and patients, we found that explanations developed with MediMate enhanced communication efficiency by providing such scaffolding. Physicians reported increased self-efficacy and perceived value in using the system as a practice tool for developing their communication skills. Our work demonstrates how interactive AI-powered tools can support clinical communication rehearsal and offers insights for the design of future clinical decision-support and educational tools.

Preparing an oral case presentation (OCP) is a crucial skill for medical students, requiring clear communication of patient information, clinical findings, and treatment plans. However, inconsistent student participation and limited guidance can make this task challenging. While Large Language Models (LLMs) can provide structured content to streamline the process, their role in facilitating skill development and supporting medical education integration remains underexplored. To address this, we conducted a formative study with six medical educators and developed CaseMaster, an interactive probe that leverages LLM-generated content tailored to medical education to help users enhance their OCP skills. The controlled study suggests CaseMaster has the potential to both improve presentation quality and reduce workload compared to traditional methods, an implication reinforced by expert feedback. We propose guidelines for educators to develop adaptive, user-centered training methods using LLMs, while considering the implications of integrating advanced technologies into medical education.

Large language models (LLMs) have shown considerable potential in supporting medical diagnosis. However, their effective integration into clinical workflows is hindered by physicians’ difficulties in perceiving and trusting LLM capabilities, which often results in miscalibrated trust. Existing model evaluations primarily emphasize standardized benchmarks and predefined tasks, offering limited insights into clinical reasoning practices. Moreover, research on human–AI collaboration has rarely examined physicians’ perceptions of LLMs’ clinical reasoning capability. In this work, we investigate how physicians perceive LLMs’ capabilities in the clinical reasoning process. We designed clinical cases, collected the corresponding analyses, and obtained evaluations from physicians (N=37) to quantitatively represent their perceived LLM diagnostic capabilities. By comparing the perceived evaluations with benchmark performance, our study highlights the aspects of clinical reasoning that physicians value and underscores the limitations of benchmark-based evaluation. We further discuss the implications of opportunities for enhancing trustworthy collaboration between physicians and LLMs in LLM-supported clinical reasoning.

Surgical emergencies often trigger acute cognitive overload in novice physicians, impairing their decision-making under pressure. Although Virtual Reality–based Stress Inoculation Training (VR-SIT) shows promise, current systems fall short in delivering real-time, effective support during moments of peak stress. To bridge this gap, we first conducted a formative study (N=12) to uncover the core needs of novice physicians for immediate assistance under acute stress and identified three key intervention strategies: self-regulation aids, procedure guidance, and emotional/sensory support. Building on these insights, we designed and implemented a novel VR-SIT system that incorporates a just-in-time adaptive intervention framework, dynamically tailoring support to learners’ cognitive and emotional states. We then validated these strategies in a user study (N=26). Our findings provide empirical evidence and design implications for next-generation VR medical training systems, supporting physicians in sustaining cognitive clarity and accurate decision-making in critical situations.

Modern scientific discovery faces challenges in integrating the rapidly expanding and diverse knowledge required for exploring novel knowledge in biology. While traditional hypothesis-driven research has proven effective, it is constrained by human cognitive limitations, knowledge complexity, and the high costs of trial-and-error experimentation. Deep learning models, particularly graph neural networks (GNNs), have accelerated scientific progress. However, the vast predictions generated make manual selection for experimental validation impractical. Attempts to leverage large language models (LLMs) for filtering predictions and generating novel hypotheses have been impeded by issues such as hallucinations and the lack of structured knowledge grounding, which undermine their reliability. To address these challenges, we propose HypoChainer, a collaborative visualization framework that integrates human expertise, LLM-driven reasoning, and knowledge graphs (KGs) to enhance scientific discovery visually. HypoChainer operates through three key stages: (1) Contextual Exploration: Domain experts employ retrieval-augmented LLMs (RAGs) and visualizations to extract insights and research focuses from vast GNN predictions, supplemented by interactive explanations for in-depth understanding; (2) Hypothesis Construction: Experts iteratively explore the KG information relevant to the predictions and hypothesis-aligned entities, gaining knowledge and insights while refining the hypothesis through suggestions from LLMs; and (3) Validation Selection: Predictions are prioritized based on the refined hypothesis chains and KG-supported evidence, identifying high-priority candidates for validation. The hypothesis chains are further optimized through visual analytics of the retrieval results. We evaluated the effectiveness of HypoChainer in hypothesis construction and scientific discovery through a case study and expert interviews.

Handheld ultrasound devices face usage limitations due to user inexperience and cannot benefit from supervised deep learning without extensive expert annotations. Moreover, the models trained on standard ultrasound device data are constrained by training data distribution and perform poorly when directly applied to handheld device data. In this study, we propose the Training-free Image Style Alignment (TISA) to align the style of handheld device data to those of standard devices. The proposed TISA eliminates the demand for source data, and can transform the image style while preserving spatial context during testing. Furthermore, our TISA avoids continuous updates to the pre-trained model compared to other test-time methods and is suited for clinical applications. We show that TISA performs better and more stably in medical detection and segmentation tasks for handheld device data than other test-time adaptation methods. We further validate TISA as the clinical model for automatic measurements of spinal curvature and carotid intima-media thickness, and the automatic measurements agree well with manual measurements made by human experts. We demonstrate the potential for TISA to facilitate automatic diagnosis on handheld ultrasound devices and expedite their eventual widespread use. Code is available at https://github.com/zenghy96/TISA.

The process of labeling medical text plays a crucial role in medical research. Nonetheless, creating accurately labeled medical texts of high quality is often a time-consuming task that requires specialized domain knowledge. Traditional methods for generating labeled data typically rely on rigid rule-based approaches, which may not adapt well to new tasks. While recent machine learning (ML) methodologies have mitigated the manual labeling efforts, configuring models to align with specific research requirements can be challenging for labelers without technical expertise. Moreover, automated labeling techniques, such as transfer learning, face difficulties in in directly incorporating expert input, whereas semi-automated methods, like data programming, allow knowledge integration through rules or knowledge bases but may lack continuous result refinement throughout the entire labeling process. In this study, we present a collaborative human-ML teaming workflow that seamlessly integrates visual cluster analysis and active learning to assist domain experts in labeling medical text with high efficiency. Additionally, we introduce an innovative neural network model called the embedding network, which incorporates expert insights to generate task-specific embeddings for medical texts. We integrate the workflow and embedding network into a visual analytics tool named KMTLabeler, equipped with coordinated multi-level views and interactions. Two illustrative case studies, along with a controlled user study, provide substantial evidence of the effectiveness of KMTLabeler in creating an efficient labeling environment for medical text classification.

In healthcare, AI techniques are widely used for tasks like risk assessment and anomaly detection. Despite AI’s potential as a valuable assistant, its role in complex medical data analysis often over-simplifies human-AI collaboration dynamics. To address this, we collaborated with a local hospital, engaging six physicians and one data scientist in a formative study. From this collaboration, we propose a framework integrating two-phase interactive visualization systems: one for Human-Led, AI-Assisted Retrospective Analysis and another for AI-Mediated, Human-Reviewed Iterative Modeling. This framework aims to enhance understanding and discussion around effective human-AI collaboration in healthcare.

Simulation-based Medical Education (SBME) has been developed as a cost-effective means of enhancing the diagnostic skills of novice physicians and interns, thereby mitigating the need for resource-intensive mentor-apprentice training. However, feedback provided in most SBME is often directed towards improving the operational proficiency of learners, rather than providing summative medical diagnoses that result from experience and time. Additionally, the multimodal nature of medical data during diagnosis poses significant challenges for interns and novice physicians, including the tendency to overlook or over-rely on data from certain modalities, and difficulties in comprehending potential associations between modalities. To address these challenges, we present DiagnosisAssistant, a visual analytics system that leverages historical medical records as a proxy for multimodal modeling and visualization to enhance the learning experience of interns and novice physicians. The system employs elaborately designed visualizations to explore different modality data, offer diagnostic interpretive hints based on the constructed model, and enable comparative analyses of specific patients. Our approach is validated through two case studies and expert interviews, demonstrating its effectiveness in enhancing medical training.