Human Activity Recognition (HAR) is pivotal for behavior monitoring in public healthcare, supporting tasks like medical rehabilitation and targeted wellness campaigns. Initially utilizing hand-crafted features and traditional machine learning models such as support vector machines and decision trees, HAR has since evolved to include deep learning techniques, especially convolutional neural networks and Transformers, which excel at modeling temporal and task-specific information from sensor data. Despite these advancements, challenges related to high task dependency and data scarcity persist. To address these issues, there have been efforts to harness the vast pre-trained knowledge of Large Language Models (LLMs) for HAR. Yet, LLMs often fail to fully capture the temporal dynamics inherent in sensor data. We introduce HyMv, a novel hybrid approach that combines an auxiliary HAR model with soft-prompt tuning of LLMs. This approach leverages the auxiliary model’s proficiency in processing sensor data to guide the parameter optimization of LLMs during prompt tuning. Importantly, the auxiliary HAR model is only active during training, augmenting the LLM’s parameters without adding computational overhead during testing. We evaluate the performance of HyMv for various HAR tasks on diverse datasets and demonstrate its adaptability to different sensor modalities, further showcasing its broad applicability.

Patients with co-morbidities often require multiple medications to manage their conditions. However, existing medication recommendation systems only offer class-level medications and regard all interactions among drugs to have the same level of severity. This limits their ability to provide personalized and safe recommendations tailored to individual needs. In this work, we introduce a deep learning-based fine-grained medication recommendation system called REFINE, which is designed to improve treatment outcomes and minimize adverse drug interactions. In order to better characterize patients’ health conditions, we model the trend in medication dosage titrations and lab test responses, and adapt the vision transformer to obtain effective patient representations. We also model drug interaction severity levels as weighted graphs to learn safe drug combinations and design a balanced loss function to avoid overly conservative recommendations and miss medications that might be needed for certain conditions. Extensive experiments on two real-world datasets show that REFINE outperforms state-of-the-art techniques.

Chronic disease management involves frequent administration of invasive lab procedures in order for clinicians to determine the best course of treatment regimes for these patients. However, patients are often put off by these invasive lab procedures and do not follow the appointment schedules. This has resulted in poor management of their chronic conditions leading to unnecessary disease complications. An AI system that is able to personalize the prediction of individual patient lab test responses will enable clinicians to titrate the medications to achieve the desired therapeutic outcome. Accurate prediction of lab test response is a challenge because these patients typically have co-morbidities and their treatments might influence the target lab test response. To address this, we model the complex interactions among different medications, diseases, lab test response, and fine-grained dosage information to learn a strong patient representation. Together with information from similar patients and external knowledge such as drug-lab interactions and diagnosis-lab interaction, we design a system called KALP to perform personalized prediction of patients’ response for a target lab result and identify the top influencing factors for the prediction. Experiment results on real-world datasets demonstrate the effectiveness of KALP in reducing prediction errors by a significant margin. Case studies show that the identified factors are consistent with clinicians’ understanding.

The broad adoption of electronic health records (EHRs) has led to vast amounts of data being accumulated on a patient’s history, diagnosis, prescriptions, and lab tests. Advances in recommender technologies have the potential to utilize this information to help doctors personalize the prescribed medications. However, existing medication recommendation systems have yet to make use of all these information sources in a seamless manner, and they do not provide a justification on why a particular medication is recommended. In this work, we design a two-stage personalized medication recommender system called PREMIER that incorporates information from the EHR. We utilize the various weights in the system to compute the contributions from the information sources for the recommended medications. Our system models the drug interaction from an external drug database and the drug co-occurrence from the EHR as graphs. Experiment results on MIMIC-III and a proprietary outpatient dataset show that PREMIER outperforms state-of-the-art medication recommendation systems while achieving the best tradeoff between accuracy and drug-drug interaction. Case studies demonstrate that the justifications provided by PREMIER are appropriate and aligned to clinical practices.