Adverse clinical events related to unsafe care are among the top ten causes of death in the U.S. Accurate modeling and prediction of clinical events from electronic health records (EHRs) play a crucial role in patient safety enhancement. An example is modeling de facto care pathways that characterize common step-by-step plans for treatment or care. However, clinical event data pose several unique challenges, including the irregularity of time intervals between consecutive events, the existence of cycles, periodicity, multi-scale event interactions, and the high computational costs associated with long event sequences. Existing neural temporal point processes (TPPs) methods do not effectively capture the multi-scale nature of event interactions, which is common in many real-world clinical applications. To address these issues, we propose the cross-temporal-scale transformer (XTSFormer), specifically designed for irregularly timed event data. Our model consists of two vital components: a novel Feature-based Cycle-aware Time Positional Encoding (FCPE) that adeptly captures the cyclical nature of time, and a hierarchical multi-scale temporal attention mechanism, where different temporal scales are determined by a bottom-up clustering approach. Extensive experiments on several real-world EHR datasets show that our XTSFormer outperforms multiple baseline methods.

Continuous glucose monitoring (CGM) devices provide critical real-time data but remain minimally invasive and require frequent replacement. This study presents a novel, personalized machine learning approach for non-invasive glucose monitoring using radiofrequency (RF) spectroscopy to address these limitations. To simulate real-world usage and ensure clinical relevance, we developed a model for a single individual using data collected during standardized meals. The model was trained and tested on data collected on separate days, ensuring that the training and test sets are drawn from distinct, non-overlapping time periods. A comprehensive machine learning pipeline was validated using 3,101 spectral features (400-3500 MHz) combined with contextual data to predict glucose levels. Our best-performing model, multi-layer perception regressor (MLP), achieved a Mean Absolute Relative Difference (MARD) of 11.6%. These findings demonstrate that a personalized machine learning model holds potential to predict glucose non-invasively. This highlights a promising path toward a more user-friendly and sustainable solution for continuous glucose management

Mobile health (mHealth) apps have gained popularity over the past decade for patient health monitoring, yet their potential for timely intervention is underutilized due to limited integration with electronic health records (EHR) systems. Current EHR systems lack real-time monitoring capabilities for symptoms, medication adherence, physical and social functions, and community integration. Existing systems typically rely on static, in-clinic measures rather than dynamic, real-time patient data. This highlights the need for automated, scalable, and human-centered platforms to integrate patient-generated health data (PGHD) within EHR. Incorporating PGHD in a user-friendly format can enhance patient symptom surveillance, ultimately improving care management and post-surgical outcomes. To address this barrier, we have developed an mHealth platform, ROAMM-EHR, to capture real-time sensor data and Patient Reported Outcomes (PROs) using a smartwatch. The ROAMM-EHR platform can capture data from a consumer smartwatch, send captured data to a secure server, and display information within the Epic EHR system using a user-friendly interface, thus enabling healthcare providers to monitor post-surgical symptoms effectively.

With Artificial Intelligence (AI) increasingly permeating various aspects of society, including healthcare, the adoption of the Transformers neural network architecture is rapidly changing many applications. Transformer is a type of deep learning architecture initially developed to solve general-purpose Natural Language Processing (NLP) tasks and has subsequently been adapted in many fields, including healthcare. In this survey paper, we provide an overview of how this architecture has been adopted to analyze various forms of healthcare data, including clinical NLP, medical imaging, structured Electronic Health Records (EHR), social media, bio-physiological signals, biomolecular sequences. Furthermore, which have also include the articles that used the transformer architecture for generating surgical instructions and predicting adverse outcomes after surgeries under the umbrella of critical care. Under diverse settings, these models have been used for clinical diagnosis, report generation, data reconstruction, and drug/protein synthesis. Finally, we also discuss the benefits and limitations of using transformers in healthcare and examine issues such as computational cost, model interpretability, fairness, alignment with human values, ethical implications, and environmental impact.

Quantifying pain in patients admitted to intensive care units (ICUs) is challenging due to the increased prevalence of communication barriers in this patient population. Previous research has posited a positive correlation between pain and physical activity in critically ill patients. In this study, we advance this hypothesis by building machine learning classifiers to examine the ability of accelerometer data collected from daily wearables to predict self-reported pain levels experienced by patients in the ICU. We trained multiple Machine Learning (ML) models, including Logistic Regression, CatBoost, and XG-Boost, on statistical features extracted from the accelerometer data combined with previous pain measurements and patient demographics. Following previous studies that showed a change in pain sensitivity in ICU patients at night, we performed the task of pain classification separately for daytime and nighttime pain reports. In the pain versus no-pain classification setting, logistic regression gave the best classifier in daytime (AUC: 0.72, F1-score: 0.72), and CatBoost gave the best classifier at nighttime (AUC: 0.82, F1-score: 0.82). Performance of logistic regression dropped to 0.61 AUC, 0.62 F1-score (mild vs. moderate pain, nighttime), and CatBoost's performance was similarly affected with 0.61 AUC, 0.60 F1-score (moderate vs. severe pain, daytime). The inclusion of analgesic information benefited the classification between moderate and severe pain. SHAP analysis was conducted to find the most significant features in each setting. It assigned the highest importance to accelerometer-related features on all evaluated settings but also showed the contribution of the other features such as age and medications in specific contexts. In conclusion, accelerometer data combined with patient demographics and previous pain measurements can be used to screen painful from painless episodes in the ICU and can be combined with analgesic information to provide moderate classification between painful episodes of different severities.

Existing pain assessment methods in the intensive care unit rely on patient self-report or visual observation by nurses. Patient self-report is subjective and can suffer from poor recall. In the case of non-verbal patients, behavioral pain assessment methods provide limited granularity, are subjective, and put additional burden on already overworked staff. Previous studies have shown the feasibility of autonomous pain expression assessment by detecting Facial Action Units (AUs). However, previous approaches for detecting facial pain AUs are historically limited to controlled environments. In this study, for the first time, we collected and annotated a pain-related AU dataset, Pain-ICU, containing 55,085 images from critically ill adult patients. We evaluated the performance of OpenFace, an open-source facial behavior analysis tool, and the trained AU R-CNN model on our Pain-ICU dataset. Variables such as assisted breathing devices, environmental lighting, and patient orientation with respect to the camera make AU detection harder than with controlled settings. Although OpenFace has shown state-of-the-art results in general purpose AU detection tasks, it could not accurately detect AUs in our Pain-ICU dataset (F1-score 0.42). To address this problem, we trained the AU R-CNN model on our Pain-ICU dataset, resulting in a satisfactory average F1-score 0.77. In this study, we show the feasibility of detecting facial pain AUs in uncontrolled ICU settings.

Sentiment analysis is a well-researched field of machine learning and natural language processing generally concerned with determining the degree of positive or negative polarity in free text. Traditionally, such methods have focused on analyzing user opinions directed towards external entities such as products, news, or movies. However, less attention has been paid towards understanding the sentiment of human emotion in the form of internalized thoughts and expressions of self-reflection. Given the rise of public social media platforms and private online therapy services, the opportunity for designing accurate tools to quantify emotional states in is at an all-time high. Based upon psychological research, in this work we propose a new type of sentiment analysis task using a two-dimensional valence scheme with four sentiment categories: positive, negative, both positive and negative, and neither positive nor negative. This work details the collection of a novel annotated dataset of real-world mental health therapy logs and compares several machine learning methodologies for the accurate classification of emotional valence. We found superior performance using deep transfer learning approaches, in particular using the recent breakthrough method of BERT. We argue that representing emotional sentiment on decoupled valence axes is an appropriate modification of traditional sentiment analysis for mental health tasks and that modern transfer learning approaches should become an essential component of automated mental health frameworks, where labeled data is often scarce.

In cognitive psychology, automatic and self-reinforcing irrational thought patterns are known as cognitive distortions. Left unchecked, patients exhibiting these types of thoughts can become stuck in negative feedback loops of unhealthy thinking, leading to inaccurate perceptions of reality commonly associated with anxiety and depression. In this paper, we present a machine learning framework for the automatic detection and classification of 15 common cognitive distortions in two novel mental health free datasets collected from both crowdsourcing and a real-world online therapy program. We also performed an exploratory analysis using unsupervised content-based clustering and topic modeling algorithms as first efforts towards a data-driven perspective on the thematic relationship between similar cognitive distortions traditionally deemed unique. Finally, we highlight the difficulties in applying mental health-based machine learning in a real-world setting and comment on the implications and benefits of our framework for improving automated delivery of therapeutic treatment in conjunction with traditional cognitive-behavioral therapy.

The modern healthcare landscape has seen the rapid emergence of techniques and devices that temporally monitor and record physiological signals. The prevalence of time-series data within the healthcare field necessitates the development of methods that can analyze the data in order to draw meaningful conclusions. Time-series behavior is notoriously difficult to intuitively understand due to its intrinsic high-dimensionality, which is compounded in the case of analyzing groups of time series collected from different patients. Our framework, which we call transition icons, renders common patterns in a visual format useful for understanding the shared behavior within groups of time series. Transition icons are adept at detecting and displaying subtle differences and similarities, e.g., between measurements taken from patients receiving different treatment strategies or stratified by demographics. We introduce various methods that collectively allow for exploratory analysis of groups of time series, while being free of distribution assumptions and including simple heuristics for parameter determination. Our technique extracts discrete transition patterns from symbolic aggregate approXimation representations, and compiles transition frequencies into a bag of patterns constructed for each group. These transition frequencies are normalized and aligned in icon form to intuitively display the underlying patterns. We demonstrate the transition icon technique for two time-series datasets-postoperative pain scores, and hip-worn accelerometer activity counts. We believe transition icons can be an important tool for researchers approaching time-series data, as they give rich and intuitive information about collective time-series behaviors.

The past decade has seen an explosion in the amount of digital information stored in electronic health records (EHRs). While primarily designed for archiving patient information and performing administrative healthcare tasks like billing, many researchers have found secondary use of these records for various clinical informatics applications. Over the same period, the machine learning community has seen widespread advances in the field of deep learning. In this review, we survey the current research on applying deep learning to clinical tasks based on EHR data, where we find a variety of deep learning techniques and frameworks being applied to several types of clinical applications including information extraction, representation learning, outcome prediction, phenotyping, and deidentification. We identify several limitations of current research involving topics such as model interpretability, data heterogeneity, and lack of universal benchmarks. We conclude by summarizing the state of the field and identifying avenues of future deep EHR research.

Electronic Health Records (EHR) are mainly designed to record relevant patient information during their stay in the hospital for administrative purposes. They additionally provide an efficient and inexpensive source of data for medical research, such as patient outcome prediction. In this study, we used preoperative Electronic Health Records to predict postoperative delirium. We compared the performance of seven machine learning models on delirium prediction: linear models, generalized additive models, random forests, support vector machine, neural networks, and extreme gradient boosting. Among the models evaluated in this study, random forests and generalized additive model outperformed the other models in terms of the overall performance metrics for prediction of delirium, particularly with respect to sensitivity. We found that age, alcohol or drug abuse, socioeconomic status, underlying medical issue, severity of medical problem, and attending surgeon can affect the risk of delirium.

Mobile health (mHealth) based on smartphone and smartwatch technology is changing the landscape for how patients and research participants communicate about their health in real time. Flexible control of the different interconnected and frequently communicating mobile devices can provide a rich set of health care applications that can adapt dynamically to their environment. In this paper, we propose a real-time online activity and mobility monitoring (ROAMM) framework consisting of a smart-watch application for data collection, a server for data storage and retrieval as well as online monitoring and administrative tasks. We evaluated this framework to collect actigraphy data on the wrist and used it for feature detection and classification of different tasks of daily living conducted by participants. The information retrieved from the smartwatches yielded high accuracy for sedentary behavior prediction (accuracy = 97.44%) and acceptable performance for activity intensity level estimation (rMSE = 0.67 and R2= 0.52).

Ambient Intelligence (AmI) is a new paradigm in information technology aimed at empowering people's capabilities by the means of digital environments that are sensitive, adaptive, and responsive to human needs, habits, gestures, and emotions. This futuristic vision of daily environment will enable innovative human-machine interactions characterized by pervasive, unobtrusive and anticipatory communications. Such innovative interaction paradigms make ambient intelligence technology a suitable candidate for developing various real life solutions, including in the health care domain. This survey will discuss the emergence of ambient intelligence (AmI) techniques in the health care domain, in order to provide the research community with the necessary background. We will examine the infrastructure and technology required for achieving the vision of ambient intelligence, such as smart environments and wearable medical devices. We will summarize of the state of the art artificial intelligence methodologies used for developing AmI system in the health care domain, including various learning techniques (for learning from user interaction), reasoning techniques (for reasoning about users' goals and intensions) and planning techniques (for planning activities and interactions). We will also discuss how AmI technology might support people affected by various physical or mental disabilities or chronic disease. Finally, we will point to some of the successful case studies in the area and we will look at the current and future challenges to draw upon the possible future research paths.

Activity recognition has received increasing attention from the machine learning community. Of particular interest is the ability to recognize activities in real time from streaming data, but this presents a number of challenges not faced by traditional offline approaches. Among these challenges is handling the large amount of data that does not belong to a predefined class. In this paper, we describe a method by which activity discovery can be used to identify behavioral patterns in observational data. Discovering patterns in the data that does not belong to a predefined class aids in understanding this data and segmenting it into learnable classes. We demonstrate that activity discovery not only sheds light on behavioral patterns, but it can also boost the performance of recognition algorithms. We introduce this partnership between activity discovery and online activity recognition in the context of the CASAS smart home project and validate our approach using CASAS data sets.

The increasing aging population in the coming decades will result in many complications for society and in particular for the healthcare system due to the shortage of healthcare professionals and healthcare facilities. To remedy this problem, researchers have pursued developing remote monitoring systems and assisted living technologies by utilizing recent advances in sensor and networking technology, as well as in the data mining and machine learning fields. In this article, we report on our fully automated approach for discovering and monitoring patterns of daily activities. Discovering and tracking patterns of daily activities can provide unprecedented opportunities for health monitoring and assisted living applications, especially for older adults and individuals with mental disabilities. Previous approaches usually rely on preselected activities or labeled data to track and monitor daily activities. In this article, we present a fully automated approach by discovering natural activity patterns and their variations in real-life data. We will show how our activity discovery component can be integrated with an activity recognition component to track and monitor various daily activity patterns. We also provide an activity visualization component to allow caregivers to visually observe and examine the activity patterns using a user-friendly interface. We validate our algorithms using real-life data obtained from two apartments during a three-month period.

In recent years, we have witnessed a rapid surge in assisted living technologies due to a rapidly aging society. The aging population, the increasing cost of formal health care, the caregiver burden, and the importance that the individuals place on living independently, all motivate development of innovative-assisted living technologies for safe and independent aging. In this survey, we will summarize the emergence of 'ambient-assisted living" (AAL) tools for older adults based on ambient intelligence paradigm. We will summarize the state-of-the-art AAL technologies, tools, and techniques, and we will look at current and future challenges.

Ubiquitous computing aims to enable and support anywhere, anytime, context-aware applications. Sensing, interpretation and integration of events, behaviors and environmental states have been keys to the success of such ubiquitous systems. Over the past two decades, there has been a constant shift of sensor observation modeling, representation, interpretation and usage, namely from low-level raw observation data and their direct/hardwired usage, data aggregation and fusion, to high-level formal context modeling and context-based computing. It is envisioned that this trend will continue towards a further higher level of abstraction, allowing situation, activity and goal modeling, representation and inference, thus realizing the vision of ubiquitous computing. The proposed "mini-track" workshop intends to bring together researchers and practitioners from relevant fields to present and disseminate the latest accomplished and/or ongoing research on Situation, Activity and Situation Awareness (SAGAware) and their novel application in ubiquitous computing. It aims to facilitate knowledge transfer and synergy, bridge gaps between different research communities/groups, lay down foundation for common purposes, and help identify opportunities and challenges for interested researchers and technology and system developers.

This report summarizes the International Workshop on Situation, Activity and Goal Awareness held at the 13th ACM International Conference on Ubiquitous Computing, on September 18 in Beijing, China.

Active learning methods are used to improve the classification accuracy when little labeled data is available. Most traditional active learning methods pose a very specific query to the oracle, i.e. they ask for the label of an unlabeled example. This paper proposes a novel active learning method called RIQY (Rule Induced active learning QuerY). It can construct generic active learning queries based on rule induction from multiple unlabeled instances. These queries are shorter and more readable for the oracle and encompass many similar cases. Also the learning algorithm can achieve higher accuracy rates by asking fewer queries. We evaluate our algorithm on 12 different real datasets. Our results show that we can achieve higher accuracy rates using fewer queries compared to the traditional active learning methods.

The machine learning and pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of smart home residents, we need to design technologies that recognize and track activities that people normally perform as part of their daily routines. Although approaches do exist for recognizing activities, the approaches are applied to activities that have been pre-selected and for which labeled training data is available. In contrast, we introduce an automated approach to activity tracking that identifies frequent activities that naturally occur in an individual's routine. With this capability we can then track the occurrence of regular activities to monitor functional health and to detect changes in an individual's patterns and lifestyle. In this paper we describe our activity mining and tracking approach and validate our algorithms on data collected in physical smart environments.

In recent years, new emerging application domains have introduced new constraints and methods in data mining field. One of such application domains is activity discovery from sensor data. Activity discovery and recognition plays an important role in a wide range of applications from assisted living to security and surveillance. Most of the current approaches for activity discovery assume a static model of the activities and ignore the problem of mining and discovering activities from a data stream over time. Inspired by the unique requirements of activity discovery application domain, in this paper we propose a new stream mining method for finding sequential patterns over time from streaming non-transaction data using multiple time granularities. Our algorithm is able to find sequential patterns, even if the patterns exhibit discontinuities (interruptions) or variations in the sequence order. Our algorithm also addresses the problem of dealing with rare events across space and over time. We validate the results of our algorithms using data collected from two different smart apartments.

Most commonly-used techniques in smart environments such as ADL recognition are designed and tested for a specific space and a specific person; therefore learning in each environmental situation is treated as a separate context. In this paper, we try to develop a method for recognizing and transferring learned knowledge of activities between different residents. Our method is able to map activities despite intra-subject variability and inter-subject variability, by using a discontinuous mining method and a similarity measurement method. At the end, we will provide the results of our experiments on real data obtained from a smart apartment.

Advancements in supporting fields have increased the likelihood that smart-home technologies will become part of our everyday environments. However, many of these technologies are brittle and do not adapt to the user's explicit or implicit wishes. Here, we introduce CASAS, an adaptive smart-home system that utilizes machine learning techniques to discover patterns in resident's daily activities and to generate automation polices that mimic these patterns. Our approach does not make any assumptions about the activity structure or other underlying model parameters but leaves it completely to our algorithms to discover the smart-home resident's patterns. Another important aspect of CASAS is that it can adapt to changes in the discovered patterns based on the resident implicit and explicit feedback and can automatically update its model to reflect the changes. In this paper, we provide a description of the CASAS technologies and the results of experiments performed on both synthetic and real-world data.