Generative modeling is a powerful technique that involves creating machine learning models capable of creating new data similar to the data it was trained on. Generative Adversarial Networks (GANs) are a leading approach for generative modeling. However, GAN training is known to be a notoriously difficult task. GAN convergence issues are largely caused by the supports of the real and generated distributions being disjoint. To tackle this open problem, we propose a novel GAN pre-training process that effectively aligns the supports of the generated and real data prior to applying traditional adversarial GAN training. The key component of our method, called AlignGAN, is learning a mapping between the input data distribution and a latent representation defined over a hypersphere, regularized by a One Class Classifier. This successfully encourages the generator to produce samples throughout the support of the real data, while not generating samples outside the support. We maintain support alignment through low-bandwidth noise convolutions and additional One Class regularization, leading to continued stable GAN training. We validate our approach against leading stabilization methods on three benchmark datasets, showing AlignGAN routinely produces the best results.

Nearly 1 million people in the USA and 10 million people worldwide are living with Parkinson’s disease (PD) [1]. The progression of the disease can be inferred from the changes in the patients’ gait to inform early intervention. Due to their near-ubiquitous ownership, smartphone sensing is a very effective, highly available way of assessing gait. Objectives: In this work, we explore the inference of PD gait, by analyzing the patient’s walk data collected from a smartphone in the patients’ homes, using Deep Learning (DL) methods. Specifically, we investigate DL models to classify the severity of the motor part of the UPDRS (MDS-UPDRS 2.10-2.13). Methodology: Using data from the mPower study [2], we selected 152 PD patients who recorded at least 3 walks before taking their medication. We extracted gait cycles from the accelerometer and gyroscope sensor records and fed them to DeePaGait; our DL multi-layer Conventional Neural Network (CNN). DeePaGait operates on 1Dimensional convolution filters to classify 30 seconds of walking data into one of five severity levels (Normal, Slight, Mild, Moderate, Severe). Results: Our DeePaGait DL network was able to Classify the PD Walking-Balance, Shaking/Tremor, and Freeze of Gait (FoG) symptoms, with an accuracy of: 99.1%,98.4%, and 98.2% respectively. Conclusion: Despite the challenges of working with a patient-labeled, crowdsourced dataset, we were able to demonstrate that gait classification based on smartphone sensor data is a viable approach to infer the severity of PD gait anomalies.

Cardiovascular diseases (CVDs) are a major and growing contributor to global mortality. The Ejection Fraction (EF) of a heart, defined as the ratio of the a mount of blood pumped out of the heart-ventricle to the volume of blood entering the ventricle at the start of each heartbeat, is an important parameter for assessing cardiac function. An echocardiogram, an ultrasound of the heart that generates videos of the beating heart, is widely used to assess cardiac function and diagnose CVDs. Interpretation of echocardiograms is currently done by experts, who are not always available in many low resource settings and can be inconsistent and make errors. Video Action Recognition (VAR) Neural Networks have recently been proposed to recognize the actions being performed by humans in videos. In this paper, we innovatively explore using state-of-the-art VAR neural networks for the binary classification of echocardiograms into healthy and unhealthy EF classes. The Gate Shift Network (GSN) with BNInception as its backbone outperformed all other baseline VAR architectures, achieving an accuracy of 90.17% and a specificity of 7 7.01% with an inference time as low a s 25.11 seconds. Other baseline VAR models explored included R(2+1)D, Temporal Pyramid Network (TPN), Temporal Adaptive Network (TANet), and Timesformer.

Human activity recognition (HAR) is the process of using mobile sensor data to determine the physical activities performed by individuals. HAR is the backbone of many mobile healthcare applications, such as passive health monitoring systems, early diagnosing systems, and fall detection systems. Effective HAR models rely on deep learning architectures and big data in order to accurately classify activities. Unfortunately, HAR datasets are expensive to collect, are often mislabeled, and have large class imbalances. State-of-the-art approaches to address these challenges utilize Generative Adversarial Networks (GANs) for generating additional synthetic data along with their labels. Problematically, these HAR GANs only synthesize continuous features — features that are represented by real numbers — recorded from gyroscopes, accelerometers, and other sensors that produce continuous data. This is limiting since mobile sensor data commonly has discrete features that provide additional context such as device location and the time-of-day, which have been shown to substantially improve HAR classification. Hence, we studied Conditional Tabular Generative Adversarial Networks (CTGANs) for data generation to synthesize mobile sensor data containing both continuous and discrete features, a task never been done by state-of-the-art approaches. We show HAR-CTGANs generate data with greater realism resulting in allowing better downstream performance in HAR models, and when state-of-the-art models were modified with HAR-CTGAN characteristics, downstream performance also improves.

Recurrent Classifier Chains (RCCs) are a leading approach for multi-label classification as they directly model the interdependencies between classes. Unfortunately, existing RCCs assume that every training instance is completely labeled with all its ground truth classes. In practice often only a subset of an instance's labels are annotated, while the annotations for other classes aremissing. RCCs fail in this missing label scenario, predicting many false negatives and potentially missing important classes. In this work, we propose Robust-RCC, the first strategy for tackling this open problem of RCCs failing formulti-label missing-label data. Robust-RCC is a new type of deep recurrent classifier chain empowered to model inter-class relationships essential for predicting thecomplete label set most likely to match the ground truth. The key to Robust-RCC is the design of the Multi Incomplete Label Risk (MILR) function, which we prove to be equal in expectation to the true risk of the ground truth full label set despite being computed from incompletely labeled data. Our experimental study demonstrates that Robust-RCC consistently beats six state-of-of-the-art methods by as much as 30% in predicting the true labels.

In important domains from video stream analytics to human context recognition, datasets are only partially-labeled. Worse yet, the labels are often applied sequentially, as annotators choose labels frame-by-frame or timestep-by-timestep in sequence. With labels not collected independently, this results in sequential bias in the labeling. Unfortunately, current state-of-the-art methods for partially labeled data are rendered ineffective under sequential bias. In this work, we propose a novel solution to tackling this open sequential bias problem, called DeepSPU. DeepSPU recovers missing labels by constructing a model of the sequentially biased labeling process itself. This labeling model is then learned jointly with the prediction model that infers the missing labels in an iterative training process. Further, we regulate this training using a theoretically-justified cost functions that prevent our model from converging to incorrect but low-cost solution. Our experimental studies demonstrate that DeepSPU consistently outperforms the state-of-the-art methods by over 10% on a rich variety of real-world datasets.

Open set recognition models address the real-world scenario where classes of data unobserved during training are encountered in testing after deployment. Closed set classifiers wrongly attempt to classify instances from an unknown class as belonging to one of the known classes from the training set, which reduces the model’s accuracy. Ideally, these unknown instances should be recognized as such, while known instances should continue to be accurately classified. Unfortunately, state-of-the-art open set methods solve this problem by making restrictive assumptions on the variance and/or boundedness of the distributions of known classes. In this paper, we propose a novel method, Variational Open-Set Recognition (VOSR) that eliminates these assumptions. VOSR incorporates a closed set classifier, an unknown detector, and a novel Structured Gaussian Mixture Variational Autoencoder (SGM-VAE) that guarantees separable class distributions with known variances in its la-tent space. Further, by encouraging a large distance between class-specific distributions, VOSR increases the likelihood that instances from unknown classes lie in low-probability regions and thus are more readily identifiable. In rigorous evaluation, we demonstrate that VOSR outperforms state-of-the-art open set classifiers with up to a 14% F1 score increase in identifying instances from unknown classes in multiple image classification and human activity recognition datasets.

COVID-19 has infected millions since November 2019. The virus spreads through close contact with those who are infected. People are often unaware of or lose track of their behaviors, which increase their risk of infection. Passive methods to continuously monitor and track dangerous user behaviors, maintain and update a COVID risk score can enable high risk users to take preventive measures early. At the organization or institution level, such systems can provide insights on organization-wide patterns that exacerbate disease spread. This paper presents our vision of pervasive, continuous infectious disease contact tracing, risky behavior tracking and continuous risk score calculation from contact, place visit information and reported behaviors. As a specific example, we describe the research, design and development of the android app Goatvid Trace that continuously gathers smartphone sensor data, using it to calculcate smartphone users’ risk of exposure to COVID-19. Machine Learning methods for proximity detection from smart- phone Bluetooth RSSI signals are also described. GoatVid trace was deployed and evaluated on a small university community. Our evaluation study found that the mean COVID-19 risk score of college students in our study was 25.6%. Our risk score model correlated well with subject questionnaire responses with an R2of 0.6166. The machine learning models for proximity detection estimated distances between two phones with a Cross Validation RMSE of 1.58766.

Human activity recognition (HAR), the task of predicting the activities performed by an individual using mobile sensor data, is an active and important area of research. Unfortunately, it is very costly to collect the data required to train robust HAR classifiers. To tackle this issue, there has been an increasing focus on generating synthetic HAR data for augmentation purposes. The state-of-the-art generative HAR approaches utilize Generative Adversarial Networks (GANs) to produce realistic synthetic HAR data. However, these solutions can not generate personalized data that matches the behavior of particular users, limiting their potential use cases. This is particularly problematic in the mobile health domain, where the target users are often elderly or disabled and are thus likely to have activity signals that are unique from the general population. To overcome this drawback, we propose a novel controllable GAN solution Control-HAR-GAN. Our approach learns user and activity signals independently, and when generating synthetic instances, practitioners can specify both the activity to be generated as well as the user the data should match. This has the added benefit in that our model supports novel user-activity pairs by generating examples that match the data that would have been recorded by a particular user if they had performed the target activity, even if the user never performed that activity during the data collection process. We show that our model outperforms the existing HAR GAN approach in generating observed user-activity pairs by up to 10%. Additionally, our approach can also perform the task of novel user-activity pair generation, which is impossible for existing approaches.

Transfer learning has been widely used in natural language processing through deep pretrained language models, such as Bidirectional Encoder Representations from Transformers and Universal Sentence Encoder. Despite the great success, language models get overfitted when applied to small datasets and are prone to forgetting when fine-tuned with a classifier. To remedy this problem of forgetting in transferring deep pretrained language models from one domain to another domain, existing efforts explore fine-tuning methods to forget less. We propose DeepEmotex an effective sequential transfer learning method to detect emotion in text. To avoid forgetting problem, the fine-tuning step is instrumented by a large amount of emotion-labeled data collected from Twitter.We conduct an experimental study using both curated Twitter data sets and benchmark data sets. DeepEmotex models achieve over 91% accuracy for multi-class emotion classification on test dataset. We evaluate the performance of the fine-tuned Deep-Emotex models in classifying emotion in EmoInt and Stimulus benchmark datasets. The models correctly classify emotion in 73% of the instances in the benchmark datasets. The proposed DeepEmotex-BERT model outperforms Bi-LSTM result on the benchmark datasets by 23%. We also study the effect of the size of the fine-tuning dataset on the accuracy of our models. Our evaluation results show that fine-tuning with a large set of emotion-labeled data improves both the robustness and effectiveness of the resulting target task model.

TBI causes distress to millions of individuals and can lead to significant motor, cognitive and emotional deficits. However, TBI patients are currently assessed infrequently especially in-between scheduled appointments. To facilitate passive, remote, population-level ailment monitoring, we propose TBI2Vec, a TBI sensing framework that continuously assesses smartphone users by using machine learning to classify smart-phone sensor features encoded as autoencoder embeddings. Passive smartphone sensing of TBI enables a small medical team to monitor a large population of patients passively and detect TBI early. In analyzing a large, real, crowd-sourced smartphone sensor TBI dataset, we extracted 106 statistical features from raw smartphone sensor data, from which we generated autoencoder embeddings that are then classified to distinguish TBI subjects from non-TBI subjects. In rigorous evaluations, we found that classifying the features encoded using embeddings outperformed the same models without embeddings. The dimension reduction process of generating autoencoder embeddings retained the most discriminative in-formation while eliminating non-discriminative ones, boosting classification both accuracy and generalizability. Even for our imbalanced dataset, using embeddings performed better than baseline models in classifying the minority class. In rigorous evaluation, using embeddings increased the F-beta(0.5) by 34-71%, decreased the False Negative Rate (FNR) by 20-100% and significantly reduced the cross-fold variation of accuracies achieved during k-fold cross validation. TBI2Vec was able to detect TBI occurrence as early as 24 hours after injury. Random Forest performed the best and a window size of 2 days 12 hours gave the best results. TBI2Vec achieved an F-beta (0.5) score of 83.0% with a True Negative Rate (TNR) of 96.0% and False Negative Rate of 33.0% by correctly identifying TBI subjects.

Stress has become a significant public health problem worldwide. Prior work has proposed methods for inferring stress from smartphone sensed data. However, in many cases subjects are reluctant to participate and annotate data in mobile sensing studies. In this work, we explored a transfer learning approach to mitigate this issue. We investigated whether a smartphone-based stress model learned from Dartmouth college students could be transferred and used to accurately predict the stress of students at Worcester Polytechnic Institute (WPI). While these two colleges have similar size, differences exist including the length of their academic terms. To validate our approach, B.stress, a smartphone app was developed, which used the transferred model to infer the stress levels of WPI subjects. To validate this approach, the students were prompted to agree or disagree with the transferred model's inference, which was compared to a stress model trained from scratch for WPI students. Our results were encouraging, demonstrating that transfer learning can quickly and autonomously achieve a stress inference accuracy of 78.9% for all stress levels. In comparison, a model learned from users directly from scratch is personalizable and achieves an accuracy of 87% as the classifier learns a specific model for each student. However, training the model from scratch without transferring learning requires weeks of manual labeling beforehand. These results are encouraging and suggest that a hybrid approach might be an optimal middle ground. A transfer learning could be used to generate an initial stress sensing model, which is then fine-tuned using feedback from the user.

In big data applications from digital health to assisted living smart systems, only a fraction of data instances used for training classifiers t end to be labeled. One important subfield of weakly labeled learning, called Positive Unlabeled (PU) learning, does not require a completely labeled dataset in order to train a strong classifier. This is crucial as in many domains it is expensive or impossible to obtain a completely labeled dataset. While prior PU work assumed that unlabeled instances occurred with a random uniform distribution, we observe that labeled (and unlabeled) data tends to occur in long contiguous sequences (or bursts) due the prevalent burst labeling behavior by human annotators. Burst labeling leads to a sequential bias in PU data not addressed by state-of-the-art methods. To tackle this open problem of learning under sequential bias, we propose BurstPU, the first framework for training a classifier on sequentially labeled PU data. BurstPU addresses the challenge that two interdependent models must be learned, namely, the classification model and the labeling likelihood model, with the later predicting the likelihood that a given instance is labeled. The labeling likelihood model is then needed during the training of the classification model to account for the bias in the labeling process. Our experimental study demonstrates that BurstPU consistently outperforms all state-of-the-art PU methods on a rich variety of diverse real-world datasets, and can learn from fewer labeled instances compared to state-of-art PU methods.

Current research in passive health monitoring utilizes machine learning methods to infer users' symptoms and health status from smartphone-sensed data, which can be gathered on a large scale. However, reasoning about smartphone-sensed health behaviors should engage health experts who may not want to solely rely on computational approaches as they provide limited insights. We designed and proposed a visualization framework for the INTeractive Observation of Smartphone-Inferred Symptoms (INTOSIS), that supports contextualization of symptomatic days by presenting a holistic picture of complex smartphone data for analysts to find c oncerning b ehavior patterns. For instance, while sedentary behavior caused by the flu is concerning, sedentary behaviors on holidays are non-concerning. INTOSIS visualizes multiple smartphone sensor data channels such as geo-location, app usage and screen usage. It uses visual metaphors to effectively represent the data to help analysts derive important human-understandable spatio-temporal contexts and assign health consequences with corresponding semantic labels. INTOSIS provides timeline visualizations of contextual clues such as screen and app usage at night that support an analyst in being able to reason about and then extract plausible explanations for the occurrence of certain symptoms, such as, sleep problems. We validate INTOSIS with intuitive use cases, using a real-world smartphone-sensed dataset, along with expert evaluation.

Infectious diseases epidemics such as the current COVID-19 pandemic have an immense impact on all facets of life. Consequently, the current dearth of effective and timely public health surveillance methods, especially at the individual level, have been accentuated, prompting research into supplementary methods. Sensor-rich, ubiquitously owned smartphones can now gather large volumes of data that has been utilized for passive and continuous physical and mental health assessment. In this paper, we propose a Deep learning based Smartphone Early Ailment Sensing (DeepSEAS) framework that predicts a smart-phone user's future manifestation of influenza-like biological symptoms (e.g. coughing and sneezing) a day early while they are still asymptomatic. DeepSEAS works by analyzing a subject's historical one-day smartphone sensor and mobility data. First, we utilize the mean shift clustering algorithm to create clusters of users with similar social and behavioral traits such as their socialization levels, social media presence, eating and working out habits. Then, DeepSEAS employs an end-to-end trainable LSTM Autoencoder (LSTM AE) coupled with a Feed Forward Neural network classifier, a chieving a sensitivity of 7 8% i n correctly identifying users who will manifest biological symptoms a day later. DeepSEAS facilitates up-to-date influenza s urveillance at the individual level, which could transform the current healthcare system. Early detection can enable asymptomatic users to be alerted, notified and isolated, which could reduce disease transmission.