Stress has a significant impact on the physical and mental health of an individual and is a growing concern for society. Facial video-based stress evaluation from non-invasive cameras has proven to be a more efficient method to evaluate stress in comparison to approaches that use questionnaires or wearable sensors. Plenty of classification models have been built for stress detection. However, most do not consider individual differences. Also, the results for such models are limited by a uni-dimensional definition of stress levels lacking a comprehensive quantitative definition of stress. We propose a framework that utilizes the multilevel video frame representations from deep learning that utilizes a baseline video and a target video of the same subject. The framework could output the quantitative stress score in multiple dimensions. We compared the assessment performance of different deep learning architectures based on the UBFC- Phys dataset. The results demonstrate the feasibility and effectiveness of using deep learning to capture the personalized stress features.

Time-Sync Comments (TSCs) are a sequence of comments associated with video contents at each timestamp. By applying textural analysis, researchers can transform the TSCs into labels that represent the semantic meaning of the original video content. Multiple studies have used the TSCs in video segmentation and tagging. TSCs can be either created by a single user or generated by various users. Thanks to the exploding of multimedia platforms, online comments have proved to be an efficient TSCs data source in multiple research since 2014. However, previous TSCs studies mainly focused on data sources targeting young non-English speaking audiences, potentially introducing data bias due to limited geographic regions and groups. This paper aims to solve this problem by proposing a universal data collection framework of TSCs generated by audiences worldwide from popular social media platforms. We first introduced an efficient data mining strategy for gathering such TSCs data in general. Then, we demonstrated how to build the autonomous pipeline and collected two large-scale TSCs datasets with different sets of keywords, namely LST-YF20 and LST-YT1000, directly from YouTube Lives. We also conducted an extensive experiment on the efficiency of our data pipeline with a group of fixed keywords. The result of our investigation suggests that our data pipeline could efficiently produce high-quality TSCs datasets while keeping a constrained budget. We believe our framework could further contribute to future research in the multimedia field.

Generative Adversarial Networks (GANs) are among the most actively researched neural networks in today’s artificial intelligence research. Scientists across different domains, particularly in image processing, constantly utilize variants of GANs to conduct research. The topic has increasingly drawn attention and interest in recent years. Our survey paper reviews the current literature and applications of GANs from both qualitative and quantitative perspectives. This survey also summarizes the challenges and improvement techniques of training GANs. We hope this paper may help researchers interested in GANs and serve as an informative source for ongoing and future work in this field.

Our brain produces electrical signals when neurons communicate with each other. The process of extracting that information of rich electrical activity through noninvasive electrodes is called Electroencephalography (EEG). With the recent surge and advancement in machine learning technology and computational power, the information is further processed to extract patterns in order to derive knowledge. The synthesized knowledge can be used in any desired fashion, ranging from executing commands to analyzing patterns. In recent times, EEG has been a compelling candidate to directly interact with different interfaces just through our brain waves, without depending upon muscular movements and peripheral nerves. It also extends its applicability in various fields, such as psychology, robotics, and cybersecurity, and aiding in the treatment and rehabilitation of various neurological disorders. In this paper, we review what has been out by the end of the 2nd decade of the 21st century. It helps readers, novice to experts, understand two areas:(1) Latest developments in the brain-computer interface (BCI) field, and (2) Different devices and their properties used to execute those developments. The aim of this paper is to help promote advancement and further development in this area.

The human affective state is a product of complex biological processes and environmental stimuli. Situation aware systems aim at identifying the affective state of an individual using data from a gamut of connected devices. The bottle necks for such systems include continuous data streams, mobility of the data collection apparatus, device ubiquity and the device cost. While there is research done using physiological sensors that can overcome these challenges, their accuracy is often dismal and the results are not granular, i.e. the affective state is singular. In this paper we present results from an experiment that enabled us to generate models to identify an individuals affective state as a mixture of emotional states and their respective activation's. Secondly, we show that the affective state of an individual is actually a mixture of emotional states ( amusement, anger, neutral, sad, fear and disgust). During an experimental study, 85 participants were induced with specific emotions using audio-visual stimulus. Physiological data including heart rate, blood volume pressure (BVP), inter beat interval(IBI) and electrodermal activity(EDA) along with a self-report indicating the levels of 6 emotional states that include Amusement, Anger, Sad, Disgust, Fear and Neutral was recorded. Additionally, we recorded a self-reported score for Anxiety. The videos used to induce emotions were validated in a recently published study in Psychology. The data collected was used to create models that identify the dominant emotional state and the emotional spectrum (activation levels of all emotional states) for an individual. We create a map between the physiological data and the dominant emotional state and also between physiological data and the self-report scores. In addition, we identify often overlooked characteristics of human emotion such as variability in perception and overlap of emotional states and finally create a topological map of emotional states based on physiological data.

ASD (Autism Spectrum Disorder) is a physiological condition that inhibits individuals from functioning in society. Such individuals suffer from high anxiety levels. This leads to both verbal and non-verbal communicative impairments and impedes self-expression. While there is no cure, there are structured interventions that teach children with ASD to cope with anxiety and function in society. However, these interventions do not account for the variability within the population with ASD. As a consequence, certain interventions fail to make a meaningful impact and, in certain scenarios, prove to be detrimental to the mental health of the participant. While there are screening measures, such as surveys conducted to avoid such circumstances, their effectiveness in identifying prospective successful candidates is poor. In this paper we propose a data driven intervention screening method that would enable Autism clinics to screen individuals most likely to benefit from the intervention and, more importantly, identify individuals that would be negatively affected.

Computer Vision has become the poster child for Deep Learning. The image classification accuracy of convolutional neural nets on benchmark data sets has increased every year since their inception. This has been aided with advances in feature fusion. The increase in the availability of imagetext occurrence has lead to text augmented feature spaces that have lead to higher accuracy in in image classification tasks. However, these works are limited to instances where text is readily available. This study presents an approach to featurize text within natural images with the goal of augmenting image features for image classification tasks. Text extraction and featurization in natural images is a challenging task due to challenges in reliable text localization and OCR results, both being impeded by the variability in image text and errors in OCR. We overcome these challenges by implementing a novel bounding box concatenation algorithm and a novel feature boosting algorithm. The result is a pipeline that encodes an image into a text feature space. Classifiers trained on the text based feature space have comparable accuracy to the state of the art Convolutional Neural Nets (CNN's) while being significantly inexpensive computationally. Moreover, the augmentation of text features to image features generates a hybrid feature space with a higher information content for a classification problem when compared to a feature space comprised exclusively of image features. Thus, we see a rise in classification accuracy across all state of the art machine learning algorithms.

Scientific gait analysis through the Internet of Things (IoT) is able to provide an overall assessment of "observations of daily living". All existing biomechanical models for predicting injuries in the elderly mainly consider the gait related parameters. Their accuracy is limited because injuries due to falls are significantly affected by different gait events in the gait cycle. The objective of this study is to develop a biomechanical model for improving subject-specific prediction of when different gait cycle events will induce falls. For this research, we designed and implemented a smart-shoe with a Wi-Fi communication module to discreetly collect insole pressure data in common environment. To the best of our knowledge, we are the first to use the gait biomechanical model implemented in smartphones to identify abnormal gait patterns for risk prediction. The proposed system, Your Walk is My Command, can warn the user about their abnormal gait and possibly save them from a forthcoming injuries.