For measuring human anxiety Competitive State Anxiety Inventory (CSAI) is a very popular method. Though this method is widely used in determining anxiety in the sports area, recently, this method is also deployed in other activities. Our objective was to determine the CSAI-based assessment non-invasively using HR-HRV for three different tasks. A baseline video and a target video of the same subject are used and the outputs are the competitive state anxiety inventor are proposed in this study. The framework uses a support vector regression, based on the abstract features derived from the deep learning, heart rate and heart rate variability features from the remote photoplethysmography signal extracted from the facial videos. Deep learning could provide good estimate results. The results proved the rPPG features could improve the accuracy of the cognitive anxiety and somatic anxiety prediction. The competitive State Anxiety Inventory (CSAI) can be assessed by HR-HRV analysis from the facial video. The machine learning algorithm can predict human anxiety in CSAI units despite sports activities. The result from our framework (StressFusion) shows a strong correlation between the ground truth (from UBFC-Phys dataset) and predicted anxiety scores.

Heart Rate (HR) and Heart Rate Variability (HRV) is an essential measurement to know the heart’s cardiovascular condition. Many works have been done for measuring HR-HRV based on the facial video non-invasively. In this paper, based on our previous work experience of measuring HR-HRV by Remote photoplethysmography signals (rPPG) analysis, we have built a prediction model from the 10-second time series data extracted from a facial video. In this work, we have used the instantaneous public dataset with several data models to predict the HR-HRV, and stress levels exclusively from the dataset. We have used here some of the popular algorithms appropriate for this task. We have also analyzed the stress level classification on the gender of a subject using the same facial videos with 16 different classifiers resulting in almost perfect accuracy for several classifiers.

Stress has a significant impact on the physical and mental health of an individual and is a growing concern for society. Facial video-based stress evaluation from non-invasive cameras has proven to be a more efficient method to evaluate stress in comparison to approaches that use questionnaires or wearable sensors. Plenty of classification models have been built for stress detection. However, most do not consider individual differences. Also, the results for such models are limited by a uni-dimensional definition of stress levels lacking a comprehensive quantitative definition of stress. We propose a framework that utilizes the multilevel video frame representations from deep learning that utilizes a baseline video and a target video of the same subject. The framework could output the quantitative stress score in multiple dimensions. We compared the assessment performance of different deep learning architectures based on the UBFC- Phys dataset. The results demonstrate the feasibility and effectiveness of using deep learning to capture the personalized stress features.

Time-Sync Comments (TSCs) are a sequence of comments associated with video contents at each timestamp. By applying textural analysis, researchers can transform the TSCs into labels that represent the semantic meaning of the original video content. Multiple studies have used the TSCs in video segmentation and tagging. TSCs can be either created by a single user or generated by various users. Thanks to the exploding of multimedia platforms, online comments have proved to be an efficient TSCs data source in multiple research since 2014. However, previous TSCs studies mainly focused on data sources targeting young non-English speaking audiences, potentially introducing data bias due to limited geographic regions and groups. This paper aims to solve this problem by proposing a universal data collection framework of TSCs generated by audiences worldwide from popular social media platforms. We first introduced an efficient data mining strategy for gathering such TSCs data in general. Then, we demonstrated how to build the autonomous pipeline and collected two large-scale TSCs datasets with different sets of keywords, namely LST-YF20 and LST-YT1000, directly from YouTube Lives. We also conducted an extensive experiment on the efficiency of our data pipeline with a group of fixed keywords. The result of our investigation suggests that our data pipeline could efficiently produce high-quality TSCs datasets while keeping a constrained budget. We believe our framework could further contribute to future research in the multimedia field.

Accurate and valid health information is crucial for effective medical management. Failure to collect adequate information from physical and mental health examinations can be a barrier to Virtual medical platforms and telemedicine. In this paper, we propose the non-invasive “Dream” project prototype to monitor and record heart rate (HR), heart rate-variation (HRV) (for physical health), and stress (for mental health) using only a smart-phone. This non-invasive mobile application, “Dream” uses the front camera to capture video to calculate HR-HRV and stress. The full “Dream” project encompasses our previous facial video HR-HRV and stress work. We have also compared our proposed “Dream” project with 39 works in this area. We found a significant positive difference between our proposed “Dream” project compared to other projects in respect to user accessibility, application, cost-effectiveness, hospitalization monitoring, and human health status. Furthermore, we can apply “Dream” in remote human health monitoring, driver monitoring, and creating vital sign records non-invasively without a health care assistant. Especially during a pandemic, this virtual health monitoring system can be useful for scaled-up telemedicine to serve the remote population.

This paper describes a cloud-based heart rate (HR) and heart rate variability (HRV) monitor, which can monitor an individual's heart rate and heart rate variability using minimal resources at the user's end. This HR and HRV monitor does not involve any contact sensors or costly medical equipment, and it does not require a high-definition camera to video record the user's face. This HR and HRV monitoring system simply requires the webcam of a personal computer or the camera of a mobile device such as a cell phone or tablet in order to video record the face of the user. The system utilizes the resources of the user's device while recording and encoding the video segments before transferring them to a server. All the processes to calculate the HR and HRV occur at the server.