We propose a novel explainable machine learning (ML) model that identifies depression from speech, by modeling the temporal dependencies across utterances and utilizing the spectrotemporal information at the vowel level. Our method first models the variable-length utterances at the local-level into a fixed-size vowel-based embedding using a convolutional neural network with a spatial pyramid pooling layer (vowel CNN). Following that, the depression is classified at the global-level from a group of vowel CNN embeddings that serve as the input to another 1D CNN (depression CNN). Different data augmentation methods are designed for both the training of vowel CNN and depression CNN. We investigate the performance of the proposed system at various temporal granularities when modeling short, medium, and long analysis windows, corresponding to 10, 21, and 42 utterances, respectively. The proposed method reaches comparable performance with previous state-of-the-art approaches and depicts explainable properties with respect to the depression outcome. The findings from this work may benefit clinicians by providing additional intuitions during joint human-ML decision-making tasks.

Speech-based machine learning (ML) has been heralded as a promising solution for tracking prosodic and spectrotemporal patterns in real-life that are indicative of emotional changes, providing a valuable window into one's cognitive and mental state. Yet, the scarcity of labelled data in ambulatory studies prevents the reliable training of ML models, which usually rely on “data-hungry” distribution-based learning. Leveraging the abundance of labelled speech data from acted emotions, this paper proposes a few-shot learning approach for automatically recognizing emotion in spontaneous speech from a small number of labelled samples. Few-shot learning is implemented via a metric learning approach through a siamese neural network, which models the relative distance between samples rather than relying on learning absolute patterns of the corresponding distributions of each emotion. Results indicate the feasibility of the proposed metric learning in recognizing emotions from spontaneous speech in four datasets, even with a small amount of labelled samples. They further demonstrate superior performance of the proposed metric learning compared to commonly used adaptation methods, including network fine-tuning and adversarial learning. Findings from this work provide a foundation for the ambulatory tracking of human emotion in spontaneous speech contributing to the real-life assessment of mental health degradation.

Public speaking is a vital skill for making good impressions, effectively exchanging ideas, and influencing others. Yet, public speaking anxiety (PSA) ranks as a top social phobia. Recent advancements in wearable devices and ubiquitous virtual reality (VR) interfaces can help measure and mitigate PSA. This research quantifies PSA through bio-behavioral markers related to individuals’ physiological and acoustic characteristics. The effect of virtual reality (VR) training on alleviating PSA is measured through self-reported and bio-behavioral indices. Psychological (e.g., general trait anxiety, personality) and demographic (e.g., age, gender, highest education, native language) traits are examined as moderating factors between bio-behavioral indices and PSA, as well as moderating factors for measuring the VR effectiveness in mitigating PSA. These measures are also used as clustering criteria for stratifying participants in group-based models of PSA. Results indicate the significance of such traits to modeling PSA with the proposed group-based models yielding Spearman’s correlation of 0.55 ($p<0.05$) between the actual and predicted outcome. Results further demonstrate that systematic exposure to public speaking in VR can alleviate PSA in terms of both self-reported ($p<0.05$) and physiological ($p<0.05$) indices. Findings from this study will enable researchers to better understand antecedents and causes of PSA and lay the foundation for personalized adaptive feedback for PSA interventions.

As interactions between individuals and services increase, requests are more frequent and policy sets are larger. The evaluation performance of PDP (Policy Decision Point) plays a key role in the operation of a system. In order to solve bottlenecks of improving the PDP evaluation performance for large-scale policy sets, we propose an evaluation engine based on artificial neural networks, namely ANNPDP. We transform rules in a large-scale policy set described in the XACML (eXtensible Access Control Markup Language) into numerical rules. Evaluation networks are established and trained by the numerical rules. In order to ensure the accuracy, a misjudgment set is constructed for error corrections and stored by hash indexes. By simulating the arrival of requests, ANNPDP is compared with the Sun PDP, HPEngine, XEngine, and SBA-XACML. The experiment results show that ANNPDP has: 1) high performance: if the number of requests reaches 10,000, the evaluation time of ANNPDP on the large-scale policy set with 100,000 rules is approximately 0.46, 0.93, 0.71, and 1.43 percent of that of the Sun PDP, HPEngine, XEngine, and SBA-XACML, respectively, and 2) stability: as the size of the large-scale policy set and the number of requests increase, the evaluation time of ANNPDP grows linearly. ANNPDP can satisfy the requirements of an authorization system with large-scale policy sets.

The international charity SolarAid aims to provide access to lights in areas without electricity, and it is a challenge to accurately and efficiently transmit the lights to the areas in need. Multisource, multitemporal, and multimodal remote sensing images can provide rich information about the target area, so using multisource remote sensing images for accurate detection of human settlements without electricity is a feasible solution. In this paper two separate detection tasks are formulated: building two attention SENet for settlements detection and light detection using the Sentinel-2 dataset and the Suomi Visible Infrared Imaging Radiometer Suite (VIIRS) night time dataset, respectively. In addition, we study a new outlier removal method based on the pixel distribution characteristics of the VIIRS dataset for data pre-processing, and propose a post-processing method based on region continuity for further correction of the results. Experiments show that our method can maximize the use of multisource data information and rank first in the detection of settlements without electricity challenge track (Track DSE) of the 2021 IEEE GRSS Data Fusion Contest.

The large amount of data captured by ambulatory sensing devices can afford us insights into longitudinal behavioral patterns, which can be linked to emotional, psychological, and cognitive outcomes. Yet, the sensitivity of behavioral data, which regularly involve speech signals and facial images, can cause strong privacy concerns, such as the leaking of the user identity. We examine the interplay between emotion-specific and user identity-specific information in image-based emotion recognition systems. We further study a user anonymization approach that preserves emotion-specific information, but eliminates user-dependent information from the convolutional kernel of convolutional neural networks (CNN), therefore reducing user re-identification risks. We formulate an adversarial learning problem implemented with a multitask CNN, that minimizes emotion classification and maximizes user identification loss. The proposed system is evaluated on three datasets achieving moderate to high emotion recognition and poor user identity recognition performance. The resulting image transformation obtained by the convolutional layer is visually inspected, attesting to the efficacy of the proposed system in preserving emotion-specific information. Implications from this study can inform the design of privacy-aware emotion recognition systems that preserve facets of human behavior, while concealing the identity of the user, and can be used in ambulatory monitoring applications related to health, well-being, and education.

Public speaking is key to effectively exchanging ideas, persuading others, and making a tangible impact. Yet, public speaking anxiety (PSA) ranks as a top social phobia among many people. This paper leverages bio-behavioural indices captured from wearable devices and virtual reality (VR) interfaces to quantify PSA. The significance of individual-specific factors, such as general trait anxiety and personality, as well as contextual factors, such as age, gender, highest education, and native language, in moderating the association between bio-behavioral indices and PSA is further examined through group-based machine learning models. Results highlight the importance of including such factors for detecting PSA with the proposed group-based PSA models yielding Spearman's correlation of 0.55(p <; 0.05) between the actual and predicted state-based anxiety scores. This work further analyzes whether systematic exposure to public speaking tasks in the VR environment can help alleviate PSA. Results indicate that systematic exposure to public speaking in VR can alleviate PSA in terms of both self-reported (p <; 0.05) and physiological (p <; 0.05) indices. Findings of this study will enable researchers to better understand antedecedents and causes of PSA contributing to behavioral interventions using VR.