Emotion Recognition systems are typically trained to classify a given psychophysiological state into emotion categories. Current platforms for emotion ground-truth collection show limitations for real-world scenarios of long-duration content (e.g., > 10m), namely: 1) Real-time annotation tools are distracting and become exhausting in a longer video; 2) Perform retrospective annotation of the whole content in bulk (providing highly coarse annotations); or 3) Are performed by external experts (depending on the number of annotators and their subjective experience). We explore a novel approach, the EmotiphAI Annotator, that allows undisturbed content visualisation and simplifies the annotation process by using segmentation algorithms that select brief clips for emotional annotation retrospectively. We compare three methods for content segmentation based on physiological data (Electrodermal Activity (EDA), emotion-based), scene (time-based), and random (control) selection. The EmotiphAI Annotator attained a B+ System Usability Scale score and low-average mental workload as per the NASA Task Load Index (40%). The reliability of the self-report was analysed by the inter-rater agreement (STD0.3 to 0.8), where the method based on EDA obtained the overall best performance.

During group interactions, we react and modulate our emotions and behaviour to the group through phenomena including emotion contagion and physiological synchrony. Previous work on emotion recognition through video/image has shown that group context information improves the classification performance. However, when using physiological data, literature mostly focuses on intrapersonal models that leave-out group information, while interpersonal models are unexplored. This paper introduces a new interpersonal Weighted Group Synchrony approach, which relies on Electrodermal Activity (EDA) and Heart-Rate Variability (HRV). We perform an analysis of synchrony metrics applied across diverse data representations (EDA and HRV morphology and features, recurrence plot, spectrogram), to identify which metrics and modalities better characterise physiological synchrony for emotion recognition. We explored two datasets (AMIGOS and K-EmoCon), covering different group sizes (4 vs dyad) and group-based activities (video-watching vs conversation). The experimental results show that integrating group information improves arousal and valence classification, across all datasets, with the exception of K-EmoCon on valence. The proposed method was able to attain mean M-F1 of$\approx$72.15% arousal and 81.16% valence for AMIGOS, and M-F1 of$\approx$52.63% arousal, 65.09% valence for K-EmoCon, surpassing previous work results for K-EmoCon on arousal, and providing a new baseline on AMIGOS for long-videos.

Epilepsy is a neurological disease that affects about 50 million people worldwide. It is a disorder of the central nervous system, characterized by recurrent seizures that can have a massive impact in the physical and mental health of the people who suffer from it, as well as their loved ones. Long-term monitoring of epilepsy in uncontrolled environments is key to provide accurate characterization of the disease, and to create tools that improve the patients' lives. Although some wearable devices (particularly with motion and cardiac-based sensors) are quickly gaining ground as everyday use monitoring devices, in the scope of epilepsy, electroencephalography (EEG) remains the gold-standard. Therefore, it is of utmost importance to include this modality in ambulatory settings, leveraging its extensive presence in literature and available databases. Nevertheless, in long-term recordings, having information about the onset of the seizure is of utmost importance for effective analysis of the collected data. Hence, this work explores the use of a single-channel, nonintrusive, EEG configuration in automatic seizure detection, with the purpose of event annotation in long-term recordings. This is a key element to the creation of multimodal datasets that can be used in seizure detection and, eventually, prediction, as well as towards comprehensive multimodal epilepsy monitoring techniques. A seizure-specific Support Vector Machines (SVM) classifier was designed for labeling eight different types of seizure, using a limited-channel configuration (Fp1-Fp2). Our work uses the TUH EEG Seizure Corpus, for which encouraging results were achieved for tonic-clonic and myoclonic seizures, with sensitivities of 98.9% and 98.2%, as well as precisions of 100% and 99.8%, respectively.

This paper presents a real-time emotion-annotation tool using a personal mobile device. To this end, an application based on the Valence-Arousal model was developed, following two different approaches (”Two-step Sequential Annotation” and ”One-step Matrix Annotation”). The application was tested through, an experiment where users performed annotations with each version of the app and then filled a feedback questionnaire. This questionnaire contained statements used to assess the usability and mental workload. A total of 16 (9 female) participants aged between 21 and 34 years old engaged in this experiment. Overall the results were quite encouraging in both versions, taking into account that this is still a work in progress. The ”Single-step Matrix Annotation” was considered the preferred version.

Aiming a deeper understanding of human emotional states, we explore deep learning techniques for the analysis of physiological data. In this work, raw two-channel pre-frontal electroencephalography and photoplethysmography signals of 25 subjects were collected using EMOTAI's headband while watching commercials. Taking as input the raw data, convolutional neural networks were used to learn data representations and classify the acquired signals according to the Positive and Negative Affect Schedule. This approach achieved promising results, with average F1-scores of 76.6% for Positive Affect and 83.3% for Negative Affect. Interpretation of the learned data representation was attempted by computing correlation values between features extracted from the raw inputs and the final classification. The features with the most significant correlations were the alpha band power, and the asymmetry and phase synchronization indexes. The extracted features seem to match the ones learnt by the neural network, hence endorsing their validity for emotional studies.

A reliable detection of the R-peaks in an electrocardiogram (ECG) is a fundamental step for further heart rate variability (HRV) analysis, biometric recognition techniques and additional ECG waveform based analysis. In this paper, a novel real-time and low-complexity R-peak detection algorithm is presented for single lead ECG signals. The detection algorithm is divided in two stages. In the first pre-processing stage, the QRS complex is enhanced by taking the double derivative, squaring and moving window integration. In the second, the detection of the R-peak is achieved based on a finite state machine (FSM) approach. The detection threshold is dynamically adapted and follows an exponential decay after each detection, making it suitable for R-peak detection under fast heart rate (HR) and R-wave amplitude changes without additional search back. The proposed algorithm was evaluated in a private single lead ECG database acquired using a FieldWiz wearable device. The database comprises five recordings from four different subjects, recorded during dynamic conditions, namely: running, trail running and weightlifting. The raw ECG signals were annotated for the R-peak and the proposed method benchmarked against common QRS detectors. The combined acquisition setup and presented approach resulted in R-peak detection Sensitivity (Se) of 99.77% and Positive Predictive Value of (PPV) of 99.18%, comparable to state of the art real time QRS detectors. Due to its low computational complexity, this method can be implemented in embedded wearable systems, suited for cardiovascular tracking devices in dynamic use cases and R-peak detection.

The aim of this study is to develop an automatic detector of the cyclic alternating pattern by first detecting the activation phases (A phases) of this pattern, analysing the electroencephalogram during sleep, and then applying a finite state machine to implement the final classification. A public database was used to test the algorithms and a total of eleven features were analysed. Sequential feature selection was employed to select the most relevant features and a post processing procedure was used for further improvement of the classification. The classification of the A phases was produced using linear discriminant analysis and the average accuracy, sensitivity and specificity was, respectively, 75%, 78% and 74%. The cyclic alternating pattern detection accuracy was 75%. When comparing with the state of the art, the proposed method achieved the highest sensitivity but a lower accuracy since the fallowed approach was to keep the REM periods, contrary to the method that is used in the majority of the state of the art publications which leads to an increase in the overall performance. However, the approach of this work is more suitable for automatic system implementation since no alteration of the EEG data is needed.

In this work, we present a context-aware ensemble fusion framework based on soft-biometric features, for long term person re-identification (Re-ID) in wild surveillance scenarios. The characteristics of a person that best correlate to its identity depend strongly on the view point. For instance, a person with a short stride gait is better perceived from a lateral view, whereas a person with a large chest is more distinct from a frontal view. Thus we associate context to the viewing direction of walking people in a surveillance scenario and choose the best features for each case. Using the MS KinectTM sensor v.2, we collect data from walking subjects and extract associated anthropometric and gait features. Each context is analysed with a Feature selection technique (Sequential Forward Selection) so that only the most relevant features for the context are retained. Then, individual context-specific classifiers are trained leveraging those selected features. Finally, we propose a contextaware ensemble fusion strategy, which we term as 'Contextspecific score-level fusion', based on the adaptive weighted sum of the results of individual classifiers. The proposed contextaware Re-ID framework demonstrate significant performance improvement both in terms of speed (up to 4.5 times faster) and accuracy (up to 17% rank-1 Re-ID rate) compared to the Context-unaware systems. From the study, we show that gait features are better for lateral views and anthropometric features are better for frontal views, confirming the results of previous studies.

The unprecedented collection and storage of data in electronic format has given rise to an interest in automated analysis for generation of knowledge and new insights. Cluster analysis is a good candidate since it makes as few assumptions about the data as possible. A vast body of work on clustering methods exist, yet, typically, no single method is able to respond to the specificities of all kinds of data. Evidence Accumulation Clustering (EAC) is a robust state of the art ensemble algorithm that has shown good results. However, this robustness comes with higher computational cost. Currently, its application is slow or restricted to small datasets. The objective of the present work is to scale EAC, allowing its applicability to big datasets, with technology available at a typical workstation. Three approaches for different parts of EAC are presented: a parallel GPU K-Means implementation, a novel strategy to build a sparse CSR matrix specialized to EAC and Single-Link based on Minimum Spanning Trees using an external memory sorting algorithm. Combining these approaches, the application of EAC to much larger datasets than before was accomplished.

All in-text\treferences\tunderlined\tin\tblue\tare\tlinked\tto\tpublications\ton\tResearchGate, letting you\taccess\tand\tread\tthem\timmediately.

Alzheimer's disease (AD) is one of the most common types of dementia that affects elderly people, with no known cure. Early diagnosis of this disease is very important to improve patients' life quality and slow down the disease progression. Over the years, researchers have been proposing several techniques to analyze brain images, like FDG-PET, to automatically find changes in the brain activity. This paper compares regions of voxels identified by an expert with regions of voxels found automatically, in terms of corresponding classification accuracies based on three well-known classifiers. The automatic identification of regions is made by segmenting FDG-PET images, and extracting features that represent each of those regions. Experimental results show that the regions found automatically are very discriminative, outperforming results with expert's defined regions.

Alzheimer’s disease accounts for an estimated 60% to 80% of cases of dementia. Recently, several computer-aided diagnosis systems have been developed, based on extracting information from FDG-PET scans. 3-dimensional FDG-PET images, under a voxel-as-feature approach, lead to high-dimensional features spaces which results in system performance problems. We propose a new approach for feature extraction of 3-dimensional images to improve the performance of a diagnosis system, and compare it with Gaussian pyramid technique. To evaluate the performance of our approach we applied it to a data base obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Experimental results have shown that the proposed approach is a good option for image feature extraction, outperforming the Gaussian pyramid technique.

Constrained data clustering algorithms allow the incorporation of a priori knowledge for specific problems into the clustering task in the form of constraints. The quality of the constraints have great impact in the performance of the constrained clustering algorithms. Therefore, special care must be taken while building the sets of constraints. In order to take the maximum advantage of the constrained clustering algorithms, these constraints must be highly informative and non-redundant. We propose two constraint acquisition methods based on user-feedback. The first method searches for non-redundant intra-cluster and inter-cluster query-candidates supported by information contained in an initial partition of the data set, ranks the candidates by decreasing order of interest and, finally, prompts the user the most relevant query-candidates. The constraints may optionally be used for learning a new data representation, which may enhance the performance of clustering. The second method iterates between using the previous method for expanding the set of constraints, and producing an updated partition of the data. The motivation is to iteratively increment the set of constraints by including new informative and non-redundant constraints at each iteration. Experimental results advocate that the proposed constraint acquisition methods increase the performance of data clustering.

The electroencephalogram (EEG) signal, acquired on the scalp, has been extensively used to understand cognitive function, and in particular attention. However, this type of signal has several drawbacks in a context of Physiological Computing, being susceptible to noise and requiring the use of impractical head-mounted apparatuses, which impacts normal human-computer interaction. For these reasons, the electrocardiogram (ECG) has been proposed as an alternative source to assess emotion, which is also continuously available, and related with the psychophysiological state of the subject. In this paper we present a study focused on the morphological analysis of the ECG signal acquired from subjects performing a task demanding high levels of attention. The analysis is made using various unsupervised learning techniques, which are validated against evidence found in a previous study by our team, where EEG signals collected for the same task exhibit distinct patterns as the subjects progress in the task.

We present a system for in-vehicle driver recognition based on biometric information extracted from electrocardiographic (ECG) signals collected at the hands. We recur to non-intrusive techniques, that are easy to integrate into components with which the driver naturally interacts with, such as the steering wheel. This system is applicable to the automatic customization of vehicle settings according to the perceived driver, being also prone to expand the security features of the vehicle through the detection of hands-off steering wheel events in a continuous or near-continuous manner. We have performed randomized tests for performance evaluation of the system, in a subject identification scenario, using closed sets of up to 5 subjects, showing promising results for the intended application.

The advance of falsification technology increases security concerns and gives biometrics an important role in security solutions. The electrocardiogram (ECG) is an emerging biometric that does not need liveliness verification. There is strong evidence that ECG signals contain sufficient discriminative information to allow the identification of individuals from a large population. Most approaches rely on ECG data and the fiducia of different parts of the heartbeat waveform. However non-fiducial approaches have proved recently to be also effective, and have the advantage of not relying critically on the accurate extraction of fiducia data. In this paper, we propose a new non-fiducial ECG biometric identification method based on data compression techniques, namely the Ziv-Merhav cross parsing algorithm for symbol sequences (strings). Our method relies on a string similarity measure derived from algorithmic cross complexity concept and its compression-based approximation. We present results on real data, one-lead ECG, acquired during a concentration task, from 19 healthy individuals. Our approach achieves 100% subject recognition rate despite the existence of differentiated stress states.

This work focuses on the scalability of the Evidence Accumulation Clustering (EAC) method. We first address the space complexity of the co-association matrix. The sparseness of the matrix is related to the construction of the clustering ensemble. Using a split and merge strategy combined with a sparse matrix representation, we empirically show that a linear space complexity is achievable in this framework, leading to the scalability of EAC method to clustering large data-sets.

We propose a new cluster validity index. A data partition is described by a set of disjoint sub-graphs, each corresponding to the minimum spanning tree of a cluster, taking as edge weight the dissimilarity between linked objects. Based on the assumption that each cluster has a characteristic parametric distribution of dissimilarity increments, graph probabilities are estimated. The validity index is defined as the minimum description length for both estimated model parameters and data partition, according to this probabilistic model. This new index can be used to evaluate various partitions of a given data set obtained by: (i) a single clustering algorithm, (ii) different clustering algorithms, or (iii) cluster ensemble methods. Experimental evaluation of the proposed index on synthetic and real data taken from the UCI repository confirms the usefulness of the method in selecting good clustering solutions.

We address the problem of classification of data with low separability. We adopt a Bayesian approach, with discriminant functions expressing a posteriori class probabilities. We propose a novel classification scheme incorporating classification error probability estimates in the decision process. We extend this approach into a classifier fusion framework. Presented methods are evaluated in the context of user authentication, using multimodal biometrics. Results on real data confirm the usefulness of the proposed method, outperforming the corresponding deterministic classifier.

We explore the idea of evidence accumulation (EAC) for combining the results of multiple clusterings. First, a clustering ensemble--a set of object partitions, is produced. Given a data set (n objects or patterns in d dimensions), different ways of producing data partitions are: 1) applying different clustering algorithms and 2) applying the same clustering algorithm with different values of parameters or initializations. Further, combinations of different data representations (feature spaces) and clustering algorithms can also provide a multitude of significantly different data partitionings. We propose a simple framework for extracting a consistent clustering, given the various partitions in a clustering ensemble. According to the EAC concept, each partition is viewed as an independent evidence of data organization, individual data partitions being combined, based on a voting mechanism, to generate a new n x n, similarity matrix between the n patterns. The final data partition of the n patterns is obtained by applying a hierarchical agglomerative clustering algorithm on this matrix. We have developed a theoretical framework for the analysis of the proposed clustering combination strategy and its evaluation, based on the concept of mutual information between data partitions. Stability of the results is evaluated using bootstrapping techniques. A detailed discussion of an evidence accumulation-based clustering algorithm, using a split and merge strategy based on the K-means clustering algorithm, is presented. Experimental results of the proposed method on several synthetic and real data sets are compared with other combination strategies, and with individual clustering results produced by well-known clustering algorithms.

In combination of multiple partitions, one is usually interested in deriving a consensus solution with a quality better than that of given partitions. Several recent studies have empirically demonstrated improved accuracy of clustering ensembles on a number of artificial and real-world data sets. Unlike certain multiple supervised classifier systems, convergence properties of unsupervised clustering ensembles remain unknown for conventional combination schemes. In this paper, we present formal arguments on the effectiveness of cluster ensemble from two perspectives. The first is based on a stochastic partition generation model related to re-labeling and consensus function with plurality voting. The second is to study the property of the "mean" partition of an ensemble with respect to a metric on the space of all possible partitions. In both the cases, the consensus solution can be shown to converge to a true underlying clustering solution as the number of partitions in the ensemble increases. This paper provides a rigorous justification for the use of cluster ensemble.

We address the problem of robust clustering by combining data partitions (forming a clustering ensemble) produced by multiple clusterings. We formulate robust clustering under an information-theoretical framework; mutual information is the underlying concept used in the definition of quantitative measures of agreement or consistency between data partitions. Robustness is assessed by variance of the cluster membership, based on bootstrapping. We propose and analyze a voting mechanism on pairwise associations of patterns for combining data partitions. We show that the proposed technique attempts to optimize the mutual information based criteria, although the optimality is not ensured in all situations. This evidence accumulation method is demonstrated by combining the well-known K-means algorithm to produce clustering ensembles. Experimental results show the ability of the technique to identify clusters with arbitrary shapes and sizes.

This paper addresses the problem of cluster defining criteria by proposing a model-based characterization of interpattern relationships. Taking a dissimilarity matrix between patterns as the basic measure for extracting group structure, dissimilarity increments between neighboring patterns within a cluster are analyzed. Empirical evidence suggests modeling the statistical distribution of these increments by an exponential density; we propose to use this statistical model, which characterizes context, to derive a new cluster isolation criterion. The integration of this criterion in a hierarchical agglomerative clustering framework produces a partitioning of the data, while exhibiting data interrelationships in terms of a dendrogram-type graph. The analysis of the criterion is undertaken through a set of examples, showing the versatility of the method in identifying clusters with arbitrary shape and size; the number of clusters is intrinsically found without requiring ad hoc specification of design parameters nor engaging in a computationally demanding optimization procedure.

The problem of cluster defining criteria has been addressed in various forms. In the paper, a cluster isolation criterion is proposed, underlying an hypothesis of smooth dissimilarity increments between neighboring patterns within a cluster. This isolation criterion is merged in a hierarchical agglomerative clustering algorithm, producing a data partitioning and simultaneous accessibility to the intrinsic data inter-relationships in terms of a dendrogram-type graph. By defining adequate dissimilarity measures, the algorithm is applied to vector based pattern analysis and to categorization of structural patterns. Both simulated data and real applications, in the context of automatic analysis of contour images, are presented to illustrate and evaluate the method. Examples demonstrate the versatility of the method in identifying arbitrary shape and size clusters, intrinsically finding the number of clusters.

This paper addresses the problem of object recognition based on contour descriptions. Two approaches, namely hidden Markov models (HMM) and syntactic modeling based on stochastic finite-state grammars (SFSG), are analyzed and applied to the classification of hardware tools. It is shown that both approaches are able to capture the data variability, leading to high classification performance. While the syntactic paradigm is flexible, the structure of the grammars being automatically inferred from the data, the HMMs are more robust in terms of training data sets requirements.

This paper addresses the problem of language recognition according to stochastic attributed context-free grammars. Starting with an application as the main motivation-automatic sleep analysis-a heuristic recognizer for attributed grammars is proposed. The algorithm (an extension of a previously developed method for stochastic context-free grammars) is there used both in the process of grammar inference and on the classification of sleep macrostructure. The generality of the method features its application into other areas, namely in natural language and speech understanding.

This paper addresses the problem of clustering of syntactic patterns using string descriptions. A new similarity measure between strings is proposed based on the search of common subpatterns by means of Solomonoff's coding. This coding scheme is further explored in an agglomerative clustering algorithm, which uses a minimum description length (MDL) type criterion.

This paper introduces an improved stochastic context-free language recognizer. The algorithm is basically a best-first search on the state space of dotted rules, following Earley's (1986) notation. Using heuristic merit functions; the algorithm produces the possible successors of a node (representing a state) and proceeds by exploring the most promising generated node. Two simple grammar independent heuristics are introduced: (1) ensuring convergence to a solution; and (2) guaranteeing the choice of the solution with the highest probability. It is shown that the algorithm outperforms Earley's method in both time and space domains. Several test grammars are used to illustrate the method and show its superior performance.>

In this paper, a solution to the problem of introducing a priori information in phrase structured syntactic pattern recognition systems is proposed. The method consists of devising subpatterns (representing regularities in the symbol sequences) by means of Solomonoff coding, and using these subpatterns as a priori information for constraining the production of structural samples, which serve as input to grammar inference algorithms. We have shown, in a previous work, the adequacy of syntactic modeling in automatic sleep analysis. In this paper, a further improvement is achieved by applying the proposed methodology.

The hypnogram is a schematic representation of sleep dynamics as a sequence of stages along the night. In a sense, it is a model of the so called sleep macrostructure. This paper considers the hypnogram as expression of a language whose structure is to be analysed and modeled in terms of stochastic grammars. These are inferred using Crespi-Reghizzi's method. A parsing algorithm, exploring the particular structure of the inferred grammars, has been developed. Classification strategies for both noisy and non-noisy patterns is also discussed. An example concerning a population of normals and a population of psychiatric (dysthymic) patients is presented.>