In many real-world applications involving static environments, the spatial layout of objects remains consistent across instances. However, state-of-the-art object detection models often fail to leverage this spatial prior, resulting in inconsistent predictions, missed detections, or misclassifications, particularly in cluttered or occluded scenes. In this work, we propose a graph-based post-processing pipeline that explicitly models the spatial relationships between objects to correct detection anomalies in egocentric frames. Using a graph neural network (GNN) trained on manually annotated data, our model identifies invalid object class labels and predicts corrected class labels based on their neighbourhood context. We evaluate our approach both as a standalone anomaly detection and correction framework and as a post-processing module for standard object detectors such as YOLOv7 and RT-DETR. Experiments demonstrate that incorporating this spatial reasoning significantly improves detection performance, with$\text{mAP} {@} 50$gains of up to 4 %. This method highlights the potential of leveraging the environment's spatial structure to improve reliability in object detection systems.

Accurate evaluation of procedural communication compliance is essential in simulation-based training, particularly in safety-critical domains where adherence to compliance checklists reflects operational competence. This paper explores a lightweight, deployable approach using prompt-based inference with open-source large language models (LLMs) that can run efficiently on consumer-grade GPUs. We present Prompt-and-Check, a method that uses context-rich prompts to evaluate whether each checklist item in a protocol has been fulfilled, solely based on transcribed verbal exchanges. We perform a case study in the maritime domain with participants performing an identical simulation task, and experiment with models such as LLama 2 7B, LLaMA 3 8B and Mistral 7B, running locally on an RTX 4070 GPU. For each checklist item, a prompt incorporating relevant transcript excerpts is fed into the model, which outputs a compliance judgment. We assess model outputs against expert-annotated ground truth using classification accuracy and agreement scores. Our findings demonstrate that prompting enables effective context-aware reasoning without task-specific training. This study highlights the practical utility of LLMs in augmenting debriefing, performance feedback, and automated assessment in training environments.

The maritime industry is actively pursuing decarbonization by transitioning to alternative fuels with low or zero emissions. Among these, ammonia is a leading candidate due to its carbon-free combustion properties. However, its high toxicity introduces significant handling risks, making effective training essential. This paper presents a two-player Virtual Reality (VR) training system designed to equip maritime personnel with the skills required to manage ammonia leaks safely. To enhance immersion and memory retention, an aroma diffuser is integrated into the VR environment to simulate the smell of ammonia during leak scenarios. Additionally, an Artificial Intelligence (AI)-based assessment module evaluates trainee performance in key areas such as communication, decisionmaking, and situational vigilance.

Seafarer competence in visual attention, communication, and stress management is critical to maritime safety. While assessment methods for these individual areas have evolved with simulators, sensor data, and artificial intelligence, there remains no established approach to systematically integrate these outputs into a practical competence measure to support more efficient and standardised debriefing and to enable reliable comparison between trainees. In this paper, we present a multimodal assessment report that fuses analysed results from eyetracking, audio recordings, and Electroencephalogram (EEG) signals collected during full-mission bridge simulator exercises. The report provides an integrated, instructor-friendly summary of each trainee's performance across key competence pillars, along with an executive summary. This approach offers maritime instructors a more comprehensive and objective tool to identify individual strengths, weaknesses, and development needs.

Articulatory trajectories like electromagnetic articulography (EMA) provide a low-dimensional representation of the vocal tract filter and have been used as natural, grounded features for speech synthesis. Differentiable digital signal processing (DDSP) is a parameter-efficient framework for audio synthesis. Therefore, integrating low-dimensional EMA features with DDSP can significantly enhance the computational efficiency of speech synthesis. In this paper, we propose a fast, high-quality, and parameter-efficient DDSP articulatory vocoder that can synthesize speech from EMA, F0, and loudness. We incorporate several techniques to solve the harmonics / noise imbalance problem, and add a multiresolution adversarial loss for better synthesis quality. Our model achieves a transcription word error rate (WER) of 6.67% and a mean opinion score (MOS) of 3.74, with an improvement of 1.63% and 0.16 compared to the state-of-the-art (SOTA) baseline. Our DDSP vocoder is 4.9 x faster than the baseline on CPU during inference, and can generate speech of comparable quality with only 0.4 M parameters, in contrast to the 9 M parameters required by the SOTA.

Estimation of fundamental frequency (F0) in voiced segments of speech signals, also known as pitch tracking, plays a crucial role in pitch synchronous speech analysis, speech synthesis, and speech manipulation. In this paper, we capitalize on the high time and frequency resolution of the pseudo Wigner-Ville distribution (PWVD) and propose a new PWVD-based pitch estimation method. We devise an efficient algorithm to compute PWVD faster and use cepstrum-based pre-filtering to avoid cross-term interference. Evaluating our approach on databases with speech and electroglottograph (EGG) recordings yields a state-of-the-art mean absolute error (MAE) of around 4Hz. Our approach is also effective at voiced/unvoiced classification and handling sudden frequency changes.

For EEG-based drowsiness recognition, it is desirable to use subject-independent recognition since conducting calibration on each subject is time-consuming. In this paper, we propose a novel Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) model for subject-independent drowsiness recognition from single-channel EEG signals. Different from existing deep learning models that are mostly treated as black-box classifiers, the proposed model can “explain” its decisions for each input sample by revealing which parts of the sample contain important features identified by the model for classification. This is achieved by a visualization technique by taking advantage of the hidden states output by the LSTM layer. Results show that the model achieves an average accuracy of 72.97% on 11 subjects for leave-one-out subject-independent drowsiness recognition on a public dataset, which is higher than the conventional baseline methods of 55.42%-69.27%, and state-of-the-art deep learning methods. Visualization results show that the model has discovered meaningful patterns of EEG signals related to different mental states across different subjects.

The maritime industry is switching to new types of fuel such as Liquefied Natural Gas (LNG). On one hand, these kinds of fuel are more sustainable to the environment, on the other hand, training on handling such fuel safely and dealing with emergency situation is necessary. Videos and lecture-based learning is commonly used to deliver such knowledge to the maritime trainees. In recent years, the advances in Virtual Reality (VR) have brought new opportunities to such training. It provides an immersive while safe environment for training on certain operations that are extraordinary or dangerous in real life. It also allows the learners to practice the tasks repeatedly. The VR-based training is mostly used for improving technical skills, however, to guarantee a more efficient and better assessment of trainee's performance, nontechnical skills such as decision making, situation awareness, vigilance are needed to be assessed and trained as well. In this paper, a VR-based LNG evacuation training system is presented. The system provides two training scenarios for learning the evacuation procedure. A novel human factors evaluation based on the behavioral data captured by VR was proposed and integrated with the training, which includes both technical and non-technical skills assessment. An experiment with 14 subjects was conducted to validate the human factors evaluation and to get feedback towards the VR-based training.

Nowadays, increasing attention has been drawn to hybrid 2D-3D visualization tools, while evaluating them with a convenient and objective tool has only been carried out in a small number of areas. In this study, a revised radar chart-based usability evaluation approach was proposed. The approach was adopted to evaluate the hybrid 2D-3D radar display in air traffic management. The holding stack in air traffic management is analyzed and simulated, two generic tasks are designed accordingly. The hybrid 2D-3D radar display settings are evaluated based on six indicators from eye-tracking and brain dynamics data, namely, the frequency of fixation, fixation mean duration, fixation time on an area of interest, emotion, workload, and stress. The results reveal that the hybrid 2D-3D radar display induces spatial memory loss and high workload, while requires a shorter fixation duration.

Abstract Shunting of trains is a task that requires meticulous adherence to all steps to guarantee safety for everyone involved during and after the procedure. These steps are currently taught using classical teaching materials, such as printouts, videos and training by experienced supervisors. However, due to limited availability of locomotives, hours for training and manpower, training of shunting operation becomes challenging in real life. In this paper, we implemented a lifelike, collaborative virtual environment for shunting training including a novel human factors evaluation system for fatigue and stress monitoring. An experiment with 12 subjects and 3 trainers has been designed and carried out to validate the usage of VR-based shunting training. Positive feedback toward the VR-based training was obtained from the subjects and trainers.

Virtual Reality (VR) has been used for training aircraft pilots, maritime seafarers, operators, etc as it provides an immersive environment with realistic lifelike quality. We developed and implemented a VR-based Liquefied Natural Gas (LNG) firefighting simulation system with head-mounted displays (HMD) and novel human factors evaluation that could train and assess both technical and non-technical skills in the firefighting scenarios. The proposed human factors evaluation is based on a competence model and the non-technical skills such as situation awareness, vigilance, and decision making of seafarers could be assessed. An experiment was carried out with 6 trainees and 2 trainers using the implemented LNG firefighting simulation system. The results show that that the maritime trainees felt the VR scene was realistic to them, evoked similar emotions (such as fear, stress) during the demanding events as in the real world and made them attentive during the experience.

Mental fatigue is common at work places, and it can lead to decreased attention, vigilance and cognitive performance, which is dangerous in the situations such as driving, vessel maneuvering, etc. By directly measuring the neurophysiological activities happening in the brain, electroencephalography (EEG) signal can be used as a good indicator of mental fatigue. A classic EEG-based brain state recognition system requires labeled data from the user to calibrate the classifier each time before the use. For fatigue recognition, we argue that it is not practical to do so since the induction of fatigue state is usually long and weary. It is desired that the system can be calibrated using readily available fatigue data, and be applied to a new user with adequate recognition accuracy. In this paper, we explore performance of cross-subject fatigue recognition algorithms using the recently published EEG dataset labeled with two levels of fatigue. We evaluate three categories of classification method: classic classifier such as logistic regression, transfer learning-enabled classifier using transfer component analysis, and deep-learning based classifier such as EEGNet. Our results show that transfer learning-enabled classifier can outperform the other two for cross-subject fatigue recognition on a consistent basis. Specifically, transfer component analysis (TCA) improves the cross-subject recognition accuracy to 72.70 % that is higher than using just logistic regression (LR) by 9.08 % and EEGNet by 8.72 - 12.86 %.

Currently, many modern humanoid robots have little appeal due to their simple designs and bland appearances. To provide recommendations for designers and improve the designs of humanoid robots, a study of human's perception on humanoid robot designs is conducted using Electroencephalogram (EEG), eye tracking information and questionnaires. We proposed and carried out an experiment with 20 subjects to collect the EEG and eye tracking data to study their reaction to different robot designs and the corresponding preference towards these designs. This study can possibly give us some insights on how people react to the aesthetic designs of different humanoid robot models and the important traits in a humanoid robot design, such as the perceived smartness and friendliness of the robots. Another point of interest is to investigate the most prominent feature of the robot, such as the head, facial features and the chest. The result shows that the head and facial features are the focus. It is also discovered that more attention is paid to the robots that appear to be more appealing. Lastly, it is affirmed that the first impressions of the robots generally do not change over time, which may imply that a good humanoid robot design impress the observers at first sight.

To deal with the increasing demands in Air Traffic Control (ATC), new working place designs are proposed and developed that need novel human factors evaluation tools. In this paper, we propose a novel application of Electroencephalogram (EEG)-based emotion, workload, and stress recognition algorithms to investigate the optimal length of training for Air Traffic Control Officers (ATCOs) to learn working with three-dimensional (3D) display as a supplementary to the existing 2D display. We tested and applied the state-of-the-art EEG-based subject-dependent algorithms. The following experiment was carried out. Twelve ATCOs were recruited to take part in the experiment. The participants were in charge of the Terminal Control Area, providing navigation assistance to aircraft departing and approaching the airport using 2D and 3D displays. EEG data were recorded, and traditional human factors questionnaires were given to the participants after 15-minute, 60-minute, and 120-minute training. Different from the questionnaires, the EEG-based evaluation tools allow the recognition of emotions, workload, and stress with different temporal resolutions during the task performance by subjects. The results showed that 50-minute training could be enough for the ATCOs to learn the new display setting as they had relatively low stress and workload. The study demonstrated that there is a potential of applying the EEG-based human factors evaluation tools to assess novel system designs in addition to traditional questionnaire and feedback, which can be beneficial for future improvements and developments of the systems and interfaces.

Affective brain-computer interface (aBCI) introduces personal affective factors into human-computer interactions, which could potentially enrich the user's experience during the interaction with a computer. However, affective neural patterns are volatile even within the same subject. To maintain satisfactory emotion recognition accuracy, the state-of-the-art aBCIs mainly tailor the classifier to the subject-of-interest and require frequent re-calibrations for the classifier. In this paper, we demonstrate that the recognition accuracy of aBCIs deteriorates when re-calibration is ruled out during the long-term usage for the same subject. Then, we propose a stable feature selection method to choose the most stable affective features, for mitigating the accuracy deterioration to a lesser extent and maximizing the aBCI performance in the long run. We validate our method on a dataset comprising six subjects' EEG data collected during two sessions per day for each subject for eight consecutive days.

When people are exhausted both physically and mentally from overexertion, they experience fatigue. Fatigue can lead to a decrease in motivation and vigilance which may result in certain accidents or injuries. It is crucial to monitor fatigue in workplace for safety reasons and well-being of the workers. In this paper, Electroencephalogram (EEG)-based evaluation of mental fatigue is investigated using the state-of-the-art machine learning algorithms. An experiment lasted around 2 hours and 30 minutes was designed and carried out to induce four levels of fatigue and collect EEG data from seven subjects. The results show that for subject-dependent 4-level fatigue recognition, the best average accuracy of 93.45% was achieved by using 6 statistical features with a linear SVM classifier. With subject-independent approach, the best average accuracy of 39.80% for 4 levels was achieved by using fractal dimension, 6 statistical features and a linear discriminant analysis classifier. The EEG-based fatigue recognition has the potential to be used in workplace such as cranes to monitor the fatigue of operators who are often subjected to long working hours with heavy workloads.

Deep investment in the maritime industries has led to many cutting edge technological advances in shipping navigation and operational safety to ensure safe and efficient logistical transportations. However, even with the best technology equipped onboard, maritime accidents are still occurring with at least three quarters of them attributed to human errors. Due to the rising need to address the human factors in shipping operations, various human factors studies are conducted in maritime domain. In this paper, an Electroencephalogram (EEG)-based cadets training and performance assessment system is proposed and implemented that could be used in the maritime virtual simulator. The system includes an EEG processing and analyses part and an evaluation part. It could recognize the brain states such as mental workload, emotions, and stress from raw EEG signal recorded during the exercises in the simulator and then give an indicative recommendation on "pass", "retrain", or "fail" of the cadet based on the EEG recognition results and input of the level of the task difficulty performed.

Many studies have shown that the majority of maritime accidents/incidents are attributed to human errors as the initiating cause. Efforts have been made to study human factors that can result in a safer maritime transportation. Among all techniques, Electroencephalogram (EEG) has the advantages such as high time resolution, possibility to continuously monitor brain states with high accuracy, recognition of human mental workload, emotion, stress, vigilance, etc. In this paper, we designed and carried out an experiment to collect the EEG signals to study stress and sharing of the mental workload among crew members during collaboration tasks performance on the ship's bridge virtual simulator. Four maritime trainees were monitored in the experiment. Each of them had a role such as an officer on watch, captain, pilot, or steersman. The results show that the captain had the highest stress and workload. However, the other three trainees experienced low workload and stress due to shared work and responsibility. The EEG is a promising evaluation tool to be used in the human factors study in the maritime domain.

Neurofeedback training is one type of the biofeedback training that allows the subject do self-regulation during the training according to his/her real-time brain activities recognized from Electroencephalogram (EEG) and given to him/her through visual, audio or haptic feedback. The Neurofeedback training has been proved to be helpful in improvement of cognitive abilities not only for patients with mental illnesses but also for healthy subjects including athletes. In this paper, we proposed an experiment in which we recruited elite shooters and conducted a novel individual beta-1/theta based neurofeedback training to confirm the usage of neurofeedback training in boosting the performance of rifle shooters. The efficiency of the neurofeedback training was examined by comparing the shooting scores and results of DAUF test assessing the ability of sustained attention of the shooters before and after neurofeedback training.

With the growing volume and complexity of air traffic, air traffic controllers (ATCOs) encounter heavier burden nowadays. Therefore, human factors study in air traffic control (ATC) is increasingly essential, paving the way to a safer air transportation system. In this paper, we conducted an ATC experiment, where Electroencephalogram (EEG) data were collected throughout the experiment. Compared to traditional questionnaires and psychological tests used in human factors study, the proposed novel EEG approach provides monitoring of situation awareness (SA) in a non-invasive and non-interruptive fashion. SA was represented as the response latency in situation-present assessment method (SPAM), which was predicted from EEG signals using three machine learning algorithms. Support vector regression obtained the lowest prediction error of 1.5 seconds, which is lower than 10% of the range of actual response latency. The results show that EEG is a promising approach forward in measuring situation awareness of ATCOs in both real-time and accurate manner.

Neuroscience-based or neuroscience-informed design is a new application area of Brain-Computer Interaction (BCI). It takes its roots in study of human well-being in architecture, human factors study in engineering and manufacturing including neuroergonomics. In traditional human factors studies and/or well-being study, mental workload, stress, and emotion are obtained through questionnaires that are administered upon completion of some task and/or the whole experiment. Recent advances in BCI research allow for using Electroencephalogram (EEG) based brain state recognition algorithms to assess the interaction between brain and human performance. We propose and develop an EEG-based system CogniMeter to monitor and analyze human factors measurements of newly designed software/hardware systems and/or working places. Machine learning techniques are applied to the EEG data to recognize levels of mental workload, stress and emotions during each task. The EEG is used as a tool to monitor and record the brain states of subjects during human factors study experiments. We describe two applications of CogniMeter system: human performance assessment in maritime simulator and EEG-based human factors evaluation in Air Traffic Control (ATC) workplace. By utilizing the proposed EEG-based system, true understanding of subjects working patterns can be obtained. Based on the analyses of the objective real time EEG-based data together with the subjective feedback from the subjects, we are able to reliably evaluate current systems/hardware and/or working place design and refine new concepts and design of future systems.

Emotion recognition is an integral part of affective computing. An affective brain-computer-interface (BCI) can benefit the user in a number of applications. In most existing studies, EEG (electroencephalograph)-based emotion recognition is explored in a classificatory manner. In this manner, human emotions are discretized by a set of emotion labels. However, human emotions are more of a continuous phenomenon than discrete. A regressive approach is more suited for continuous emotion recognition. Few studies have looked into a regressive approach. In this study, we investigate a portfolio of EEG features including fractal dimension, statistics and band power. Support vector regression (SVR) is employed in this study to estimate subject's valence level by means of different features under two evaluation schemes. In the first scheme, a SVR is constructed with full training resources, whereas in the second scheme, a SVR only receives minimal training resources. MAE (mean absolute error) averages of 0.74 and 1.45 can be achieved under the first and the second scheme, respectively, by fractal feature. The advantages of a regressive approach over classificatory approach lie in continuous emotion recognition and the possibility to reduce training resources to minimal level.

The individual alpha peak frequency (IAPF) is an important biological indicator in Electroencephalogram (EEG) studies, with many research publications linking it to various cognitive functions. In this paper, we propose novel Power Spectral Density (PSD) alpha features based on IAPF to classify 2 and 4 levels of EEG multitasking workload data. When optimized IAPF was considered, a 1.55% and 1.56% increase in average accuracy for 48 subjects' data, with 35 and 33 subjects showing improvement was observed for 2 and 4 class cases respectively. This trend suggests that individual specific features are able to improve classification performance compared to generalized features for subject dependent cases. The proposed features, which incorporates the biological meaning of the IAPF and provides subject specific information, can be considered as a viable alternative to the general alpha power feature when designing novel subject dependent feature sets for BCI workload recognition applications.

Haptic interaction is a form of a user-computer interaction where physical forces are delivered to the user via vibrations, displacements and rotations of special haptic devices. When quality of the experience of the haptic interaction is assessed, mostly subjective tests using various questionnaires are performed. We proposed novel neurocognitive tools for assessing both overall experience of the haptic interaction, as well as particular time-stamped activities. Our assessment tools are based on recognition of emotions and stress obtained from Electroencephalograms (EEG). We used them in a feasibility study on adding haptic interaction to Skype video conversation.

Real-time EEG (Electroencephalogram)-based user's emotion, mental workload and stress monitoring is a new direction in research and development of human-machine interfaces. It has attracted recently more attention from the research community and industry as wireless portable EEG devices became easily available on the market. EEG-based technology has been applied in anesthesiology, psychology, serious games or even in marketing. In this work, we describe available real-time algorithms of emotion recognition, mental workload, and stress recognition from EEG and propose a novel interface Cogni Meter for the user's mental state visual monitoring. The system can be used in real time to assess human current emotions, levels of mental workload and stress. Currently, it is applied to monitor the user's emotional state, mental workload and stress in simulation scenarios or used as a tool to assess the subject's mental state in human factor study experiments.

The difference in cognitive abilities of humans could be assessed by indexes extracted from EEG. In this paper, we propose and implement an experiment with 60 subjects to study how cognitive abilities can be identified through EEG. We analyzed parameters of the individual frequency band that can be used for prediction of cognitive abilities of subjects. In the experiment, the subjects performed cognitive tests with EEG recording done prior to the tests. Different patterns of alpha band activity are proven to be indicators of cognitive abilities and performances. Our hypothesis is that cognitive abilities can be predicted based on EEG measurements. The results of analysis of the experiment show significant correlation between subjects' cognitive abilities assessed by the tests and the EEG measurements.

Everyone experiences stress in life. Moderate stress can be beneficial to human, however, excessive stress is harmful to the health. To monitor stress, different methods can be used. In this work, an algorithm for stress level recognition from Electroencephalogram (EEG) is proposed. To validate the algorithm, an experiment is designed and carried out with 9 subjects. A Stroop colour-word test is used as a stressor to induce 4 levels of stress, and the EEG data are recorded during the experiment. Different feature combinations and classifiers are proposed and analyzed. By combining fractal dimension and statistical features and using Support Vector Machine (SVM) as the classifier, four levels of stress can be recognized with an average accuracy of 67.06%, three levels of stress can be recognized with an accuracy of 75.22%, and two levels of stress can be recognized with an accuracy of 85.71%. The algorithm is integrated into the system CogniMeter for stress state monitoring. Stress level of the user is visualized on the meter in real time. The system can be applied for stress monitoring of air traffic controllers, operators, etc.

Stability of algorithms is very important for electroencephalogram (EEG) based applications. Stable features should exhibit consistency among repeated measurements of the same subject. Previously, power features were reported to be one of the most stable EEG features in medical application. In this paper, stability of features in emotion recognition algorithms is studied. Our hypothesis is that the most stable features give the best intra-subject accuracy across different days in real-time emotion recognition algorithm. An experiment to induce 4 emotions such as pleasant, happy, frightened, and angry is designed and carried out in 8 consecutive days (two sessions per day) for 4 subjects to record EEG data. A novel real-time subject dependent algorithm with the most stable features is proposed and implemented. The algorithm needs just one training for each subject. The training results can be used in real-time emotion recognition applications without re-training with the adequate accuracy. The proposed algorithm is integrated with a real-time application "Emotional Avatar".

Neurofeeback training can be used to enhance cognitive abilities related to multi-tasking such as working memory, attention, etc. We propose and implement a neurofeedback system which includes a number of neurofeedback training algorithms and a Shooting game. To make neurofeedback training more effective, an Individual Alpha Peak (IAP) frequency and individual alpha bandwidth are calculated and applied in the algorithms. We do preliminary study on the effectiveness of the proposed neurofeedback system with three subjects taking 6 sessions each. The neurofeedback protocols based on the power of individual upper alpha or beta-1/theta ratio training are used. Our hypothesis is that after the neurofeedback training by playing the Shooting game, the individual alpha peak frequency increases. The results show that all subjects had a higher individual alpha peak frequency after the training that indicated an enhancement of the subjects' cognitive abilities related to multi-tasking.

In this paper, a real-time Electroencephalogram (EEG)-based emotion recognition algorithm using Higuchi Fractal Dimension (FD) Spectrum is proposed. As EEG is a nonlinear and multi-fractal signal, its FD spectrum can give a better understanding of the nonlinear property of EEG. Three values are selected from the whole spectrum and are combined with the other features such as statistical and Higher Order Crossings ones. The Support Vector Machine is used as the classifier. The proposed algorithm is validated on both benchmark database DEAP with video stimuli and our own dataset which used visual stimuli to evoke emotions. Up to 8 emotions can be recognized with only 4 channels. The experiment analysis results show that using FD spectrum features it is possible to improve classification accuracy.

Nowadays, advertising is a part of our daily life driving consumer behavior, social behavior, personal preferences, etc. As volumes of advertisements increase people become more immune to different types of advertisements. To make the advertisements more efficient is a challenging problem. It is confirmed in the experiments that the emotions felt by the participants during the viewing of an advertisement influence on the effectiveness of the advertisement. In this paper, we propose an emotion-enabled algorithm that can be used to personalize an advertising movie according to the user's current emotions to make the advertisement more efficient. Electroencephalogram (EEG) signals are used to recognize emotions of the user in real time. The proposed emotion-enabled algorithm can adjust the scene of the movie based on the real-time emotion feedback. An advertising movie that applies the emotion-enabled algorithm is designed and implemented.

Using Electroencephalogram (EEG) signals for affective interaction can make interfaces more intuitive. This project includes development of different affective applications based on an EEG-based real-time emotion recognition algorithm. The algorithm is subject-dependent one and consists from two parts: feature extraction and classification. The algorithm can recognize up to eight emotions with good accuracy. The demo includes affective games and emotional avatar applications. After a short session to train the classifier, an application is able to monitor the user's emotions, and the recognized emotions are used as the input to the applications.

Emotion recognition from Electroencephalogram (EEG) rapidly gains interest from research community. Two affective EEG databases are presented in this paper. Two experiments are conducted to set up the databases. Audio and visual stimuli are used to evoke emotions during the experiments. The stimuli are selected from IADS and IAPS databases.14 subjects participated in each experiment. Emotiv EEG device is used for the data recording. The EEG data are rated by the participants with arousal, valence, and dominance levels. The correlation between powers of different EEG bands and the affective ratings is studied. The results agree with the literature findings and analyses of benchmark DEAP database that proves the reliability of the two databases. Similar brain patterns of emotions are obtained between the established databases and the benchmark database. A SVM-based emotion recognition algorithm is proposed and applied to both databases and the benchmark database. Use of a Fractal Dimension feature in combination with statistical and Higher Order Crossings (HOC) features gives us results with the best accuracy. Up to 8 emotions can be recognized. The accuracy is consistent between the established databases and the benchmark database.

Emotions are important in human-computer interaction. Emotions could be classified based on 3-dimensional Valence-Arousal-Dominance model which allows defining any number of emotions even without discrete emotion labels. In this paper, we proposed a real-time EEG-based subject-dependent valence level recognition algorithm, where the thresholds were used to identify different levels of the valence dimension of the human emotion. The algorithm was tested by using the EEG data labeled with valence levels. The algorithm could identify valence levels continuously. The algorithm was tested with the experiment data and with the benchmark affective EEG database DEAP where up to 9 levels of valence dimension with high/low dominance were recognized. Then, the algorithm was applied to recognize 16 emotions defined by high/low arousal, high/low dominance and 4 levels of valence. At least 14 electrodes should be used to get the better accuracy. The proposed algorithm could be implemented in different real-time applications such as emotional avatar and E-learning systems.

Emotions recognized from Electroencephalogram (EEG) could reflect the real "inner" feelings of the human. Recently, research on real-time emotion recognition received more attention since it could be applied in games, e-learning systems or even in marketing. EEG signal can be divided into the delta, theta, alpha, beta, and gamma waves based on their frequency bands. Based on the Valence-Arousal-Dominance emotion model, we proposed a subject-dependent algorithm using the beta/alpha ratio to recognize high and low dominance levels of emotions from EEG. Three experiments were designed and carried out to collect the EEG data labeled with emotions. Sound clips from International Affective Digitized Sounds (IADS) database and music pieces were used to evoke emotions in the experiments. Our approach would allow real-time recognition of the emotions defined with different dominance levels in Valence-Arousal-Dominance model.

Emotions accompany everyone in the daily life, playing a key role in non-verbal communication, and they are essential to the understanding of human behavior. Emotion recognition could be done from the text, speech, facial expression or gesture. In this paper, we concentrate on recognition of “inner” emotions from electroencephalogram (EEG) signals as humans could control their facial expressions or vocal intonation. The need and importance of the automatic emotion recognition from EEG signals has grown with increasing role of brain computer interface applications and development of new forms of human-centric and human-driven interaction with digital media. We propose fractal dimension based algorithm of quantification of basic emotions and describe its implementation as a feedback in 3D virtual environments. The user emotions are recognized and visualized in real time on his/her avatar adding one more so-called “emotion dimension” to human computer interfaces.