In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this article, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.

For EEG-based drowsiness recognition, it is desirable to use subject-independent recognition since conducting calibration on each subject is time-consuming. In this paper, we propose a novel Convolutional Neural Network (CNN)-Long Short-Term Memory (LSTM) model for subject-independent drowsiness recognition from single-channel EEG signals. Different from existing deep learning models that are mostly treated as black-box classifiers, the proposed model can “explain” its decisions for each input sample by revealing which parts of the sample contain important features identified by the model for classification. This is achieved by a visualization technique by taking advantage of the hidden states output by the LSTM layer. Results show that the model achieves an average accuracy of 72.97% on 11 subjects for leave-one-out subject-independent drowsiness recognition on a public dataset, which is higher than the conventional baseline methods of 55.42%-69.27%, and state-of-the-art deep learning methods. Visualization results show that the model has discovered meaningful patterns of EEG signals related to different mental states across different subjects.

The maritime industry is switching to new types of fuel such as Liquefied Natural Gas (LNG). On one hand, these kinds of fuel are more sustainable to the environment, on the other hand, training on handling such fuel safely and dealing with emergency situation is necessary. Videos and lecture-based learning is commonly used to deliver such knowledge to the maritime trainees. In recent years, the advances in Virtual Reality (VR) have brought new opportunities to such training. It provides an immersive while safe environment for training on certain operations that are extraordinary or dangerous in real life. It also allows the learners to practice the tasks repeatedly. The VR-based training is mostly used for improving technical skills, however, to guarantee a more efficient and better assessment of trainee's performance, nontechnical skills such as decision making, situation awareness, vigilance are needed to be assessed and trained as well. In this paper, a VR-based LNG evacuation training system is presented. The system provides two training scenarios for learning the evacuation procedure. A novel human factors evaluation based on the behavioral data captured by VR was proposed and integrated with the training, which includes both technical and non-technical skills assessment. An experiment with 14 subjects was conducted to validate the human factors evaluation and to get feedback towards the VR-based training.

Nowadays, increasing attention has been drawn to hybrid 2D-3D visualization tools, while evaluating them with a convenient and objective tool has only been carried out in a small number of areas. In this study, a revised radar chart-based usability evaluation approach was proposed. The approach was adopted to evaluate the hybrid 2D-3D radar display in air traffic management. The holding stack in air traffic management is analyzed and simulated, two generic tasks are designed accordingly. The hybrid 2D-3D radar display settings are evaluated based on six indicators from eye-tracking and brain dynamics data, namely, the frequency of fixation, fixation mean duration, fixation time on an area of interest, emotion, workload, and stress. The results reveal that the hybrid 2D-3D radar display induces spatial memory loss and high workload, while requires a shorter fixation duration.

Abstract Shunting of trains is a task that requires meticulous adherence to all steps to guarantee safety for everyone involved during and after the procedure. These steps are currently taught using classical teaching materials, such as printouts, videos and training by experienced supervisors. However, due to limited availability of locomotives, hours for training and manpower, training of shunting operation becomes challenging in real life. In this paper, we implemented a lifelike, collaborative virtual environment for shunting training including a novel human factors evaluation system for fatigue and stress monitoring. An experiment with 12 subjects and 3 trainers has been designed and carried out to validate the usage of VR-based shunting training. Positive feedback toward the VR-based training was obtained from the subjects and trainers.

Virtual Reality (VR) has been used for training aircraft pilots, maritime seafarers, operators, etc as it provides an immersive environment with realistic lifelike quality. We developed and implemented a VR-based Liquefied Natural Gas (LNG) firefighting simulation system with head-mounted displays (HMD) and novel human factors evaluation that could train and assess both technical and non-technical skills in the firefighting scenarios. The proposed human factors evaluation is based on a competence model and the non-technical skills such as situation awareness, vigilance, and decision making of seafarers could be assessed. An experiment was carried out with 6 trainees and 2 trainers using the implemented LNG firefighting simulation system. The results show that that the maritime trainees felt the VR scene was realistic to them, evoked similar emotions (such as fear, stress) during the demanding events as in the real world and made them attentive during the experience.

Mental fatigue is common at work places, and it can lead to decreased attention, vigilance and cognitive performance, which is dangerous in the situations such as driving, vessel maneuvering, etc. By directly measuring the neurophysiological activities happening in the brain, electroencephalography (EEG) signal can be used as a good indicator of mental fatigue. A classic EEG-based brain state recognition system requires labeled data from the user to calibrate the classifier each time before the use. For fatigue recognition, we argue that it is not practical to do so since the induction of fatigue state is usually long and weary. It is desired that the system can be calibrated using readily available fatigue data, and be applied to a new user with adequate recognition accuracy. In this paper, we explore performance of cross-subject fatigue recognition algorithms using the recently published EEG dataset labeled with two levels of fatigue. We evaluate three categories of classification method: classic classifier such as logistic regression, transfer learning-enabled classifier using transfer component analysis, and deep-learning based classifier such as EEGNet. Our results show that transfer learning-enabled classifier can outperform the other two for cross-subject fatigue recognition on a consistent basis. Specifically, transfer component analysis (TCA) improves the cross-subject recognition accuracy to 72.70 % that is higher than using just logistic regression (LR) by 9.08 % and EEGNet by 8.72 - 12.86 %.

To deal with the increasing demands in Air Traffic Control (ATC), new working place designs are proposed and developed that need novel human factors evaluation tools. In this paper, we propose a novel application of Electroencephalogram (EEG)-based emotion, workload, and stress recognition algorithms to investigate the optimal length of training for Air Traffic Control Officers (ATCOs) to learn working with three-dimensional (3D) display as a supplementary to the existing 2D display. We tested and applied the state-of-the-art EEG-based subject-dependent algorithms. The following experiment was carried out. Twelve ATCOs were recruited to take part in the experiment. The participants were in charge of the Terminal Control Area, providing navigation assistance to aircraft departing and approaching the airport using 2D and 3D displays. EEG data were recorded, and traditional human factors questionnaires were given to the participants after 15-minute, 60-minute, and 120-minute training. Different from the questionnaires, the EEG-based evaluation tools allow the recognition of emotions, workload, and stress with different temporal resolutions during the task performance by subjects. The results showed that 50-minute training could be enough for the ATCOs to learn the new display setting as they had relatively low stress and workload. The study demonstrated that there is a potential of applying the EEG-based human factors evaluation tools to assess novel system designs in addition to traditional questionnaire and feedback, which can be beneficial for future improvements and developments of the systems and interfaces.

When people are exhausted both physically and mentally from overexertion, they experience fatigue. Fatigue can lead to a decrease in motivation and vigilance which may result in certain accidents or injuries. It is crucial to monitor fatigue in workplace for safety reasons and well-being of the workers. In this paper, Electroencephalogram (EEG)-based evaluation of mental fatigue is investigated using the state-of-the-art machine learning algorithms. An experiment lasted around 2 hours and 30 minutes was designed and carried out to induce four levels of fatigue and collect EEG data from seven subjects. The results show that for subject-dependent 4-level fatigue recognition, the best average accuracy of 93.45% was achieved by using 6 statistical features with a linear SVM classifier. With subject-independent approach, the best average accuracy of 39.80% for 4 levels was achieved by using fractal dimension, 6 statistical features and a linear discriminant analysis classifier. The EEG-based fatigue recognition has the potential to be used in workplace such as cranes to monitor the fatigue of operators who are often subjected to long working hours with heavy workloads.

With the growing volume and complexity of air traffic, air traffic controllers (ATCOs) encounter heavier burden nowadays. Therefore, human factors study in air traffic control (ATC) is increasingly essential, paving the way to a safer air transportation system. In this paper, we conducted an ATC experiment, where Electroencephalogram (EEG) data were collected throughout the experiment. Compared to traditional questionnaires and psychological tests used in human factors study, the proposed novel EEG approach provides monitoring of situation awareness (SA) in a non-invasive and non-interruptive fashion. SA was represented as the response latency in situation-present assessment method (SPAM), which was predicted from EEG signals using three machine learning algorithms. Support vector regression obtained the lowest prediction error of 1.5 seconds, which is lower than 10% of the range of actual response latency. The results show that EEG is a promising approach forward in measuring situation awareness of ATCOs in both real-time and accurate manner.

Neuroscience-based or neuroscience-informed design is a new application area of Brain-Computer Interaction (BCI). It takes its roots in study of human well-being in architecture, human factors study in engineering and manufacturing including neuroergonomics. In traditional human factors studies and/or well-being study, mental workload, stress, and emotion are obtained through questionnaires that are administered upon completion of some task and/or the whole experiment. Recent advances in BCI research allow for using Electroencephalogram (EEG) based brain state recognition algorithms to assess the interaction between brain and human performance. We propose and develop an EEG-based system CogniMeter to monitor and analyze human factors measurements of newly designed software/hardware systems and/or working places. Machine learning techniques are applied to the EEG data to recognize levels of mental workload, stress and emotions during each task. The EEG is used as a tool to monitor and record the brain states of subjects during human factors study experiments. We describe two applications of CogniMeter system: human performance assessment in maritime simulator and EEG-based human factors evaluation in Air Traffic Control (ATC) workplace. By utilizing the proposed EEG-based system, true understanding of subjects working patterns can be obtained. Based on the analyses of the objective real time EEG-based data together with the subjective feedback from the subjects, we are able to reliably evaluate current systems/hardware and/or working place design and refine new concepts and design of future systems.

Real-time EEG (Electroencephalogram)-based user's emotion, mental workload and stress monitoring is a new direction in research and development of human-machine interfaces. It has attracted recently more attention from the research community and industry as wireless portable EEG devices became easily available on the market. EEG-based technology has been applied in anesthesiology, psychology, serious games or even in marketing. In this work, we describe available real-time algorithms of emotion recognition, mental workload, and stress recognition from EEG and propose a novel interface Cogni Meter for the user's mental state visual monitoring. The system can be used in real time to assess human current emotions, levels of mental workload and stress. Currently, it is applied to monitor the user's emotional state, mental workload and stress in simulation scenarios or used as a tool to assess the subject's mental state in human factor study experiments.

Everyone experiences stress in life. Moderate stress can be beneficial to human, however, excessive stress is harmful to the health. To monitor stress, different methods can be used. In this work, an algorithm for stress level recognition from Electroencephalogram (EEG) is proposed. To validate the algorithm, an experiment is designed and carried out with 9 subjects. A Stroop colour-word test is used as a stressor to induce 4 levels of stress, and the EEG data are recorded during the experiment. Different feature combinations and classifiers are proposed and analyzed. By combining fractal dimension and statistical features and using Support Vector Machine (SVM) as the classifier, four levels of stress can be recognized with an average accuracy of 67.06%, three levels of stress can be recognized with an accuracy of 75.22%, and two levels of stress can be recognized with an accuracy of 85.71%. The algorithm is integrated into the system CogniMeter for stress state monitoring. Stress level of the user is visualized on the meter in real time. The system can be applied for stress monitoring of air traffic controllers, operators, etc.

Emerging next-generation sequencing technologies have opened up exciting new opportunities for genome sequencing by generating read data with a massive throughput. However, the generated reads are significantly shorter compared to the traditional Sanger shotgun sequencing method. This poses challenges for de novo assembly algorithms in terms of both accuracy and efficiency. And due to the continuing explosive growth of short read databases, there is a high demand to accelerate the often repeated long-runtime assembly task. In this paper, we present a scalable parallel algorithm to accelerate the de Bruijn graph-based genome assembly for high-throughput short read data.

In computer graphics, textures represent the detail appearance of the surface of objects, such as colors and patterns. Example-based texture synthesis is to construct a larger visual pattern from a small example texture image. In this paper, we present a simple and efficient method which can synthesize a large scale texture in real-time based on a given example texture by simply tiling and deforming the example texture. Different from most of the existing techniques, our method does not perform search operation and it can compute texture values at any given points (random access). In addition, our method requires small storage which is only to store one example texture. Our method is suitable for synthesizing irregular and near-stochastic texture. We also propose methods to efficiently synthesize and map 3D solid textures on 3D meshes.

Searching protein sequence database is a fundamental and often repeated task in computational biology and bioinformatics. However, the high computational cost and long runtime of many database scanning algorithms on sequential architectures heavily restrict their applications for large-scale protein databases, such as GenBank. The continuing exponential growth of sequence databases and the high rate of newly generated queries further deteriorate the situation and establish a strong requirement for time-efficient scalable database searching algorithms. In this paper, we demonstrate how GPU clusters, powered by the Compute Unified Device Architecture (CUDA), OpenMP, and MPI parallel programming models can be used as an efficient computational platform to accelerate the popular BLASTP algorithm. Compared to GPU-BLAST 1.0-2.2.24, our implementation achieves speedups up to 1.6 on a single GPU and up to 6.6 on the 6 GPUs of a Tesla S1060 quad-GPU computing system. The source code is available at: http://sites.google.com/site/liuweiguohome/mpicuda-blastp

Line drawings are an effective way to convey shapes in a relatively succinct manner by ignoring the less important or distracting details. In the past decade, many promising computer-generated line drawing algorithms have been proposed, which can be roughly classified into two categories: object-space and image-space.

Scanning protein sequence database is an often repeated task in computational biology and bioinformatics. However, scanning large protein databases, such as GenBank, with popular tools such as BLASTP requires long runtimes on sequential architectures. Due to the continuing rapid growth of sequence databases, there is a high demand to accelerate this task. In this paper, we demonstrate how GPUs, powered by the Compute Unified Device Architecture (CUDA), can be used as an efficient computational platform to accelerate the BLASTP algorithm. In order to exploit the GPU’s capabilities for accelerating BLASTP, we have used a compressed deterministic finite state automaton for hit detection as well as a hybrid parallelization scheme. Our implementation achieves speedups up to 10.0 on an NVIDIA GeForce GTX 295 GPU compared to the sequential NCBI BLASTP 2.2.22. CUDA-BLASTP source code which is available at https://sites.google.com/site/liuweiguohome/software.

Art. 10, 2 S.

Emerging DNA sequencing technologies open up exciting new opportunities for genome sequencing by generating read data with a massive throughput. However, produced reads are significantly shorter and more error-prone compared to the traditional Sanger shotgun sequencing method. This poses challenges for de-novo DNA fragment assembly algorithms in terms of both accuracy (to deal with short, error-prone reads) and scalability (to deal with very large input data sets). In this paper we present a scalable parallel algorithm for correcting sequencing errors in high-throughput short-read data. It is based on spectral alignment and uses the CUDA programming model. Our computational experiments on a GTX 280 GPU show runtime savings between 10 and 19 times (for different error-rates using simulated datasets as well as real Solexa/Illumina datasets).

Using modern graphics processing units for no-graphics high performance computing is motivated by their enhanced programmability, attractive price/performance ratio and incredible growth in speed. Although the pipeline of a modern graphics processing unit (GPU) permits high throughput and more concurrency, they bring more complexities in analyzing the performance of GPU-based applications. In this paper, we identify factors that determine performance of GPU-based applications. We then classify them into three categories: data-linear, data-constant and computation-dependent. According to the characteristics of these factors, we propose a performance model for each factor. These models are then used to predict the performance of bio-sequence database scanning application on GPUs. Theoretical analyses and measurements show that our models can achieve precise performance predictions.

Sequence alignment is a common and often repeated task in molecular biology. Typical alignment operations consist of finding similarities between a pair of sequences (pairwise sequence alignment) or a family of sequences (multiple sequence alignment). The need for speeding up this treatment comes from the rapid growth rate of biological sequence databases: every year their size increases by a factor 1.5 to 2. In this paper we present a new approach to high performance biological sequence alignment based on commodity PC graphics hardware. Using modern graphics processing units (GPUs) for high performance computing is facilitated by their enhanced programmability and motivated by their attractive price/performance ratio and incredible growth in speed. To derive an efficient mapping onto this type of architecture, we have reformulated dynamic programming based alignment algorithms as streaming algorithms in terms of computer graphics primitives. Our experimental results show that the GPU-based approach allows speedups of over one order of magnitude with respect to optimized CPU implementations.

Protein sequences with unknown functionality are often compared to a set of known sequences to detect functional similarities. Efficient dynamic programming algorithms exist for this problem, however current solutions still require significant scan times. These scan time requirements are likely to become even more severe due to the rapid growth in the size of these databases. In this paper, we present a new approach to bio-sequence database scanning using computer graphics hardware to gain high performance at low cost. To derive an efficient mapping onto this type of architecture, we have reformulated the Smith-Waterman dynamic programming algorithm in terms of computer graphics primitives. Our OpenGL implementation achieves a speedup of approximately sixteen on a high-end graphics card over available straightforward and optimized CPU Smith-Waterman implementations

Rapid advances in the field of virtual reality (VR) technology has opened up many new areas for research and development. One of the recent important research fields that has emerged is in the field of digital heritage. VR technology is an important educational tool that provides immersive and interactive qualities to the cultural heritage content thus enabling us to accurately represent digital reconstructions of heritage sites that are no longer in existence or are inaccessible. We present our project on the digital heritage scenario focusing on cultural heritage content specific to the Singapore region. Innovative interaction techniques specific to the selected digital heritage content will also be presented. We outline the motivation, early developments, and implementation currently in progress.

Computer assisted operation planning systems win more and more recognition in the field of surgery. These systems offer new possibilities to prepare an intervention with the goal to shorten the expansive time in the operation room required for the intervention. The safest and most effective surgical approach should be selected. But often, it is difficult to transfer the output of the planning system to the intra-operative situation and so to consider the planning results in the real intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore, methods are developed to bridge the gap between the external planning session and the intra-operative case: augmented reality (AR) techniques are used to overlap preoperative scanned image data as well as results of the planning session to the operation field.

A hierarchically structured constraint-based data model for solid modelling in a virtual reality environment is presented. The data model integrates a high-level constraint-based model for precise object definition, a mid-level CSG/Brep hybrid solid model for supporting hierarchical geometry abstractions and object creation, and a low-level polygon model for real-time visualization and interaction in the virtual reality environment. Constraints are embedded in the solid model and organized at different levels to reflect the entire solid modelling process from features and parts, to assemblies. This model not only provides precise object definition, but also supports real-time visualization and interaction in the VR environment. Furthermore, it has the potential to obtain precise 3D interactions and precise constraint-based manipulations can be deduced from the data model to carry out precise solid modelling in the VR environment.

The absence of constraints is one of the major limitations in current Virtual Reality (VR) environments. Without constraints, it is difficult to perform precise 3D interactive manipulations in VR environments and precise solid modelling in VR environments cannot be guaranteed. In this paper, constraints are incorporated into the VR environment for intuitive and precise solid modelling. A hierarchically structured constraint-based data model is developed to support solid modelling in the VR environment. Solid modelling in the VR environment is precisely performed in an intuitive manner through constraint-based manipulations. Constraint-based manipulations are accompanied with automatic constraint recognition and precise constraint satisfaction to establish the hierarchically structured constraint-based data model and are realized by allowable motions for precise 3D interactions in the VR environment. The allowable motions are represented as a mathematical matrix for conveniently deriving allowable motions from constraints. A procedure-based degree-of-freedom incorporation approach for 3D constraint solving is presented for deriving the allowable motions. A rule-based constraint recognition engine is developed for both constraint-based manipulations and implicitly incorporating constraints into the VR environment. A prototype system has been implemented for precise solid modelling in an intuitive manner through constraint-based manipulations in the VR environment.

With today's virtual reality systems, it is difficult to directly and precisely create and modify complex objects in a virtual reality environment. One of the most important reasons is the absence of a suitable model representation that can efficiently support solid modelling in a virtual reality environment. A hierarchically structured constraint-based data model for solid modelling in the virtual reality environment is presented in this paper. The data model integrates a high-level constraint-based model for precise object definition, a mid-level CSG/B rep hybrid solid model for supporting hierarchical geometry abstractions and object creation, and a low-level polygon model for real-time visualization and interaction in the virtual reality environment. Constraints are embedded in the solid model and are organized at different levels to reflect the process of solid modelling. This data model not only provides precise object definition, but also supports real-time visualization and interaction in the virtual reality environment.