Existing conversational search systems can synthesize information into responses, but they lack principled ways to adapt response formats to users’ cognitive states. This paper investigates whether aligning format and distance, which involves matching information granularity and media to users’ psychological distance, improves user experience. In a between-subjects experiment (N = 464) on travel planning, we crossed two distance dimensions (temporal/spatial × near/far) with four formats varying in granularity (abstract/concrete) and media (text/image-and-text). The experiment established that format–distance alignment reduced users’ risk perceptions while increasing decision confidence, perceptions of information usefulness, ease of use, enjoyment, and credibility, and adoption intentions. Concrete formats imposed higher cognitive load, but yielded productive effort when matched to near-distance tasks. Images enhanced concrete but not abstract text, suggesting multimedia benefits depend on complementarity. These findings establish format–distance alignment as a distinctive and important design dimension, enabling systems to tailor response formats to users’ psychological distance.

Distracted driving refers to multisensory integration and attention shifts between attentional driving and different interferences from different modalities, including visual and auditory stimuli. Here, we compared the behavioral performance with interacting multisensory distractors during attentional driving. Then, the independent component analysis (ICA) and event-related spectral perturbation (ERSP) were applied to investigate the neural oscillation changes. The behavioral results showed that the response times (RTs) increased when distractors appeared in response to attentional driving. Moreover, the RTs were longer when the distractor interference was presented in the auditory modality compared with the visual modality. Eye movement intervals showed shorter tracking saccades under distractor interference. These results may indicate that attentional driving performance was impaired under the exposure to multisensory distractor interference. The ERSPs under visual and auditory distraction exposure showed decreased beta power in the frontal area, increased theta and delta power in the central area, and decreased alpha power in the parietal area. During this process, distracted driving under cross-modal sensory interference required more neural oscillation involvement. Moreover, the visual modality showed increased gamma power in the frontal, central, parietal and occipital areas, while the auditory modality showed decreased gamma power in the frontal area, indicating that auditory interference could intervene in top-down attentional processing.

Fatigue is one problem with driving as it can lead to difficulties with sustaining attention, behavioral lapses, and a tendency to ignore vital information or operations. In this research, we explore multimodal physiological phenomena in response to driving fatigue through simultaneous functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) recordings with the aim of investigating the relationships between hemodynamic and electrical features and driving performance. Sixteen subjects participated in an event-related lane-deviation driving task while measuring their brain dynamics through fNIRS and EEGs. Three performance groups, classified as Optimal, Suboptimal, and Poor, were defined for comparison. From our analysis, we find that tonic variations occur before a deviation, and phasic variations occur afterward. The tonic results show an increased concentration of oxygenated hemoglobin (HbO2) and power changes in the EEG theta, alpha, and beta bands. Both dynamics are significantly correlated with deteriorated driving performance. The phasic EEG results demonstrate event-related desynchronization associated with the onset of steering vehicle in all power bands. The concentration of phasic HbO2 decreased as performance worsened. Further, the negative correlations between tonic EEG delta and alpha power and HbO2 oscillations suggest that activations in HbO2 are related to mental fatigue. In summary, combined hemodynamic and electrodynamic activities can provide complete knowledge of the brain’s responses as evidence of state changes during fatigue driving.

Inherent fuzzy entropy is an objective measurement of electroencephalography (EEG) complexity reflecting the robustness of brain systems. In this study, we present a novel application of multiscale relative inherent fuzzy entropy using repetitive steady-state visual evoked potentials (SSVEPs) to investigate EEG complexity change between two migraine phases, i.e., interictal (baseline) and preictal (before migraine attacks) phases. We used a wearable headband EEG device with O1, Oz, O2, and Fpz electrodes to collect EEG signals from 80 participants [40 migraine patients and 40 healthy controls (HCs)] under the following two conditions: During resting state and SSVEPs with five 15-Hz photic stimuli. We found a significant enhancement in occipital EEG entropy with increasing stimulus times in both HCs and patients in the interictal phase, but a reverse trend in patients in the preictal phase. In the 1st SSVEP, occipital EEG entropy of the HCs was significantly lower than that of patents in the preictal phase (FDRadjustedp <; 0.05). Regarding the transitional variance of EEG entropy between the 1st and 5th SSVEPs, patients in the preictal phase exhibited significantly lower values than patients in the interictal phase (FDR-adjustedp <; 0.05). Furthermore, in the classification model, the AdaBoost ensemble learning showed an accuracy of 81 ± 6% and area under the curve of 0.87 for classifying interictal and preictal phases. In contrast, there were no differences in EEG entropy among groups or sessions by using other competing entropy models, including approximate entropy, sample entropy, and fuzzy entropy on the same dataset. In conclusion, inherent fuzzy entropy offers novel applications in visual stimulus environments and may have the potential to provide a preictal alert to migraine patients.

It is believed that failures of people's reaction to emergencies occurred during driving are closely related to the inhibitory mechanism of brain's operations. To investigate the role of this function in emergency driving, two virtual realistic driving conditions based on stop signal task were designed and time limitation was manipulated to increase the stress in one condition. Sixteen subjects with behavioral encephalography recordings were collected and analyzed. By comparing successful and unsuccessful stop trials with event-related spectral perturbation analysis, δ and θ band power increases in frontal and central areas are correlated with driving inhibitory control of the brain. Moreover, β and y band power in frontal and central areas showed more increases upon stress condition. Time pressure in driving could adjust the operation of brain's inhibition control, to benefit the people's reactive ability upon emergency.

Prior studies have evaluated various designs for audio notifications. However, calls for more in-depth research on how such notifications work, especially at the level of users' cognitive states, have gone unanswered; and studies evaluating audio notifications with large numbers of participants in multiple environments have been rare. This study conducted an electroencephalography study (N=20) and an online study (N=967) to enhance understandings of how three musical parameters - melody (simple, complex), pitch (high, low), and tempo (fast, slow) - influenced users' cognition and behaviors. There are eight different notifications with different combinations of these parameters. The online study analyzed the effects of user-specific and environmental information on users' behaviors while they listened to these notifications. The results revealed that tempo and pitch have the main effect on the speed and strength (accuracy) of users' cognition and behaviors. The users' characteristics and environments influenced the effects of these musical parameters.

Auditory alarms that repeatedly interrupt users until they react are common, especially in the context of alarms. However, when an alarm repeats, our brains habituate to it and perceive it less and less, with reductions in both perception and attention-shifting: a phenomenon known as the repetition-suppression effect (RS). To retain users' perception and attention, this paper proposes and tests the use of pitch- and intensity-modulated alarms. Its experimental findings suggest that the proposed modulated alarms can reduce RS, albeit in different patterns, depending on whether pitch or intensity is the focus of the modulation. Specifically, pitch-modulated alarms were found to reduce RS more when the number of repetitions was small, while intensity-modulated alarms reduced it more as the number of repetitions increased. Based on these results, we make several recommendations for the design of improved repeating alarms, based on which modulation approach should be adopted in various situations.

Auditory saliency is an important mechanism that helps humans extract relevant information from environments. Audio notifications of mobile devices with high saliency can increase users' receptivity, yet overly high saliency could cause annoyance. Accurately measuring auditory saliency of a notification is critical for evaluating its usability. Previous studies adopted behavioral methods. However, their results may not accurately reflect auditory saliency as humans' perception of auditory saliency often involves complicated cognitive processes. Thus, we propose an electroencephalography (EEG)-based approach that can complement behavioral studies to provide a more nuanced analysis of auditory saliency. We evaluated our method by conducting an EEG experiment that measured the mismatch negativity and P3a of the sounds in realistic scenarios. We also conducted a behavioral experiment to link the EEG-based method with the behavioral method. The results suggested that EEG can provide detailed information about how human perceive auditory saliency and complement the behavioral measures.

Electroencephalogram (EEG) signals are frequently used in brain-computer interfaces (BC!s), but they are easily contaminated by artifacts and noise, so preprocessing must be done before they are fed into a machine learning algorithm for classification or regression. Spatial filters have been widely used to increase the signal-to-noise ratio of EEG for BC! classification problems, but their applications in BC! regression problems have been very limited. This paper proposes two common spatial pattern (CSP) filters for EEG-based regression problems in BC!, which are extended from the CSP filter for classification, by using fuzzy sets. Experimental results on EEG-based response speed estimation from a large-scale study, which collected 143 sessions of sustained-attention psychomotor vigilance task data from 17 subjects during a 5-month period, demonstrate that the two proposed spatial filters can significantly increase the EEG signal quality. When used in LASSO and k-nearest neighbors regression for user response speed estimation, the spatial filters can reduce the root-mean-square estimation error by 10.02-19.77%, and at the same time increase the correlation to the true response speed by 19.39-86.47%.

© 2017 IEEE. Brain-Computer interface (BCI) which aims at enabling users to perform tasks through their brain waves has been a feasible and worth developing solution for growing demand of healthcare. Current proposed BCI systems are often with lower applicability and do not provide much help for reducing burdens of users because of the time-consuming preparation required by adopted wet sensors and the shortage of provided interactive functions. Here, by integrating a state visually evoked potential (SSVEP)-based BCI system and a robotic eating assistive system, we propose a non-invasive wireless steady state visually evoked potential (SSVEP)-based BCI eating assistive system that enables users with physical disabilities to have meals independently. The analysis compared different methods of classification and indicated the best method. The applicability of the integrated eating assistive system was tested by an Amyotrophic Lateral Sclerosis (ALS) patient, and a questionnaire reply and some suggestion are provided. Fifteen healthy subjects engaged the experiment, and an average accuracy of 91.35%, and information transfer rate (ITR) of 20.69 bit per min are achieved. For online performance evaluation, the ALS patient gave basic affirmation and provided suggestions for further improvement. In summary, we proposed a usable SSVEP-based BCI system enabling users to have meals independently. With additional adjustment of movement design of the robotic arm and classification algorithm, the system may offer users with physical disabilities a new way to take care of themselves.

Graphic icons play an increasingly important role in interface design due to the proliferation of digital devices in recent years. Their ability to express information in a universal fashion allows us to immediately interact with new applications, systems, and devices. Icons can, however, cause user confusion and frustration if designed poorly. Several studies have evaluated icons using behavioral-performance metrics such as reaction time as well as self-report methods. However, determining the usability of icons based on behavioral measures alone is not straightforward, because users' interpretations of the meaning of icons involve various cognitive processes and perceptual mechanisms. Moreover, these perceptual mechanisms are affected not only by the icons themselves, but by usage scenarios. Thus, we need a means of sensitively and continuously measuring users' different cognitive processes when they are interacting with icons. In this study, we propose an EEG-based approach to icon evaluation, in which users' EEG signals are measured in multiple usage scenarios. Based on a combination of EEG and behavioral results, we provide a novel interpretation of the participants' perception during these tasks, and identify some important implications for icon design.

Motion sickness (MS) is a common experience of travelers. To provide insights into brain dynamics associated with MS, this study recruited 19 subjects to participate in an electroencephalogram (EEG) experiment in a virtual-reality driving environment. When riding on consecutive winding roads, subjects experienced postural instability and sensory conflict between visual and vestibular stimuli. Meanwhile, subjects rated their level of MS on a six-point scale. Independent component analysis (ICA) was used to separate the filtered EEG signals into maximally temporally independent components (ICs). Then, reduced logarithmic spectra of ICs of interest, using principal component analysis, were decomposed by ICA again to find spectrally fixed and temporally independent modulators (IMs). Results demonstrated that a higher degree of MS accompanied increased activation of alpha (r = 0.421) and gamma (r =0.478) IMs across remote-independent brain processes, covering motor, parietal and occipital areas. This co-modulatory spectral change in alpha and gamma bands revealed the neurophysiological demand to regulate conflicts among multi-modal sensory systems during MS.

Research has indicated that fatigue is a critical factor in cognitive lapses because it negatively affects an individual's internal state, which is then manifested physiologically. This study explores neurophysiological changes, measured by electroencephalogram (EEG), due to fatigue. This study further demonstrates the feasibility of an online closed-loop EEG-based fatigue detection and mitigation system that detects physiological change and can thereby prevent fatigue-related cognitive lapses. More importantly, this work compares the efficacy of fatigue detection and mitigation between the EEG-based and a nonEEG-based random method. Twelve healthy subjects participated in a sustained-attention driving experiment. Each participant's EEG signal was monitored continuously and a warning was delivered in real-time to participants once the EEG signature of fatigue was detected. Study results indicate suppression of the alpha- and theta-power of an occipital component and improved behavioral performance following a warning signal; these findings are in line with those in previous studies. However, study results also showed reduced warning efficacy (i.e. increased response times (RTs) to lane deviations) accompanied by increased alpha-power due to the fluctuation of warnings over time. Furthermore, a comparison of EEG-based and nonEEG-based random approaches clearly demonstrated the necessity of adaptive fatigue-mitigation systems, based on a subject's cognitive level, to deliver warnings. Analytical results clearly demonstrate and validate the efficacy of this online closed-loop EEG-based fatigue detection and mitigation mechanism to identify cognitive lapses that may lead to catastrophic incidents in countless operational environments.

Online learning is increasingly prevalent as an option for self-learning and as a resource for instructional design. Prerecorded video is currently the main medium of online education content delivery and instruction; this affords asynchronicity and flexibility, and enables the dissemination of lecture content in a distributed and scalable manner. However, the same properties may impede learners' engagement due to the lack of social interaction and peer support. In this paper, we propose a time-anchored commenting interface to allow online learners who watch the same video clips to exchange comments on them. Comments left by previous learners at specific time points of a video are displayed to new learners when they watch the same video and reach those time points. We investigated how the display of time-anchored comments (dynamic or static) and type of comments (content-related or social-oriented) influenced users' perceived engagement, perceived social interactivity, and learning outcomes. Our results show that dynamically displaying time-anchored comments can indeed enhance learners' perceived social interactivity. Moreover, the content of comments would further affect learners' intention of commenting. Based on our findings, we make various recommendations for the improvement of social interaction and learning experience in online education.

Audio notifications are an important means of prompting users of electronic products. Although useful in most environments, audio notifications are ineffective in certain situations, especially against particular auditory backgrounds or when the user is distracted. Several studies have used behavioral performance to evaluate audio notifications, but these studies failed to achieve consistent results due to factors including user subjectivity and environmental differences; thus, a new method and more objective indicators are necessary. In this study, we propose an approach based on electroencephalography (EEG) to evaluate audio notifications by measuring users' auditory perceptual responses (mismatch negativity) and attention shifting (P3a). We demonstrate our approach by applying it to the usability testing of audio notifications in realistic scenarios, such as users performing a major task amid ambient noises. Our results open a new perspective for evaluating the design of the audio notifications.