Before fully autonomous driving technology is developed, drivers are not free from the responsibility of Take-Over Request (TOR). Even with level 5 automation, a driver still can play a role as a final decision maker for various driving and non-driving functions, even though most of the tasks will be conducted by the vehicle. In this regard, the performance of the driver’s reaction to the request from the car is crucial in autonomous driving throughout and it is highly dependent on the driver’s attention. It is important to understand the state of the driver during autonomous driving and utilize this information in the driver-vehicle interaction system to enhance safety. Accordingly, it is important to extract features from information about the driver and the surrounding environment in order to increase the accuracy of the model that predicts the driver’s state. This paper aims to examine a method of predicting an attentional status based on the driver’s emotional state and explore the possibility of applying brain-computer interaction (BCI) with consideration of the driver’s type to increase accuracy. As a result, the prediction model for attentional state with emotional information was developed, and the driver’s characteristics and types were investigated. Based on the results, a driver-vehicle interaction system was proposed for the context of the autonomous vehicle in the future. Although the prediction model for attentional status was not powerful, it can be developed by considering more critical features such as driver’s characteristics and types. In addition, it can be used as a constraint for effective interaction when a driver uses the BCI system to deliver his/her decision to a vehicle.

The rise of ubiquitous computing has led to the rapid development of intelligent systems such as augmented reality (AR). AR is being employed for various usage including common tasks such as web browsing, navigating and controlling other internet of things (IoT) devices. In these types of system usage, similar to mobile phones, the presentation of notifications is inevitable. However, studies with regards to the effective delivery of notification signals within the AR environment is still lacking. Thus, the present study aims to investigate the effects of different output modalities (visual, auditory and multimodal) for presenting notifications during simple search and selection tasks within the AR system. Twenty individuals participated in a user study where they were asked to perform a simple search and selection task while using the AR device. Participants were asked to confirm the notifications that were presented at random in each trial. Results of the user study showed that multimodal notification signals (both visual and auditory) yielded a higher notification recognition rate and were rated as the most appropriate modality and most preferred among the conditions tested. Also, it was found that speech input modality was used more frequently in response to notifications, regardless of the output type. Several design implications were derived from the findings of the study.

Efficient visual search and accurate target detection are crucial during flight operations. Despite the continuous effort to improve cockpit displays, visual complexity remains to be a problem that affects pilots’ search and detection performance. This has become more evident with the integration of high fidelity and information loaded cockpit displays of the fifth-generation fighter aircraft. One known but often overlooked solution to mitigate the problem with visual complexity is by decluttering. This study aims to provide an objective definition of visual complexity in cockpit displays and investigate its effects on visual search time and target detection accuracy. Moreover, it also aims to compare the visual search and target detection performance across different decluttering methods. Lastly, it intends to propose threat-based criteria for decluttering information in cockpit displays. The present study is divided into three sections, each addressing one of the three main objectives of this research. In study 1, visual search time and target detection accuracy were compared across three conditions that consist of displays with varying levels of visual complexity (low, medium and high). Visual complexity is operationally defined as the combination of different features (color, shape and orientation) and its variations (e.g., color: red, green, white). In study 2, differences in visual search time and target detection accuracy across four decluttering methods (dimming, dotting, small sizing and removal) and between groups (pilot and non-pilot) were analyzed. In study 3, through AHP analysis a threat scoring equation was proposed and tested. The results of the first study suggest that increased visual complexity leads to an increase in visual search time and more target detection errors. On the other hand, findings from the latter study revealed that there is a significant difference in the visual search time and target detection accuracy between different decluttering methods. Moreover, group differences were found when the threat scoring equation was applied to the targets. The results of the present study could provide inferences on how to alleviate the adverse effects of visual complexity through the utilization of effective decluttering methods, which in turn could further improve pilots’ overall performance.

Virtual reality (VR) is receiving attention enough to be considered as its revival age in both industrial and academic field. Since VR systems have various types of interaction with users and new types of interaction are constantly being developed, various studies investigating user experience (UX) of VR systems are continuously needed. However, there is still a lack of research on the taxonomy that can recognize the main characteristics of VR system at a glance by reflecting the influencing factors of UX. Therefore, we collected and reviewed the research related to the UX evaluation of the VR system in order to identify the current research status and to suggest future research direction. To achieve this, a systematic review was conducted on UX studies for VR, and taxonomies of VR system including influencing factors of UX were proposed. A total of 393 unique articles were collected, and 65 articles were selected to be reviewed via Systematic Reviews and Meta-Analyses methodology. The selected articles were analyzed according to predefined taxonomies. As a result, current status of research can be identified base on the proposed taxonomies. Besides, issues related to VR devices and technology, and research method for future research directions can be suggested.

This study explores TV viewers’ user experience (UX) of the conversational agent (CA) assisted interactions while watching TV. In the human-computer interaction field, the user experience is primarily measured through performance and subjective self-reported data. But TV is an entertainment medium, the user experience of TV viewers should be determined based on how much the TV viewers enjoyed and immersed, rather than productivity or performance. It is also necessary to collect objective data as well as subjective self-report results of emotional experiences. Although the CA is becoming increasingly common in everyday life, the user satisfaction is still not high. However, this is mainly a result of evaluation in the performance aspect, and few studies have been carried out on the overall user experience including emotional aspects. To comparatively analyze the user experience with the CA interface and remote control unit (RCU) interface, we adopted physiological measurements for objective data as well as self-report questionnaires. Among the physiological measurement results, skin conductance (SC) differed between CA and RCU interactions. SC was high at the beginning of the CA interaction but decreased over time. In RCU sessions, SC maintained a constant value or rather increased. In the self-reporting results, the ergonomic quality of the RCU was higher than that of the CA. However, CA was evaluated more positively in terms of hedonic quality. CA had a greater effect on the overall attractiveness while having more emotional appeal. Additionally, a partial correlation between objective data and self-report results could be observed. In sum, subjective self-reports showed that CA provides a more positive TV UX with regard to emotional aspects. Physiological measurement results provide objective data for further understanding user experience by showing specific differences in cognitive involvement and emotional response changes during user interactions.

With growing interest in the topic of smart computing and context-aware services, identifying driver's intention has become important in automotive industry. Among existing studies on driver assistance systems, few studies have used the concept of the levels of processing (LOP) to understand driver's information processing. This paper introduces experimental and computational studies that connect human behavior data to internal goal of a driver when an in-vehicle task involves visual search. In the case of consuming information displayed on a car instrument cluster, we considered two levels of information processing, perceptual and cognitive. Through an empirical study, we observed different human oculomotor behaviors that depend on the LOP by evaluating the reaction time (RT) and eye movement patterns. Further, we suggested different ways of information processing that can be represented as different routes in the underlying cognitive architecture of a queueing network. Simulation study demonstrated that trends in simulated oculomotor behavior were similar to those observed in the experiment, that is, RT at the cognitive LOP was shorter than that at the perceptual LOP, while eyes fixated for shorter duration and with lower frequency. In terms of application, this study reveals the possibility of using human-generated data for evaluating the innate purposes of drivers. Together with further development of sensors and invasive computing, the approach proposed in this paper could assist the realization of cognitive cars by understanding drivers' intent using eye tracking technology.

Eye mouse devices are operated by ocular movements, which have proven to be useful for disabled people who find eye control to be the easiest input method. However, there are few studies comparing the two typical click methods for eye mouse (blink and dwell) in identical experimental conditions for general object selection operations. Especially, investigation on task performance (effectiveness and efficiency) of selecting objects is not sufficient. Therefore, this study aims to investigate the relationships between task performance of selecting a stimulus and independent variables with comparing the two different click methods. To investigate the relationship, movement distance, eye input method, stimulus position, and stimulus size were considered as independent variables. Results of the blink method revealed a higher efficiency than the dwell method, particularly as the stimulus size increased. Also, both efficiency and effectiveness of stimuli in upper parts of screen was higher than for those in lower ones. Additionally, efficiency was remarkably higher when the stimulus size increased. The results of this study may be used as guidelines to design interfaces using an eye mouse device. Especially, it can be applied to various conditions of selecting an object in the working environment using the eye mouse.

This study aims to analyze and categorize user experience (UX) of smartphones by utilizing social media data (Twitter). Social media (e.g., Facebook, Twitter) can be helpful for observing natural UX through the users’ words. It is a potentially valuable source of data that can be used to examine the thoughts of millions of people. To gather UXs of smartphones, mining social media data (Twitter) techniques were used. Collected UXs were categorized according to the product smartness and the relationship between the product smartness and UXs were identified. A total of 19,288 tweets involving the term “smartphone” were collected from 2014.06.01 to 2014.08.31. Among these, a total of 699 tweets were related to UXs of smartphones. In addition, 478 tweets were categorized according to the five dimensions of product smartness (Autonomy, Adaptability, Multi-functionality, Connectivity, and Personalization). Results found that many satisfactory experiences for all dimensions, but there were unsatisfactory experiences associated with the multi-functionality and connectivity. Findings suggest that mining techniques can be used to gather and analyze UX effectively and efficiently. Also, the results of this study can be helpful for understanding user’s implicit needs concerning smartphones, and provide valuable insights for developing smartphones.

With increasing recognition of user experience (UX), it became important to identify UX issues from customers’ opinions expressed in a form of text that is unstructured qualitative data. Since reliability of qualitative data has little been investigated, it requires much time and effort for researchers to define meanings and values from the qualitative data. Therefore, this article proposes a novel approach to measuring the stability of textual data by adopting semantic network analysis, as shown by our analysis on in-depth interview transcripts and online customer review data. Among the semantic networks generated from text, subnetworks were sampled from the original networks until the representativeness of each sample size was determined. Then, similarities between the subnetworks and the original network were calculated by applying a correlation analysis to determine whether stability occurs. The results indicated that variables of word frequency and network-level statistics affect stability of the network. The suggested method will be able to assist in identifying UX issues.

This article reviewed both studies on general smart car technologies and human–computer interaction (HCI)/human–vehicle interaction studies that were published in journals and conferences so that the current status of research can be identified and future research directions can be suggested. Furthermore, previous studies on elderly drivers were reviewed, as these drivers could be the most vulnerable social group in terms of new technology acceptance. A total of 257 articles for HCI research and 45 articles for elderly drivers were selected and reviewed from 11,267 collected articles (2010–2014). According to the results, most articles were mainly related to safety and adaptive features (e.g., driver’s state recognition, vehicle surrounding monitoring, driver action-suggestion), and infotainment research in terms of HCI (e.g., information technology devices–vehicle interaction, vehicle–vehicle interaction) was relatively insufficient despite its high research demand. According to the results of the literature review and technological trends analysis based on previous technical road maps, from HCI/human factors engineering (HFE) perspectives, research related to “Assistance systems,” “Physiological & mental state recognition,” “Position sensor technology,” “Behavior recognition,” and “Infotainment” was suggested to HCI/HFE researchers for further research. In particular, HCI/HFE researchers need to focus on research on acceptable levels of automation, observing new driving behaviors, investigation of driver characteristics to develop personalized services, and new technology acceptance to develop and improve smart cars in the future.

In the last decade, the research of the usability of mobile phones has been a newly evolving area with few established methodologies and realistic practices that ensure capturing usability in evaluation. Thus, there exists growing demand to explore appropriate evaluation methodologies that evaluate the usability of mobile phones quickly as well as comprehensively. This study aims to develop a task-based usability checklist based on heuristic evaluations in views of mobile phone user interface (UI) practitioners. A hierarchical structure of UI design elements and usability principles related to mobile phones were developed and then utilized to develop the checklist. To demonstrate the practical effectiveness of the proposed checklist, comparative experiments were conducted on the usability checklist and usability testing. The majority of usability problems found by usability testing and additional problems were discovered by the proposed checklist. It is expected that the usability checklist proposed in this study could be used quickly and efficiently by usability practitioners to evaluate the mobile phone UI in the middle of the mobile phone development process.

This article reports the results of a survey conducted in 2003 of 184 information technology (IT) development practitioners and 90 user interface (UI)/usability practitioners. The survey covered a broad range of issues including the respondents' profiles, current development environment, usability and user-centered design (UCD) adoption issues in Korean IT development environments, overall assessment of UCD/usability, and the most widely used methods and techniques. The results reveal various UCD/usability practices and problems in the IT industry. For both development practitioners and UI/usability practitioners, the most serious problems encountered in a project are the differences between output and customer requirements, as well as further requirements from customer, delays in a project schedule, and overages on project budgets. The degree of recognition of usability/UCD is higher than expected, but UCD/usability is not yet fully employed in the project, because 42% of respondents used usability/UCD. Profiles from UI/usability practitioners and projects show a relatively weak position and short HCI tradition in the Korean IT industry. This study is expected to provide both an empirical basis for usability and UCD planning, training, adoption, and execution in the IT industry of Korea and an important information source for many HCI practitioners outside Europe and the United States who struggle to introduce usability into their IT industries.

Hand posture and force, which define aspects of the way an object is grasped, are features of robotic manipulation. A means for specifying these grasping "flavors" has been developed that uses an instrumented glove equipped with joint and force sensors. The new grasp specification system will be used at the Pennsylvania State University (Penn State) in a Virtual Reality based Point-and-Direct (VR-PAD) robotics implementation. Here, an operator gives directives to a robot in the same natural way that human may direct another. Phrases such as "put that there" cause the robot to define a grasping strategy and motion strategy to complete the task on its own. In the VR-PAD concept, pointing is done using virtual tools such that an operator can appear to graphically grasp real items in live video. Rather than requiring full duplication of forces and kinesthetic movement throughout a task as is required in manual telemanipulation, hand posture and force are now specified only once. The grasp parameters then become object flavors. The robot maintains the specified force and hand posture flavors for an object throughout the task in handling the real workpiece or item of interest. In the Computer integrated Manufacturing (CIM) Laboratory at Penn State, hand posture and force data were collected for manipulating bricks and other items that require varying amounts of force at multiple pressure points. The feasibility of measuring desired grasp characteristics was demonstrated for a modified Cyberglove impregnated with Force-Sensitive Resistor (FSR) (pressure sensors in the fingertips. A joint/force model relating the parameters of finger articulation and pressure to various lifting tasks was validated for the instrumented "wired" glove. Operators using such a modified glove may ultimately be able to configure robot grasping tasks in environments involving hazardous waste remediation, flexible manufacturing, space operations and other flexible robotics applications. In each case, the VR-PAD approach will finesse the computational and delay problems of real-time multiple-degree-of-freedom force feedback telemanipulation.