Multi-view visualizations have gained prominence for their ability to simultaneously present multiple perspectives in a single display, aiding users in making informed decisions. Although several tools have been developed to facilitate the design of multi-view visualizations, there is a lack of support for effective color design in these systems. Thoughtful color design can enhance readability and provide cues, while careless color choices may lead to confusion. Moreover, human short-term memory imposes constraints on the number of colors that can be effectively employed in a multi-view visualization. To address these challenges, we introduce MVcolor, a color encoding recommendation system designed to maintain color encoding consistency and ensure adequate color discriminability within the constraints of human short-term memory. Our approach employs a unified color scheme across the entire multi-view visualization and groups views and visual objects based on their visual and semantic similarities. We conducted user studies to evaluate the effectiveness of our system and demonstrate its ability to improve color encoding in multi-view visualizations.

Rekindling old social bonds with individuals who were once a part of our lives but have since faded away is crucial for our well-being. Such connections with dormant ties help us overcome loneliness and provide social support. Recently, virtual reality (VR) emerged as a promising tool for facilitating social interactions, such as online gatherings for formal or casual activities. VR can offer immersive and shared experiences, facilitating genuine connections between people. This provides a unique advantage over traditional computer-mediated communication methods. However, while prior research has explored how VR can aid in forming new social connections, its potential to reconnect dormant ties is largely unexplored. This paper aims to bridge this gap by examining how different features of VR, specifically avatar appearance and virtual environments, influence reactivations of dormant ties. We conducted an experiment involving 24 dyads to investigate the effect of different avatar-self similarities and virtual environments on the perceptions and interactions between dormant ties. Our findings indicate that avatars resembling oneself and dormant ties promote social closeness. Familiar virtual environments evoke shared memories, while unfamiliar ones stimulate more conversations. We discuss the impact of VR features on reconnecting dormant ties and provide implications for re-connecting relationships in VR.

Adjusting video playback speed during extensive viewing is crucial for English-as-a-Second-Language (ESL) learners to enhance their learning experience. Since existing research suggests that abrupt speed changes might negatively impact the viewing experience, several novel speed-adjustment systems have been proposed to provide adaptive and optimal video playback speed for learners. However, empirical evidence is still sparse on whether gradual adjustments truly offer a superior experience compared to immediate changes. To delve into this, we conducted a study with 32 ESL participants, comparing direct and gradual adjustments on flow state, cognitive load, and behavioral measures. Employing both objective metrics, such as pupil diameter, and subjective feedback from surveys, our results strongly favor the gradual method. It not only enhanced flow state and video comprehension but was also less obtrusive to learners. These findings underscore the advantages of gradual speed adjustment for ESL learners, offering insights for the design of next-generation speed-adjustment systems.

Abstract Interface icons are prevalent in various digital applications. Due to limited time and budgets, many designers rely on informal evaluation, which often results in poor usability icons. In this paper, we propose a unique human‐in‐the‐loop framework that allows our target users, that is novice and professional user interface (UI) designers, to improve the usability of interface icons efficiently. We formulate several usability criteria into a perceptual usability function and enable users to iteratively revise an icon set with an interactive design tool, EvIcon. We take a large‐scale pre‐trained joint image‐text embedding (CLIP) and fine‐tune it to embed icon visuals with icon tags in the same embedding space (IconCLIP). During the revision process, our design tool provides two types of instant perceptual usability feedback. First, we provide perceptual usability feedback modelled by deep learning models trained on IconCLIP embeddings and crowdsourced perceptual ratings. Second, we use the embedding space of IconCLIP to assist users in improving icons' visual distinguishability among icons within the user‐prepared icon set. To provide the perceptual prediction, we compiled IconCEPT10K, the first large‐scale dataset of perceptual usability ratings over 10,000 interface icons, by conducting a crowdsourcing study. We demonstrated that our framework could benefit UI designers' interface icon revision process with a wide range of professional experience. Moreover, the interface icons designed using our framework achieved better semantic distance and familiarity, verified by an additional online user study.

Online platforms commonly collect and display user-generated information to support subsequent users’ decision-making. However, studies have noticed that presenting collective information can pose social influences on individuals’ opinions and alter their preferences accordingly. It is essential to deepen understanding of people’s preferences when exposed to others’ opinions and the underlying cognitive mechanisms to address potential biases. Hence, we conducted a laboratory study to investigate how products’ ratings and reviews influence participants’ stated preferences and cognitive responses assessed by their Electroencephalography (EEG) signals. The results showed that social ratings and reviews could alter participants’ preferences and affect their status of attention, working memory, and emotion. We further conducted predictive analyses to show that participants’ Electroencephalography-based measures can achieve higher power than behavioral measures to discriminate how collective information is displayed to users. We discuss the design implications informed by the results to shed light on the design of collective rating systems.

Prior studies have evaluated various designs for audio notifications. However, calls for more in-depth research on how such notifications work, especially at the level of users' cognitive states, have gone unanswered; and studies evaluating audio notifications with large numbers of participants in multiple environments have been rare. This study conducted an electroencephalography study (N=20) and an online study (N=967) to enhance understandings of how three musical parameters - melody (simple, complex), pitch (high, low), and tempo (fast, slow) - influenced users' cognition and behaviors. There are eight different notifications with different combinations of these parameters. The online study analyzed the effects of user-specific and environmental information on users' behaviors while they listened to these notifications. The results revealed that tempo and pitch have the main effect on the speed and strength (accuracy) of users' cognition and behaviors. The users' characteristics and environments influenced the effects of these musical parameters.

Auditory alarms that repeatedly interrupt users until they react are common, especially in the context of alarms. However, when an alarm repeats, our brains habituate to it and perceive it less and less, with reductions in both perception and attention-shifting: a phenomenon known as the repetition-suppression effect (RS). To retain users' perception and attention, this paper proposes and tests the use of pitch- and intensity-modulated alarms. Its experimental findings suggest that the proposed modulated alarms can reduce RS, albeit in different patterns, depending on whether pitch or intensity is the focus of the modulation. Specifically, pitch-modulated alarms were found to reduce RS more when the number of repetitions was small, while intensity-modulated alarms reduced it more as the number of repetitions increased. Based on these results, we make several recommendations for the design of improved repeating alarms, based on which modulation approach should be adopted in various situations.

Auditory saliency is an important mechanism that helps humans extract relevant information from environments. Audio notifications of mobile devices with high saliency can increase users' receptivity, yet overly high saliency could cause annoyance. Accurately measuring auditory saliency of a notification is critical for evaluating its usability. Previous studies adopted behavioral methods. However, their results may not accurately reflect auditory saliency as humans' perception of auditory saliency often involves complicated cognitive processes. Thus, we propose an electroencephalography (EEG)-based approach that can complement behavioral studies to provide a more nuanced analysis of auditory saliency. We evaluated our method by conducting an EEG experiment that measured the mismatch negativity and P3a of the sounds in realistic scenarios. We also conducted a behavioral experiment to link the EEG-based method with the behavioral method. The results suggested that EEG can provide detailed information about how human perceive auditory saliency and complement the behavioral measures.

Abstract MOOCs (Massive Open Online Courses) are increasingly prevalent as an online educational resource open to everyone and have attracted hundreds of thousands learners enrolling these online courses. At such scale, there is potentially rich information of learners' behaviors embedded in the interactions between learners and videos that may help instructors and content producers adjust the instructions and refine the online courses. However, the lack of tools to visualize information from interactive data, including messages left to the videos at particular timestamps as well as the temporal variations of learners' online participation and perceived experience, has prevented people from gaining more insights from video‐watching logs. In this paper, we focus on extracting and visualizing useful information from time‐anchored comments that learners left to specific time points of the videos when watching them. Timestamps as a kind of metadata of messages can be useful to recover the interactive dynamics of learners occurring around the videos. Therefore, we present a visualization system to analyze and categorize time‐anchored comments based on topics and content types. Our system integrates visualization methods of temporal text data, namely ToPIN and ThemeRiver, which can help people understand the quality and quantity of online learners' feedback and their states of learning. To evaluate the proposed system, we visualized time‐anchored commenting data from two online course videos, and conducted two user studies participated by course instructors and third‐party educational evaluators. The results validate the usefulness of the approach and show how the quantitative and qualitative visualizations can be used to gain interesting insights around learners' online learning behaviors.

Graphic icons play an increasingly important role in interface design due to the proliferation of digital devices in recent years. Their ability to express information in a universal fashion allows us to immediately interact with new applications, systems, and devices. Icons can, however, cause user confusion and frustration if designed poorly. Several studies have evaluated icons using behavioral-performance metrics such as reaction time as well as self-report methods. However, determining the usability of icons based on behavioral measures alone is not straightforward, because users' interpretations of the meaning of icons involve various cognitive processes and perceptual mechanisms. Moreover, these perceptual mechanisms are affected not only by the icons themselves, but by usage scenarios. Thus, we need a means of sensitively and continuously measuring users' different cognitive processes when they are interacting with icons. In this study, we propose an EEG-based approach to icon evaluation, in which users' EEG signals are measured in multiple usage scenarios. Based on a combination of EEG and behavioral results, we provide a novel interpretation of the participants' perception during these tasks, and identify some important implications for icon design.

Online learning is increasingly prevalent as an option for self-learning and as a resource for instructional design. Prerecorded video is currently the main medium of online education content delivery and instruction; this affords asynchronicity and flexibility, and enables the dissemination of lecture content in a distributed and scalable manner. However, the same properties may impede learners' engagement due to the lack of social interaction and peer support. In this paper, we propose a time-anchored commenting interface to allow online learners who watch the same video clips to exchange comments on them. Comments left by previous learners at specific time points of a video are displayed to new learners when they watch the same video and reach those time points. We investigated how the display of time-anchored comments (dynamic or static) and type of comments (content-related or social-oriented) influenced users' perceived engagement, perceived social interactivity, and learning outcomes. Our results show that dynamically displaying time-anchored comments can indeed enhance learners' perceived social interactivity. Moreover, the content of comments would further affect learners' intention of commenting. Based on our findings, we make various recommendations for the improvement of social interaction and learning experience in online education.

Abstract Flow visualization is recognized as an essential tool for many scientific research fields and different visualization approaches are proposed. Several studies are also conducted to evaluate their effectiveness but these studies rarely examine the performance from the perspective of visual perception. In this paper, we aim at exploring how users’ visual perception is influenced by different 2D flow visualization methods. An eye tracker is used to analyze users’ visual behaviors when they perform the free viewing, advection prediction, flow feature detection, and flow feature identification tasks on the flow field images generated by different visualizations methods. We evaluate the illustration capability of five representative visualization algorithms. Our results show that the eye‐tracking‐based evaluation provides more insights to quantitatively analyze the effectiveness of these visualization methods.

Audio notifications are an important means of prompting users of electronic products. Although useful in most environments, audio notifications are ineffective in certain situations, especially against particular auditory backgrounds or when the user is distracted. Several studies have used behavioral performance to evaluate audio notifications, but these studies failed to achieve consistent results due to factors including user subjectivity and environmental differences; thus, a new method and more objective indicators are necessary. In this study, we propose an approach based on electroencephalography (EEG) to evaluate audio notifications by measuring users' auditory perceptual responses (mismatch negativity) and attention shifting (P3a). We demonstrate our approach by applying it to the usability testing of audio notifications in realistic scenarios, such as users performing a major task amid ambient noises. Our results open a new perspective for evaluating the design of the audio notifications.