Xingbo Wang 0001

dblp:40/4886-1 · DBLP profile ↗
← Back
28ranked-venue papers
5as first author
25since 2021 · last 2026
0000-0001-5693-1128ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 16 · 3 first-author · 15 since 2021Human-computer interaction and ubiquitous computing · 10 · 2 first-author · 9 since 2021Artificial intelligence and machine learning · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1
YearPublicationVenuePosition
2026 ConSearcher: Supporting Conversational Information Seeking in Online Communities with Member Personas
abstract
Many people browse online communities to learn from others’ experiences and opinions, e.g., for constructing travel plans. Conversational search powered by large language models (LLMs) could ease this information-seeking task, but it remains under-investigated within the online community. In this paper, we first conducted an exploratory study (N=10) that indicated the helpfulness of a classic conversational search tool and identified room for improvement. Then, we proposed ConSearcher, an LLM-powered tool with dynamically generated member personas based on user queries to facilitate conversational search in the community. In ConSearcher, users can clarify their interests by checking what a simulated member similar to them may ask and get responses from diverse members’ perspectives. A within-subjects study (N=27) showed that compared to two conversational search baselines, ConSearcher led to significantly higher information-seeking outcome and user engagement but raised concerns about over-personalization. We discuss implications for supporting conversational information seeking in online communities.
Xingbo Wang 0001, Qingyu Guo, Chuhan Shi, Zhenhui Peng
DIS4
2026 When the Chain Breaks: Interactive Diagnosis of LLM Chain-of-Thought Reasoning Errors
abstract
Abstract Current Large Language Models (LLMs), especially Large Reasoning Models, can generate Chain‐of‐Thought (CoT) reasoning traces to illustrate how they produce final outputs, thereby facilitating trust calibration for users. However, these CoT reasoning traces are usually lengthy and tedious, and can contain various issues, such as logical and factual errors, which make it difficult for users to interpret the reasoning traces efficiently and accurately. To address these challenges, we develop an error detection pipeline that combines external fact‐checking with symbolic formal logical validation to identify errors at the step level. Building on this pipeline, we propose ReasonDiag, an interactive visualization system for diagnosing CoT reasoning traces. ReasonDiag provides 1) an integrated arc diagram to show reasoning‐step distributions and error‐propagation patterns, and 2) a hierarchical node‐link diagram to visualize high‐level reasoning flows and premise dependencies. We evaluate Reason‐Diag through a technical evaluation for the error detection pipeline, two case studies, and user interviews with 16 participants. The results indicate that ReasonDiag helps users effectively understand CoT reasoning traces, identify erroneous steps, and determine their root causes.
Niruthikka Sritharan, Xiaolin Wen, Xingbo Wang 0001, Yong Wang 0021
Comput. Graph. Forum5
2026 EMINDS: Understanding User Behavior Progression for Mental Health Exploration on Social Media
abstract
Mental health is an urgent societal issue, and social scientists are increasingly turning to online mental health communities (OMHCs) to analyze user behavior data for early intervention. However, existing sequence mining techniques fall short of the urgent need to explore the behavior progression of different groups (e.g., recovery or deterioration groups) and track the potential long-term impact of behaviors on mental health status. To address this issue, we introduce EMINDS, a visual analytics system built on a novel automatic mining pipeline that extracts distinct behavior stages and assesses the potential impact of frequent stage patterns on mental health status over time. The system includes a set of interactive visualizations that summarize the meaning of each behavior stage and the evolution of different stage patterns. We feature a pattern-centric Sankey diagram to reveal contextual information about the impact of stage patterns on mental health, helping experts understand the specific changes in sequences before and after a stage pattern. We evaluated the effectiveness and usability of EMINDS through two case studies and expert interviews, which examined the potential stage patterns impacting long-term mental health by analyzing user behaviors on Reddit.
Rui Sheng, Yifang Wang 0001, Xingbo Wang 0001, Shun Dai, Qingyu Guo, Tai-Quan Peng, Huamin Qu, Dongyu Liu
IEEE Trans. Vis. Comput. Graph.3
2026 TrialCompass: Visual Analytics for Enhancing the Eligibility Criteria Design of Clinical Trials
abstract
Eligibility criteria play a critical role in clinical trials by determining the target patient population, which significantly influences the outcomes of medical interventions. However, current approaches for designing eligibility criteria have limitations to support interactive exploration of the large space of eligibility criteria. They also ignore incorporating detailed characteristics from the original electronic health record (EHR) data for criteria refinement. To address these limitations, we proposed TrialCompass, a visual analytics system integrating a novel workflow, which can empower clinicians to iteratively explore the vast space of eligibility criteria through knowledge-driven and outcome-driven approaches. TrialCompass supports history-tracking to help clinicians trace the evolution of their adjustments and decisions when exploring various forms of data (i.e., eligibility criteria, outcome metrics, and detailed characteristics of original EHR data) through these two approaches. This feature can help clinicians comprehend the impact of eligibility criteria on outcome metrics and patient characteristics, which facilitates systematic refinement of eligibility criteria. Using a real-world dataset, we demonstrated the effectiveness of TrialCompass in providing insights into designing eligibility criteria for septic shock and sepsis-associated acute kidney injury. We also discussed the research prospects of applying visual analytics to clinical trials.
Rui Sheng, Xingbo Wang 0001, Jiachen Wang 0001, Xiaofu Jin, Zhonghua Sheng, Suraj Rajendran, Huamin Qu, Fei Wang 0001
IEEE Trans. Vis. Comput. Graph.2
2025 Exploring Personalized Health Support through Data-Driven, Theory-Guided LLMs: A Case Study in Sleep Health
abstract
Despite the prevalence of sleep-tracking devices, many individuals struggle to translate data into actionable improvements in sleep health. Current methods often provide data-driven suggestions but may not be feasible and adaptive to real-life constraints and individual contexts. We present HealthGuru, a novel large language model-powered chatbot to enhance sleep health through data-driven, theory-guided, and adaptive recommendations with conversational behavior change support. HealthGuru's multi-agent framework integrates wearable device data, contextual information, and a contextual multi-armed bandit model to suggest tailored sleep-enhancing activities. The system facilitates natural conversations while incorporating data-driven insights and theoretical behavior change techniques. Our eight-week in-the-wild deployment study with 16 participants compared HealthGuru to a baseline chatbot. Results show improved metrics like sleep duration and activity scores, higher quality responses, and increased user motivation for behavior change with HealthGuru. We also identify challenges and design considerations for personalization and user engagement in health chatbots.
Xingbo Wang 0001, Janessa Griffith, Daniel A. Adler, Joey Castillo, Tanzeem Choudhury, Fei Wang 0001
CHI1
2025 POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models
abstract
Large language models (LLMs) have exhibited impressive abilities for multimodal content comprehension and reasoning with proper prompting in zero- or few-shot settings. Despite the proliferation of interactive systems developed to support prompt engineering for LLMs across various tasks, most have primarily focused on textual or visual inputs, thus neglecting the complex interplay between modalities in multimodal inputs. This oversight hinders the development of effective prompts that guide models’ multimodal reasoning processes by fully exploiting the rich context provided by multiple modalities. In this paper, we present POEM, a visual analytics system to facilitate efficient prompt engineering for steering the multimodal reasoning performance of LLMs. The system enables users to explore the interaction patterns across modalities at varying levels of detail for a comprehensive understanding of the multimodal knowledge elicited by various prompts. Through diverse recommendations of demonstration examples and instructional principles, POEM supports users in iteratively crafting and refining prompts to better align and enhance model knowledge with human insights. The effectiveness and efficiency of our system are validated through quantitative and qualitative evaluations with experts.
Jianben He, Xingbo Wang 0001, Shiyi Liu 0001, Guande Wu, Cláudio T. Silva, Huamin Qu
PacificVis2
2025 InsightLens: Augmenting LLM-Powered Data Analysis With Interactive Insight Management and Navigation
abstract
The proliferation of large language models (LLMs) has revolutionized the capabilities of natural language interfaces (NLIs) for data analysis. LLMs can perform multi-step and complex reasoning to generate data insights based on users' analytic intents. However, these insights often entangle with an abundance of contexts in analytic conversations such as code, visualizations, and natural language explanations. This hinders efficient recording, organization, and navigation of insights within the current chat-based LLM interfaces. In this paper, we first conduct a formative study with eight data analysts to understand their general workflow and pain points of insight management during LLM-powered data analysis. Accordingly, we introduce InsightLens, an interactive system to overcome such challenges. Built upon an LLM-agent-based framework that automates insight recording and organization along with the analysis process, InsightLens visualizes the complex conversational contexts from multiple aspects to facilitate insight navigation. A user study with twelve data analysts demonstrates the effectiveness of InsightLens, showing that it significantly reduces users' manual and cognitive effort without disrupting their conversational data analysis workflow, leading to a more efficient analysis experience.
Luoxuan Weng, Xingbo Wang 0001, Yingchaojie Feng, Haozhe Feng, Danqing Huang, Wei Chen 0001
IEEE Trans. Vis. Comput. Graph.2
2025 From Requirement to Solution: Unveiling Problem-Driven Design Patterns in Visual Analytics
abstract
Visual Analytics (VA) researchers frequently collaborate closely with domain experts to derive requirements and select appropriate solutions to fulfill these requirements. Despite strides made in exploring requirement and solution spaces, challenges persist due to the absence of guidance in the initial consideration space and the lack of shared problem-solving knowledge, often resulting in suboptimal solutions. To address these issues, we conducted an empirical study of VA research, with a focus on mapping the relations between requirement and solution spaces. Analyzing 220 VA papers, we formulate refined topologies for data, requirements, and solutions. We propose conceptualizing the connections between requirements, data, and solutions through knowledge graphs and utilizing solution paths to encapsulate fundamental problem-solving knowledge in visual analytics research. Through the integration of solution paths into a graph and analyzing their interconnections, we identified a subset of problem-driven design patterns that demonstrated the efficacy of our approach. By externalizing problem-solving knowledge and formulating problem-driven design patterns, our aim is to streamline the exploration of consideration space, facilitating the inclusion of "good" solutions, and establish a benchmark for shared design decisions among researchers and readers.
Shizhen Zhang, Xiaofeng Dou, Xingbo Wang 0001, Quan Li 0002
IEEE Trans. Vis. Comput. Graph.5
2024 RetAssist: Facilitating Vocabulary Learners with Generative Images in Story Retelling Practices
abstract
Reading and repeatedly retelling a short story is a common and effective approach to learning the meanings and usages of target words. However, learners often struggle with comprehending, recalling, and retelling the story contexts of these target words. Inspired by the Cognitive Theory of Multimedia Learning, we propose a computational workflow to generate relevant images paired with stories. Based on the workflow, we work with learners and teachers to iteratively design an interactive vocabulary learning system named RetAssist. It can generate sentence-level images of a story to facilitate the understanding and recall of the target words in the story retelling practices. Our within-subjects study (N=24) shows that compared to a baseline system without generative images, RetAssist significantly improves learners’ fluency in expressing with target words. Participants also feel that RetAssist eases their learning workload and is more useful. We discuss insights into leveraging text-to-image generative models to support learning tasks.
Qiaoyi Chen, Kaihui Huang, Xingbo Wang 0001, Xiaojuan Ma, Junkai Zhu, Zhenhui Peng
Conference on Designing Interactive Systems4
2024 LessonPlanner: Assisting Novice Teachers to Prepare Pedagogy-Driven Lesson Plans with Large Language Models
abstract
Preparing a lesson plan, e.g., a detailed road map with strategies and materials for instructing a 90-minute class, is beneficial yet challenging for novice teachers. Large language models (LLMs) can ease this process by generating adaptive content for lesson plans, which would otherwise require teachers to create from scratch or search existing resources. In this work, we first conduct a formative study with six novice teachers to understand their needs for support of preparing lesson plans with LLMs. Then, we develop LessonPlanner that assists users to interactively construct lesson plans with adaptive LLM-generated content based on Gagne’s nine events. Our within-subjects study (N = 12) shows that compared to the baseline ChatGPT interface, LessonPlanner can significantly improve the quality of outcome lesson plans and ease users’ workload in the preparation process. Our expert interviews (N = 6) further demonstrate LessonPlanner ’s usefulness in suggesting effective teaching strategies and meaningful educational resources. We discuss concerns on and design considerations for supporting teaching activities with LLMs.
Haoxiang Fan, Guanzheng Chen, Xingbo Wang 0001, Zhenhui Peng
UIST3
2024 ComPeer: A Generative Conversational Agent for Proactive Peer Support
abstract
Conversational Agents (CAs) acting as peer supporters have been widely studied and demonstrated beneficial for people’s mental health. However, previous peer support CAs either are user-initiated or follow predefined rules to initiate the conversations, which may discourage users to engage and build relationships with the CAs for long-term benefits. In this paper, we develop ComPeer, a generative CA that can proactively offer adaptive peer support to users. ComPeer leverages large language models to detect and reflect significant events in the dialogue, enabling it to strategically plan the timing and content of proactive care. In addition, ComPeer incorporates peer support strategies, conversation history, and its persona into the generative messages. Our one-week between-subjects study (N=24) demonstrates ComPeer ’s strength in providing peer support over time and boosting users’ engagement compared to a baseline user-initiated CA. We report users’ interaction patterns with ComPeer and discuss implications for designing proactive generative agents to promote people’s well-being.
Hongzheng Zhao, Xingbo Wang 0001, Zhenhui Peng
UIST4
2024 XNLI: Explaining and Diagnosing NLI-Based Visual Data Analysis
abstract
Natural language interfaces (NLIs) enable users to flexibly specify analytical intentions in data visualization. However, diagnosing the visualization results without understanding the underlying generation process is challenging. Our research explores how to provide explanations for NLIs to help users locate the problems and further revise the queries. We present XNLI, an explainable NLI system for visual data analysis. The system introduces a Provenance Generator to reveal the detailed process of visual transformations, a suite of interactive widgets to support error adjustments, and a Hint Generator to provide query revision hints based on the analysis of user queries and interactions. Two usage scenarios of XNLI and a user study verify the effectiveness and usability of the system. Results suggest that XNLI can significantly enhance task accuracy without interrupting the NLI-based analysis process.
Yingchaojie Feng, Xingbo Wang 0001, Bo Pan 0004, Kamkwai Wong, Yuxin Ma 0001, Huamin Qu, Wei Chen 0001
IEEE Trans. Vis. Comput. Graph.2
2024 PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
abstract
Generative text-to-image models have gained great popularity among the public for their powerful capability to generate high-quality images based on natural language prompts. However, developing effective prompts for desired images can be challenging due to the complexity and ambiguity of natural language. This research proposes PromptMagician, a visual analysis system that helps users explore the image results and refine the input prompts. The backbone of our system is a prompt recommendation model that takes user prompts as input, retrieves similar prompt-image pairs from DiffusionDB, and identifies special (important and relevant) prompt keywords. To facilitate interactive prompt refinement, PromptMagician introduces a multi-level visualization for the cross-modal embedding of the retrieved images and recommended keywords, and supports users in specifying multiple criteria for personalized exploration. Two usage scenarios, a user study, and expert interviews demonstrate the effectiveness and usability of our system, suggesting it facilitates prompt engineering and improves the creativity support of the generative text-to-image model.
Yingchaojie Feng, Xingbo Wang 0001, Kamkwai Wong, Yuhong Lu, Minfeng Zhu 0001, Baicheng Wang, Wei Chen 0001
IEEE Trans. Vis. Comput. Graph.2
2024 : A Visual Analytics Approach for Interactive Video Programming
abstract
Constructing supervised machine learning models for real-world video analysis require substantial labeled data, which is costly to acquire due to scarce domain expertise and laborious manual inspection. While data programming shows promise in generating labeled data at scale with user-defined labeling functions, the high dimensional and complex temporal information in videos poses additional challenges for effectively composing and evaluating labeling functions. In this paper, we propose VideoPro, a visual analytics approach to support flexible and scalable video data programming for model steering with reduced human effort. We first extract human-understandable events from videos using computer vision techniques and treat them as atomic components of labeling functions. We further propose a two-stage template mining algorithm that characterizes the sequential patterns of these events to serve as labeling function templates for efficient data labeling. The visual interface of VideoPro facilitates multifaceted exploration, examination, and application of the labeling templates, allowing for effective programming of video data at scale. Moreover, users can monitor the impact of programming on model performance and make informed adjustments during the iterative programming process. We demonstrate the efficiency and effectiveness of our approach with two case studies and expert interviews.
Jianben He, Xingbo Wang 0001, Kamkwai Wong, Xijie Huang, Changjian Chen, Zixin Chen, Fengjie Wang, Min Zhu 0005, Huamin Qu
IEEE Trans. Vis. Comput. Graph.2
2024 ShortcutLens: A Visual Analytics Approach for Exploring Shortcuts in Natural Language Understanding Dataset
abstract
Benchmark datasets play an important role in evaluating Natural Language Understanding (NLU) models. However, shortcuts-unwanted biases in the benchmark datasets-can damage the effectiveness of benchmark datasets in revealing models' real capabilities. Since shortcuts vary in coverage, productivity, and semantic meaning, it is challenging for NLU experts to systematically understand and avoid them when creating benchmark datasets. In this paper, we develop a visual analytics system, ShortcutLens, to help NLU experts explore shortcuts in NLU benchmark datasets. The system allows users to conduct multi-level exploration of shortcuts. Specifically, Statistics View helps users grasp the statistics such as coverage and productivity of shortcuts in the benchmark dataset. Template View employs hierarchical and interpretable templates to summarize different types of shortcuts. Instance View allows users to check the corresponding instances covered by the shortcuts. We conduct case studies and expert interviews to evaluate the effectiveness and usability of the system. The results demonstrate that ShortcutLens supports users in gaining a better understanding of benchmark dataset issues through shortcuts, inspiring them to create challenging and pertinent benchmark datasets.
Zhihua Jin, Xingbo Wang 0001, Furui Cheng, Chunhui Sun, Qun Liu 0001, Huamin Qu
IEEE Trans. Vis. Comput. Graph.2
2024 Anchorage: Visual Analysis of Satisfaction in Customer Service Videos Via Anchor Events
abstract
Delivering customer services through video communications has brought new opportunities to analyze customer satisfaction for quality management. However, due to the lack of reliable self-reported responses, service providers are troubled by the inadequate estimation of customer services and the tedious investigation into multimodal video recordings. We introduce Anchorage, a visual analytics system to evaluate customer satisfaction by summarizing multimodal behavioral features in customer service videos and revealing abnormal operations in the service process. We leverage the semantically meaningful operations to introduce structured event understanding into videos which help service providers quickly navigate to events of their interest. Anchorage supports a comprehensive evaluation of customer satisfaction from the service and operation levels and efficient analysis of customer behavioral dynamics via multifaceted visualization views. We extensively evaluate Anchorage through a case study and a carefully-designed user study. The results demonstrate its effectiveness and usability in assessing customer satisfaction using customer service videos. We found that introducing event contexts in assessing customer satisfaction can enhance its performance without compromising annotation precision. Our approach can be adapted in situations where unlabelled and unstructured videos are collected along with sequential records.
Kamkwai Wong, Xingbo Wang 0001, Yong Wang 0021, Jianben He, Rong Zhang 0011, Huamin Qu
IEEE Trans. Vis. Comput. Graph.2
2024 : Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models
abstract
Recently, large pretrained language models have achieved compelling performance on commonsense benchmarks. Nevertheless, it is unclear what commonsense knowledge the models learn and whether they solely exploit spurious patterns. Feature attributions are popular explainability techniques that identify important input concepts for model outputs. However, commonsense knowledge tends to be implicit and rarely explicitly presented in inputs. These methods cannot infer models' implicit reasoning over mentioned concepts. We present CommonsenseVIS, a visual explanatory system that utilizes external commonsense knowledge bases to contextualize model behavior for commonsense question-answering. Specifically, we extract relevant commonsense knowledge in inputs as references to align model behavior with human knowledge. Our system features multi-level visualization and interactive model probing and editing for different concepts and their underlying relations. Through a user study, we show that CommonsenseVIS helps NLP experts conduct a systematic and scalable visual analysis of models' relational reasoning over concepts in different situations.
Xingbo Wang 0001, Renfei Huang, Zhihua Jin, Tianqing Fang, Huamin Qu
IEEE Trans. Vis. Comput. Graph.1
2024 LiveRetro: Visual Analytics for Strategic Retrospect in Livestream E-Commerce
abstract
Livestream e-commerce integrates live streaming and online shopping, allowing viewers to make purchases while watching. However, effective marketing strategies remain a challenge due to limited empirical research and subjective biases from the absence of quantitative data. Current tools fail to capture the interdependence between live performances and feedback. This study identified computational features, formulated design requirements, and developed LiveRetro, an interactive visual analytics system. It enables comprehensive retrospective analysis of livestream e-commerce for streamers, viewers, and merchandise. LiveRetro employs enhanced visualization and time-series forecasting models to align performance features and feedback, identifying influences at channel, merchandise, feature, and segment levels. Through case studies and expert interviews, the system provides deep insights into the relationship between live performance and streaming statistics, enabling efficient strategic analysis from multiple perspectives.
Yuansong Xu, Xingbo Wang 0001, Wenkai Song, Zhiheng Nie, Xiaomeng Fan, Quan Li 0002
IEEE Trans. Vis. Comput. Graph.4
2023 MultiViz: Towards Visualizing and Understanding Multimodal Models
Paul Pu Liang, Yiwei Lyu 0001, Gunjan Chhablani, Nihal Jain, Xingbo Wang 0001, Louis-Philippe Morency, Ruslan Salakhutdinov
ICLR6
2023 Storyfier: Exploring Vocabulary Learning Support with Text Generation Models
abstract
Vocabulary learning support tools have widely exploited existing materials, e.g., stories or video clips, as contexts to help users memorize each target word. However, these tools could not provide a coherent context for any target words of learners’ interests, and they seldom help practice word usage. In this paper, we work with teachers and students to iteratively develop Storyfier, which leverages text generation models to enable learners to read a generated story that covers any target words, conduct a story cloze test, and use these words to write a new story with adaptive AI assistance. Our within-subjects study (N=28) shows that learners generally favor the generated stories for connecting target words and writing assistance for easing their learning workload. However, in the read-cloze-write learning sessions, participants using Storyfier perform worse in recalling and using target words than learning with a baseline tool without our AI features. We discuss insights into supporting learning tasks with generative models.
Zhenhui Peng, Xingbo Wang 0001, Qiushi Han, Junkai Zhu, Xiaojuan Ma, Huamin Qu
UIST2
2023 GestureLens: Visual Analysis of Gestures in Presentation Videos
abstract
Appropriate gestures can enhance message delivery and audience engagement in both daily communication and public presentations. In this article, we contribute a visual analytic approach that assists professional public speaking coaches in improving their practice of gesture training through analyzing presentation videos. Manually checking and exploring gesture usage in the presentation videos is often tedious and time-consuming. There lacks an efficient method to help users conduct gesture exploration, which is challenging due to the intrinsically temporal evolution of gestures and their complex correlation to speech content. In this article, we propose GestureLens, a visual analytics system to facilitate gesture-based and content-based exploration of gesture usage in presentation videos. Specifically, the exploration view enables users to obtain a quick overview of the spatial and temporal distributions of gestures. The dynamic hand movements are firstly aggregated through a heatmap in the gesture space for uncovering spatial patterns, and then decomposed into two mutually perpendicular timelines for revealing temporal patterns. The relation view allows users to explicitly explore the correlation between speech content and gestures by enabling linked analysis and intuitive glyph designs. The video view and dynamic view show the context and overall dynamic movement of the selected gestures, respectively. Two usage scenarios and expert interviews with professional presentation coaches demonstrate the effectiveness and usefulness of GestureLens in facilitating gesture exploration and analysis of presentation videos.
Haipeng Zeng, Xingbo Wang 0001, Yong Wang 0021, Aoyu Wu, Ting-Chuen Pong, Huamin Qu
IEEE Trans. Vis. Comput. Graph.2
2022 Designing a Data Visualization Dashboard for Pre-Screening Hong Kong Students with Specific Learning Disabilities
abstract
Students with specific learning disabilities (SLDs) often experience reading, writing, attention, and physical movement coordination difficulties. However, in Hong Kong, it takes years for special education needs coordinators (SENCOs) and special-ed teachers to pre-screen and diagnose students with SLDs. Therefore, many students with SLDs missed the golden time for special interventions (i.e., before six years old). In addition, although there are screening tools for students with SLDs in Chinese and Indo-European languages (e.g., English and Spanish), they did not provide a student data visualization dashboard that could help teachers speed up the pre-screening process. Therefore, we designed a new visualization dashboard for Hong Kong SENCOs and special-ed teachers to assist them in pre-screening students with SLDs. Our formative study showed that our current design met teachers’ need to quickly identify a student’s specific under-performing tasks and effectively collect evidence about how the student was affected by SLDs. Future work will further test the efficacy of our design in real life.
Ka Yan Fung, Zikai Wen, Haotian Li 0001, Xingbo Wang 0001, Shenghui Song 0001, Huamin Qu
ASSETS4
2022 Persua: A Visual Interactive System to Enhance the Persuasiveness of Arguments in Online Discussion
abstract
Persuading people to change their opinions is a common practice in online discussion forums on topics ranging from political campaigns to relationship consultation. Enhancing people's ability to write persuasive arguments could not only practice their critical thinking and reasoning but also contribute to the effectiveness and civility in online communication. It is, however, not an easy task in online discussion settings where written words are the primary communication channel. In this paper, we derived four design goals for a tool that helps users improve the persuasiveness of arguments in online discussions through a survey with 123 online forum users and interviews with five debating experts. To satisfy these design goals, we analyzed and built a labeled dataset of fine-grained persuasive strategies (i.e., logos, pathos, ethos, and evidence) in 164 arguments with high ratings on persuasiveness from ChangeMyView, a popular online discussion forum. We then designed an interactive visual system, Persua, which provides example-based guidance on persuasive strategies to enhance the persuasiveness of arguments. In particular, the system constructs portfolios of arguments based on different persuasive strategies applied to a given discussion topic. It then presents concrete examples based on the difference between the portfolios of user input and high-quality arguments in the dataset. A between-subjects study shows suggestive evidence that Persua encourages users to submit more times for feedback and helps users improve more on the persuasiveness of their arguments than a baseline system. Finally, a set of design considerations was summarized to guide future intelligent systems that improve the persuasiveness in text.
Meng Xia 0002, Qian Zhu 0010, Xingbo Wang 0001, Fei Nie, Huamin Qu, Xiaojuan Ma
Proc. ACM Hum. Comput. Interact.3
2022 M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis
abstract
Multimodal sentiment analysis aims to recognize people's attitudes from multiple communication channels such as verbal content (i.e., text), voice, and facial expressions. It has become a vibrant and important research topic in natural language processing. Much research focuses on modeling the complex intra- and inter-modal interactions between different communication channels. However, current multimodal models with strong performance are often deep-learning-based techniques and work like black boxes. It is not clear how models utilize multimodal information for sentiment predictions. Despite recent advances in techniques for enhancing the explainability of machine learning models, they often target unimodal scenarios (e.g., images, sentences), and little research has been done on explaining multimodal models. In this paper, we present an interactive visual analytics system, M2Lens, to visualize and explain multimodal models for sentiment analysis. M2Lens provides explanations on intra- and inter-modal interactions at the global, subset, and local levels. Specifically, it summarizes the influence of three typical interaction types (i.e., dominance, complement, and conflict) on the model predictions. Moreover, M2Lens identifies frequent and influential multimodal features and supports the multi-faceted exploration of model behaviors from language, acoustic, and visual modalities. Through two case studies and expert interviews, we demonstrate our system can help users gain deep insights into the multimodal models for sentiment analysis.
Xingbo Wang 0001, Jianben He, Zhihua Jin, Muqiao Yang, Yong Wang 0021, Huamin Qu
IEEE Trans. Vis. Comput. Graph.1
2022 DeHumor: Visual Analytics for Decomposing Humor
abstract
Despite being a critical communication skill, grasping humor is challenging-a successful use of humor requires a mixture of both engaging content build-up and an appropriate vocal delivery (e.g., pause). Prior studies on computational humor emphasize the textual and audio features immediately next to the punchline, yet overlooking longer-term context setup. Moreover, the theories are usually too abstract for understanding each concrete humor snippet. To fill in the gap, we develop DeHumor, a visual analytical system for analyzing humorous behaviors in public speaking. To intuitively reveal the building blocks of each concrete example, DeHumor decomposes each humorous video into multimodal features and provides inline annotations of them on the video script. In particular, to better capture the build-ups, we introduce content repetition as a complement to features introduced in theories of computational humor and visualize them in a context linking graph. To help users locate the punchlines that have the desired features to learn, we summarize the content (with keywords) and humor feature statistics on an augmented time matrix. With case studies on stand-up comedy shows and TED talks, we show that DeHumor is able to highlight various building blocks of humor examples. In addition, expert interviews with communication coaches and humor researchers demonstrate the effectiveness of DeHumor for multimodal humor analysis of speech content and vocal delivery.
Xingbo Wang 0001, Yao Ming, Sherry Tongshuang Wu, Haipeng Zeng, Yong Wang 0021, Huamin Qu
IEEE Trans. Vis. Comput. Graph.1
2020 VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking
abstract
The modulation of voice properties, such as pitch, volume, and speed, is crucial for delivering a successful public speech. However, it is challenging to master different voice modulation skills. Though many guidelines are available, they are often not practical enough to be applied in different public speaking situations, especially for novice speakers. We present VoiceCoach, an interactive evidence-based approach to facilitate the effective training of voice modulation skills. Specifically, we have analyzed the voice modulation skills from 2623 high-quality speeches (i.e., TED Talks) and use them as the benchmark dataset. Given a voice input, VoiceCoach automatically recommends good voice modulation examples from the dataset based on the similarity of both sentence structures and voice modulation skills. Immediate and quantitative visual feedback is provided to guide further improvement. The expert interviews and the user study provide support for the effectiveness and usability of VoiceCoach.
Xingbo Wang 0001, Haipeng Zeng, Yong Wang 0021, Aoyu Wu, Zhida Sun, Xiaojuan Ma, Huamin Qu
CHI1
2020 EmoCo: Visual Analysis of Emotion Coherence in Presentation Videos
abstract
Emotions play a key role in human communication and public presentations. Human emotions are usually expressed through multiple modalities. Therefore, exploring multimodal emotions and their coherence is of great value for understanding emotional expressions in presentations and improving presentation skills. However, manually watching and studying presentation videos is often tedious and time-consuming. There is a lack of tool support to help conduct an efficient and in-depth multi-level analysis. Thus, in this paper, we introduce EmoCo, an interactive visual analytics system to facilitate efficient analysis of emotion coherence across facial, text, and audio modalities in presentation videos. Our visualization system features a channel coherence view and a sentence clustering view that together enable users to obtain a quick overview of emotion coherence and its temporal evolution. In addition, a detail view and word view enable detailed exploration and comparison from the sentence level and word level, respectively. We thoroughly evaluate the proposed system and visualization techniques through two usage scenarios based on TED Talk videos and interviews with two domain experts. The results demonstrate the effectiveness of our system in gaining insights into emotion coherence in presentations.
Haipeng Zeng, Xingbo Wang 0001, Aoyu Wu, Yong Wang 0021, Quan Li 0002, Alex Endert, Huamin Qu
IEEE Trans. Vis. Comput. Graph.2
2019 RoadNet: Learning to Comprehensively Analyze Road Networks in Complex Urban Scenes From High-Resolution Remotely Sensed Images
abstract
It is a classical task to automatically extract road networks from very high-resolution (VHR) images in remote sensing. This paper presents a novel method for extracting road networks from VHR remotely sensed images in complex urban scenes. Inspired by image segmentation, edge detection, and object skeleton extraction, we develop a multitask convolutional neural network (CNN), called RoadNet, to simultaneously predict road surfaces, edges, and centerlines, which is the first work in such field. The RoadNet solves seven important issues in this vision problem: 1) automatically learning multiscale and multilevel features [gained by the deeply supervised nets (DSN) providing integrated direct supervision] to cope with the roads in various scenes and scales; 2) holistically training the mentioned tasks in a cascaded end-to-end CNN model; 3) correlating the predictions of road surfaces, edges, and centerlines in a network model to improve the multitask prediction; 4) designing elaborate architecture and loss function, by which the well-trained model produces approximately single-pixel width road edges/centerlines without nonmaximum suppression postprocessing; 5) cropping and bilinear blending to deal with the large VHR images with finite-computing resources; 6) introducing rough and simple user interaction to obtain desired predictions in the challenging regions; and 7) establishing a benchmark data set which consists of a series of VHR remote sensing images with pixelwise annotation. Different from the previous works, we pay more attention to the challenging situations, in which there are lots of shadows and occlusions along the road regions. Experimental results on two benchmark data sets show the superiority of our proposed approaches.
Jian Yao 0002, Xiaohu Lu, Menghan Xia, Xingbo Wang 0001
IEEE Trans. Geosci. Remote. Sens.5