Yang Shi 0007

dblp:15/5233-7 · DBLP profile ↗
← Back
36ranked-venue papers
15as first author
28since 2021 · last 2026
0000-0002-1065-4038ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Graphics, computer vision, multimedia, augmented reality and games · 18 · 4 first-author · 14 since 2021Human-computer interaction and ubiquitous computing · 15 · 10 first-author · 11 since 2021Artificial intelligence and machine learning · 4 · 3 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021Applied, interdisciplinary, general and emerging computing · 1 · 1 first-author · 1 since 2021
YearPublicationVenuePosition
2026 All Futures at Once: Supporting Speculative Design for Placemaking with Multi-Agent Social Simulation
abstract
Placemaking transforms physical spaces into socially meaningful places, with long-term impacts depending on how future communities inhabit and interact with them. Speculative design helps envision such futures, yet existing approaches often produce static representations that emphasize spatial form over evolving activity. We present ParaScape, a design support system that facilitates speculative design for placemaking by generating dynamic speculative objects through an underlying LLM-based multi-agent social simulation framework. The framework models heterogeneous agents with group-specific preferences and sensitivities, simulating context-sensitive behaviors and interactions that produce evolving scenarios. These scenarios are visualized as image sequences, where each scenario depicts multiple activities unfolding within a place at a given moment. ParaScape builds on this framework to allow designers to explore scenarios, analyze activity diversity and evolvability, and reflect on trade-offs among stakeholder needs. Evaluations through two experiments, a user study, and two case studies show that ParaScape supports critical reasoning and inclusive placemaking.
Jiarui Jiang, Shuqing Tang, Mutao Yu, Caoyang Xue, Yunsheng Su, Xinyang Tan, Yang Shi 0007
CHI9
2026 From Touch to Change: Understanding Public Engagement in Data Physicalization for Social Good
abstract
Data physicalization, which encodes data in physical form, has been increasingly used to engage the public with issues of social good. While public engagement is often invoked as a motivation or expected outcome, it has not been systematically examined as a design objective. This gap raises two key challenges: what characterizes engagement in data physicalization for social good (Phys4Good), and how it can be effectively designed. In this work, we address these challenges by first curating a corpus of 45 Phys4Good projects and deriving a design space structured around a modified three-act framework comprising Stage, Encounter, and Impact. We then conducted semi-structured interviews with designers of eight projects to identify recurring challenges and strategies for fostering engagement. Finally, we demonstrated the effectiveness of our design space and strategies through a case study, which showed that they can guide designers in structuring engagement, anticipating barriers, and creating more impactful Phys4Good experiences.
Yechun Peng, Runxi Wu, Nan Cao 0001, Yang Shi 0007
CHI4
2026 Vistoryteller: Designing Data Stories with LLM Agent-Based Generation and Interactive User Control
abstract
Data stories that combine data, visualizations, and prose are widely used for communication, decision making, and persuasion, but producing them typically requires coordinated effort across specialized roles such as analysts, scripters, and designers, which is time consuming and difficult to manage. Existing AI-assisted methods generally treat storytelling as a single-agent task and offer only coarse, global controls, limiting an author’s ability to preserve and shape their communication intention over the course of a narrative. In this work, we present Vistoryteller, a multi-agent authoring system that models the division of labor found in human teams by assigning specialized large language model agents to complementary roles and orchestrating their interactions to generate cohesive, intention-aligned data stories. Vistoryteller supports fine-grained authorial control through two complementary mechanisms: a sketch-based tension-flow control for specifying how thematic emphasis and narrative tension should evolve, and a conversational interface for issuing localized directives to individual agents or to the team. We evaluate Vistoryteller with two controlled experiments and a qualitative user study. Results show that Vistoryteller generates narratives that align more closely with user intentions, preserve coherence across agent contributions, and surface diverse and expressive insights.
Yang Shi 0007, Chuyi Zheng, Nan Cao 0001
IUI1
2026 How We Map Possibilities: Understanding Design Spaces for Visualization
abstract
Design spaces serve as conceptual frameworks that enable systematic exploration of possibilities and constraints for particular design problems. Despite growing recognition of their importance in visualization research, the community faces two main challenges: characterizing what constitute a design space, given the lack of consensus on its definition, and determining how to construct these spaces in the absence of established methodologies. To address the challenges, we first conducted a literature review of visualization design space research, identifying three distinct research threads. Focusing on the thread that views design spaces as multi-dimensional frameworks, we refined our corpus to 49 papers and developed a unified conceptualization of design spaces. Building on this foundation, we proposed a systematic approach to design space construction, synthesized from an analysis of practices spanning five phases: exploration, data collection, creation, evaluation, and communication.
Zichun Dai, Yechun Peng, Nan Cao 0001, Yang Shi 0007
IEEE Trans. Vis. Comput. Graph.4
2026 Data Speaks, But who Gives It a Voice? Understanding Persuasive Strategies in Data-Driven News Articles
abstract
Data-driven news articles combine narrative storytelling with data visualizations to inform and influence public opinion on pressing societal issues. These articles often employ persuasive strategies, which are rhetorical techniques in narrative framing, visual rhetoric, or data presentation, to influence audience interpretation and opinion formation regarding information communication. While previous research has examined whether and when data visualizations persuade, the strategic choices made by persuaders remain largely unexplored. Addressing this gap, our work presents a taxonomy of persuasive strategies grounded in psychological theories and expert insights, categorizing 15 strategies across five dimensions: Credibility, Guided Interpretation, Reference-based Framing, Emotional Appeal, and Participation Invitation. To facilitate large-scale analysis, we curated a dataset of 936 data-driven news articles annotated with both persuasive strategies and their perceived effects. Leveraging this corpus, we developed a multimodal, multi-task learning model that jointly predicts the presence of persuasive strategies and their persuasive effects by incorporating both embedded (text and visualization) and explicit (visual narrative and psycholinguistic) features. Our evaluation demonstrates that our model outperforms state-of-the-art baselines in identifying persuasive strategies and measuring their effects.
Zikai Li, Chuyi Zheng, Yang Shi 0007
IEEE Trans. Vis. Comput. Graph.4
2025 Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graph
abstract
Large language models (LLMs) have demonstrated exceptional performance across a wide variety of domains. Nonetheless, generalist LLMs continue to fall short in reasoning tasks necessitating specialized knowledge, e.g., emotional sociology and medicine. Prior investigations into specialized LLMs focused on domain-specific training, which entails substantial efforts in domain data acquisition and model parameter fine-tuning. To address these challenges, this paper proposes the Way-to-Specialist (WTS) framework, which synergizes retrieval-augmented generation with knowledge graphs (KGs) to enhance the specialized capability of LLMs in the absence of specialized training. In distinction to existing paradigms that merely utilize external knowledge from general KGs or static domain KGs to prompt LLM for enhanced domain-specific reasoning, WTS proposes an innovative ''LLM↻KG'' paradigm, which achieves bidirectional enhancement between specialized LLM and domain knowledge graph (DKG). The proposed paradigm encompasses two closely coupled components: the DKG-Augmented LLM and the LLM-Assisted DKG Evolution. The former retrieves question-relevant domain knowledge from DKG and uses it to prompt LLM to enhance the reasoning capability for domain-specific tasks; the latter leverages LLM to generate new domain knowledge from processed tasks and use it to evolve DKG. WTS closes the loop between DKG-Augmented LLM and LLM-Assisted DKG Evolution, enabling continuous improvement in the domain specialization as it progressively answers and learns from domain-specific questions. We validate the performance of WTS on 7 datasets (e.g., TweetQA, ChatDoctor5k) spanning 6 domains, e.g., emotional sociology, medical, ect. The experimental results show that WTS surpasses the previous SOTA in 5 specialized domains, and achieves a maximum performance improvement of 11.3%.
Yutong Zhang 0003, Lixing Chen, Shenghong Li 0001, Nan Cao 0001, Yang Shi 0007, Jiaxin Ding 0001, Pan Zhou 0001, Yang Bai 0010
KDD (1)5
2025 ViviClay: Designing and Fabricating Ceramics with Animation Effects on Physical Surfaces
Guanhong Liu, Jingxin Ye, Qiaoqiao Jin, Xuechen Li 0003, Yang Shi 0007, Qing Chen 0001, Nan Cao 0001
UIST6
2025 Double Tap for This Post: Understanding the Communication of Data Visualization on Social Media
abstract
Data visualizations are increasingly used by news outlets on social media to communicate insights to a broad audience. However, little is known about how readers interact with and respond to data visualizations in these quick-consumption environments. In this work, we introduce a conceptual model that categorizes visualization reading that leads to the communication effect of likes on Instagram. The model was developed through a grounded theory analysis of the statements explaining the reasoning behind the likes of visualization, which were recorded from a preliminary study. Informed by coding the statements from two dimensions including scopes and design patterns concerning visualization, our model consists of three levels: depicting the "look" of a visualization (e.g., artistic style and color scheme); interpreting the "flesh and bones" of a visualization (e.g., visualization and narrative); and elucidating the "heart and soul" of a visualization (e.g., insights and conclusion). We also conducted an online crowdsourcing user study with 200 participants to demonstrate how our model can be applied to improve the communication of visualization by comparing the three levels.
Yang Shi 0007, Yechun Peng, Jieying Ding, Xingyu Lan, Nan Cao 0001
Proc. ACM Hum. Comput. Interact.1
2025 MV-Crafter: An Intelligent System for Music-Guided Video Generation
abstract
Music videos, as a prevalent form of multimedia entertainment, deliver engaging audio-visual experiences to audiences and have gained immense popularity among singers and fans. Creators can express their interpretations of music naturally through visual elements. However, the creation process of music video demands proficiency in script design, video shooting, and music-video synchronization, posing significant challenges for non-professionals. Previous work has designed automated music video generation frameworks. However, they suffer from complexity in input and poor output quality. In response, we present MV-Crafter, a system capable of producing high-quality music videos with synchronized music-video rhythm and style. Our approach involves three technical modules that simulate the human creation process: the script generation module, video generation module, and music-video synchronization module. MV-Crafter leverages a large language model to generate scripts considering the musical semantics. To address the challenge of synchronizing short video clips with music of varying lengths, we propose a dynamic beat-matching algorithm and visual envelope-induced warping method to ensure precise, monotonic music-video synchronization. Besides, we design a user-friendly interface to simplify the creation process with intuitive editing features. Extensive experiments have demonstrated that MV-Crafter provides an effective solution for improving the quality of generated music videos.
Chuer Chen, Shengqi Dang, Nanxuan Zhao, Yang Shi 0007, Nan Cao 0001
ACM Trans. Interact. Intell. Syst.5
2025 Leveraging Foundation Models for Crafting Narrative Visualization: A Survey
abstract
Narrative visualization transforms data into engaging stories, making complex information accessible to a broad audience. Foundation models, with their advanced capabilities such as natural language processing, content generation, and multimodal integration, hold substantial potential for enriching narrative visualization. Recently, a collection of techniques have been introduced for crafting narrative visualizations based on foundation models from different aspects. We build our survey upon 66 articles to study how foundation models can progressively engage in this process and then propose a reference model categorizing the reviewed literature into four essential phases: Analysis, Narration, Visualization, and Interaction. Furthermore, we identify eight specific tasks (e.g., Insight Extraction and Authoring) where foundation models are applied across these stages to facilitate the creation of visual narratives. Detailed descriptions, related literature, and reflections are presented for each task. To make it a more impactful and informative experience for diverse readers, we discuss key research problems and provide the strengths and weaknesses in each task to guide people in identifying and seizing opportunities while navigating challenges in this field.
Shixiong Cao, Yang Shi 0007, Qing Chen 0001, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.4
2025 Complex Surface Fabrication Via Developable Surface Approximation: A Survey
abstract
Complex surfaces are commonly observed in various applications and have significant value in enhancing comfort, aesthetics, and functionality. However, their fabrication often involves complex and costly processes. To simplify the fabrication difficulty, significant research has focused on using 3D developable surfaces to approximate target 3D surfaces. This process involves converting target 3D surfaces into developable surfaces and then flattening them into 2D patterns. Since the geometric and topological diversity of target surfaces, this task is both comprehensive and intricate, encompassing multiple aspects from design to fabrication. In this paper, we review relevant technologies and methods in fabrication processes, classify them, and summarize a pipeline from design to fabrication. This provides a comprehensive introduction to the field for researchers and practitioners. Through the analysis of relevant literature, we also discuss some of the research challenges and future research opportunities.
Nan Cao 0001, Yang Shi 0007
IEEE Trans. Vis. Comput. Graph.3
2024 Personalizing Products with Stylized Head Portraits for Self-Expression
abstract
Personalizing products aesthetically or functionally can help users increase personal relevance and support self-expression. However, using non-abstract personal data such as head portraits for product personalization has been understudied. While recent advances in Artificial Intelligence have enabled generating stylized head portraits, these images also raise concerns about lack of control, artificiality, and ethics, which potentially limit their broader use. In this work, we present PicMe, a design support tool that converts user face photos into stylized head portraits as vector graphics that can be used to personalize products. To enable style transfer, PicMe leverages a deep-learning-based algorithm trained on an extended open-source illustration dataset of characters in a cartoonish and minimalistic style. We evaluated PicMe through two experiments and a user study. The results of our evaluation showed that PicMe can help create personalized head portraits that support self-expression.
Yang Shi 0007, Yechun Peng, Shengqi Dang, Nanxuan Zhao, Nan Cao 0001
CHI1
2024 Supporting Guided Exploratory Visual Analysis on Time Series Data with Reinforcement Learning
abstract
The exploratory visual analysis (EVA) of time series data uses visualization as the main output medium and input interface for exploring new data. However, for users who lack visual analysis expertise, interpreting and manipulating EVA can be challenging. Thus, providing guidance on EVA is necessary and two relevant questions need to be answered. First, how to recommend interesting insights to provide a first glance at data and help develop an exploration goal. Second, how to provide step-by-step EVA suggestions to help identify which parts of the data to explore. In this work, we present a reinforcement learning (RL)-based system, Visail, which generates EVA sequences to guide the exploration of time series data. As a user uploads a time series dataset, Visail can generate step-by-step EVA suggestions, while each step is visualized as an annotated chart combined with textual descriptions. The RL-based algorithm uses exploratory data analysis knowledge to construct the state and action spaces for the agent to imitate human analysis behaviors in data exploration tasks. In this way, the agent learns the strategy of generating coherent EVA sequences through a well-designed network. To evaluate the effectiveness of our system, we conducted an ablation study, a user study, and two case studies. The results of our evaluation suggested that Visail can provide effective guidance on supporting EVA on time series data.
Yang Shi 0007, Bingchang Chen, Zhuochen Jin, Xiaohan Jiao, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.1
2023 Bring Clipart to Life
abstract
The development of face editing has been boosted since the birth of StyleGAN. While previous works have explored different interactive methods, such as sketching and exemplar photos, they have been limited in terms of expressiveness and generality. In this paper, we propose a new interaction method by guiding the editing with abstract clipart, composed of a set of simple semantic parts, allowing users to control across face photos with simple clicks. However, this is a challenging task given the large domain gap between colorful face photos and abstract clipart with limited data. To solve this problem, we introduce a frame-work called ClipFaceShop1built on top of StyleGAN. The key idea is to take advantage of $\mathcal{W} +$ latent code encoded rich and disentangled visual features, and create a new lightweight selective feature adaptor to predict a modifiable path toward the target output photo. Since no pairwise labeled data exists for training, we design a set of losses to provide supervision signals for learning the modifiable path. Experimental results show that ClipFaceShop generates realistic and faithful face photos, sharing the same facial attributes as the reference clipart. We demonstrate that ClipFaceShop supports clipart in diverse styles, even in form of a free-hand sketch.
Nanxuan Zhao, Shengqi Dang, Hexun Lin, Yang Shi 0007, Nan Cao 0001
ICCV4
2023 Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature Review
abstract
Recent interest in design through the artificial intelligence (AI) lens is rapidly increasing. Designers, as a special user group interacting with AI, have received more attention in the Human-Computer Interaction community. Prior work has discussed emerging challenges that persist in designing for AI. However, few systematic reviews focus on AI for design to understand how designers and AI can augment each other's complementary strengths in design collaboration. In this work, we conducted a landscape analysis of AI for design, via a systematic literature review of 93 papers. The analysis first provides a bird's eye view of overall patterns in this area. The analysis also reveals three themes interpreted from the paper corpus associated with AI for design, including AI assisting designers, designers assisting AI, and characterizing designer-AI collaboration. We discuss the implications of our findings and suggested methodological proposals to guide HCI toward research and practices that center on collaborative creativity.
Yang Shi 0007, Xiaohan Jiao, Nan Cao 0001
Proc. ACM Hum. Comput. Interact.1
2023 Diverse Interaction Recommendation for Public Users Exploring Multi-view Visualization using Deep Learning
abstract
Interaction is an important channel to offer users insights in interactive visualization systems. However, which interaction to operate and which part of data to explore are hard questions for public users facing a multi-view visualization for the first time. Making these decisions largely relies on professional experience and analytic abilities, which is a huge challenge for non-professionals. To solve the problem, we propose a method aiming to provide diverse, insightful, and real-time interaction recommendations for novice users. Building on the Long-Short Term Memory Model (LSTM) structure, our model captures users' interactions and visual states and encodes them in numerical vectors to make further recommendations. Through an illustrative example of a visualization system about Chinese poets in the museum scenario, the model is proven to be workable in systems with multi-views and multiple interaction types. A further user study demonstrates the method's capability to help public users conduct more insightful and diverse interactive explorations and gain more accurate data insights.
Yusheng Qi, Yang Shi 0007, Qing Chen 0001, Nan Cao 0001, Siming Chen 0001
IEEE Trans. Vis. Comput. Graph.3
2023 Breaking the Fourth Wall of Data Stories through Interaction
abstract
Interaction is increasingly integrating into data stories to support data exploration and explanation. Interaction can also be combined with the narrative device, breaking the fourth wall (BTFW), to build a deeper connection between readers and data stories. BTFW interaction directly addresses readers by requiring their input. Such user input is then integrated into the narrative or visuals of data stories to encourage readers to inspect the stories more closely. In this work, we explore the design patterns of BTFW interaction commonly used in data stories. Six design patterns were identified through the analysis of 58 high-quality data stories collected from a range of online sources. Specifically, the data stories were categorized using a coding framework, including the input of BTFW interaction provided by readers and the output of BTFW interaction generated by data stories to respond to the input. To explore the benefits as well as concerns of using BTFW interaction, we conducted a three-session user study including the reading, interview, and recall sessions. The results of our user study suggested that BTFW interaction has a positive impact on self-story connection, user engagement, and information recall. We also discussed design implications to address the possible negative effects on the interactivity-comprehensibility balance, information privacy, and the learning curve of interaction brought by BTFW interaction.
Yang Shi 0007, Xiaohan Jiao, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.1
2023 Supporting Expressive and Faithful Pictorial Visualization Design with Visual Style Transfer
abstract
Pictorial visualizations portray data with figurative messages and approximate the audience to the visualization. Previous research on pictorial visualizations has developed authoring tools or generation systems, but their methods are restricted to specific visualization types and templates. Instead, we propose to augment pictorial visualization authoring with visual style transfer, enabling a more extensible approach to visualization design. To explore this, our work presents Vistylist, a design support tool that disentangles the visual style of a source pictorial visualization from its content and transfers the visual style to one or more intended pictorial visualizations. We evaluated Vistylist through a survey of example pictorial visualizations, a controlled user study, and a series of expert interviews. The results of our evaluation indicated that Vistylist is useful for creating expressive and faithful pictorial visualizations.
Yang Shi 0007, Siji Chen, Mengdi Sun, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.1
2023 Erato: Cooperative Data Story Editing via Fact Interpolation
abstract
As an effective form of narrative visualization, visual data stories are widely used in data-driven storytelling to communicate complex insights and support data understanding. Although important, they are difficult to create, as a variety of interdisciplinary skills, such as data analysis and design, are required. In this work, we introduce Erato, a human-machine cooperative data story editing system, which allows users to generate insightful and fluent data stories together with the computer. Specifically, Erato only requires a number of keyframes provided by the user to briefly describe the topic and structure of a data story. Meanwhile, our system leverages a novel interpolation algorithm to help users insert intermediate frames between the keyframes to smooth the transition. We evaluated the effectiveness and usefulness of the Erato system via a series of evaluations including a Turing test, a controlled user study, a performance validation, and interviews with three expert users. The evaluation results showed that the proposed interpolation technique was able to generate coherent story content and help users create data stories more efficiently.
Mengdi Sun, Ligan Cai, Weiwei Cui 0001, Yanqiu Wu 0001, Yang Shi 0007, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.5
2022 Negative Emotions, Positive Outcomes? Exploring the Communication of Negativity in Serious Data Stories
abstract
Recent work has highlighted that emotion is key to the user experience with data stories. However, limited attention has been paid to negative emotions specifically. This work investigates the outcomes of negative emotions in the context of serious data stories and examines how they can be augmented by design methods from the perspectives of both storytellers and viewers. First, we conducted a workshop with 9 data story experts to understand the possible benefits of eliciting negative emotions in serious data stories and 19 potential design methods that contribute to negative emotions. Based on the findings from the workshop, we then conducted a lab study with 35 participants to explore the outcomes of eliciting negative emotions as well as the effectiveness of the design methods. The results indicated that negative emotions mainly facilitated contemplative experiences and long-term memory. Besides, the design methods showed varied effectiveness in augmenting negative emotions and being recalled.
Xingyu Lan, Yanqiu Wu 0001, Yang Shi 0007, Qing Chen 0001, Nan Cao 0001
CHI3
2022 ColorCook: Augmenting Color Design for Dashboarding with Domain-Associated Palettes
abstract
Visualization dashboards serve as an information presentation that uses a tiled layout of key metrics visualized in charts for collaborative decision-making. Existing work has developed tools and techniques for computational color design. Much of these efforts have focused on selecting effective color palettes for independent charts while few attempts have been made to support the expressive color design of multiple coordinated charts in dashboards. In this work, we describe ColorCook, an interactive system that helps design expressive and effective dashboard colorings using domain-associated palettes. ColorCook employs an integrated color workflow for dashboarding, consisting of color selection, assignment, and adjustment. We evaluated ColorCook through a crowdsourcing experiment and a user study. The results of our evaluation indicated that ColorCook is useful for effective and expressive color design.
Yang Shi 0007, Siji Chen, Nan Cao 0001
Proc. ACM Hum. Comput. Interact.1
2022 Visual Analytics of Anomalous User Behaviors: A Survey
abstract
With the pervasive use of information technologies, the increasing availability of data provides new opportunities for understanding user behaviors. Unearthing anomalies in user behavior is of particular importance as it helps signal harmful incidents such as network intrusions, terrorist activities, and financial frauds. In this article, we survey state-of-the-art research work in visual analytics of anomalous user behaviors and classify them into four application domains, which are social interaction, travel, network communication, and financial transaction. We further examine the research work in each category in terms of data types, visualization techniques, and interactive analysis methods. We hope that our survey can provide systematic guidelines for researchers and practitioners to find effective solutions to their research problems in specific application domains. Finally, we discuss trends of academic interest over the past decades and suggest potential directions across visual analytics of these user behaviors for future research.
Yang Shi 0007, Yuyin Liu, Hanghang Tong, Jingrui He, Nan Cao 0001
IEEE Trans. Big Data1
2022 Kineticharts: Augmenting Affective Expressiveness of Charts in Data Stories with Animation Design
abstract
Data stories often seek to elicit affective feelings from viewers. However, how to design affective data stories remains under-explored. In this work, we investigate one specific design factor, animation, and present Kineticharts, an animation design scheme for creating charts that express five positive affects: joy, amusement, surprise, tenderness, and excitement. These five affects were found to be frequently communicated through animation in data stories. Regarding each affect, we designed varied kinetic motions represented by bar charts, line charts, and pie charts, resulting in 60 animated charts for the five affects. We designed Kineticharts by first conducting a need-finding study with professional practitioners from data journalism and then analyzing a corpus of affective motion graphics to identify salient kinetic patterns. We evaluated Kineticharts through two user studies. The results suggest that Kineticharts can accurately convey affects, and improve the expressiveness of data stories, as well as enhance user engagement without hindering data comprehension compared to the animation design from DataClips, an authoring tool for data videos.
Xingyu Lan, Yang Shi 0007, Yanqiu Wu 0001, Xiaohan Jiao, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.2
2022 A Design Space for Applying the Freytag's Pyramid Structure to Data Stories
abstract
Data stories integrate compelling visual content to communicate data insights in the form of narratives. The narrative structure of a data story serves as the backbone that determines its expressiveness, and it can largely influence how audiences perceive the insights. Freytag's Pyramid is a classic narrative structure that has been widely used in film and literature. While there are continuous recommendations and discussions about applying Freytag's Pyramid to data stories, little systematic and practical guidance is available on how to use Freytag's Pyramid for creating structured data stories. To bridge this gap, we examined how existing practices apply Freytag's Pyramid by analyzing stories extracted from 103 data videos. Based on our findings, we proposed a design space of narrative patterns, data flows, and visual communications to provide practical guidance on achieving narrative intents, organizing data facts, and selecting visual design techniques through story creation. We evaluated the proposed design space through a workshop with 25 participants. Results show that our design space provides a clear framework for rapid storyboarding of data stories with Freytag's Pyramid.
Leni Yang, Xingyu Lan, Shunan Guo, Yang Shi 0007, Huamin Qu, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.6
2021 Communicating with Motion: A Design Space for Animated Visual Narratives in Data Videos
abstract
Data videos are a genre of narrative visualization that communicates stories by combining data visualization and motion graphics. While data videos are increasingly gaining popularity, few systematic reviews or structured analyses exist for their design. In this work, we introduce a design space for animated visual narratives in data videos. The design space combines a dimension for animation techniques that are frequently used to facilitate data communication with one for visual narrative strategies served by such animation techniques to support story presentation. We derived our design space from the analysis of 82 high-quality data videos collected from online sources. We conducted a workshop with 20 participants to evaluate the effectiveness of our design space. Qualitative and quantitative feedback suggested that our design space is inspirational and useful for designing and creating data videos.
Yang Shi 0007, Xingyu Lan, Zhaorui Li, Nan Cao 0001
CHI1
2021 Vinci: An Intelligent Graphic Design System for Generating Advertising Posters
abstract
Advertising posters are a commonly used form of information presentation to promote a product. Producing advertising posters often takes much time and effort of designers when confronted with abundant choices of design elements and layouts. This paper presents Vinci, an intelligent system that supports the automatic generation of advertising posters. Given the user-specified product image and taglines, Vinci uses a deep generative model to match the product image with a set of design elements and layouts for generating an aesthetic poster. The system also integrates online editing-feedback that supports users in editing the posters and updating the generated results with their design preference. Through a series of user studies and a Turing test, we found that Vinci can generate posters as good as human designers and that the online editing-feedback improves the efficiency in poster modification.
Shunan Guo, Zhuochen Jin, Fuling Sun, Zhaorui Li, Yang Shi 0007, Nan Cao 0001
CHI6
2021 Smile or Scowl? Looking at Infographic Design Through the Affective Lens
abstract
Infographics are frequently promoted for their ability to communicate data to audiences affectively. To facilitate the creation of affect-stirring infographics, it is important to characterize and understand people's affective responses to infographics and derive practical design guidelines for designers. To address these research questions, we first conducted two crowdsourcing studies to identify 12 infographic-associated affective responses and collect user feedback explaining what triggered affective responses in infographics. Then, by coding the user feedback, we present a taxonomy of design heuristics that exemplifies the affect-related design factors in infographics. We evaluated the design heuristics with 15 designers. The results showed that our work supports assessing the affective design in infographics and facilitates the ideation and creation of affective infographics.
Xingyu Lan, Yang Shi 0007, Yueyao Zhang, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.2
2021 Calliope: Automatic Visual Data Story Generation from a Spreadsheet
abstract
Visual data stories shown in the form of narrative visualizations such as a poster or a data video, are frequently used in data-oriented storytelling to facilitate the understanding and memorization of the story content. Although useful, technique barriers, such as data analysis, visualization, and scripting, make the generation of a visual data story difficult. Existing authoring tools rely on users' skills and experiences, which are usually inefficient and still difficult. In this paper, we introduce a novel visual data story generating system, Calliope, which creates visual data stories from an input spreadsheet through an automatic process and facilities the easy revision of the generated story based on an online story editor. Particularly, Calliope incorporates a new logic-oriented Monte Carlo tree search algorithm that explores the data space given by the input spreadsheet to progressively generate story pieces (i.e., data facts) and organize them in a logical order. The importance of data facts is measured based on information theory, and each data fact is visualized in a chart and captioned by an automatically generated description. We evaluate the proposed technique through three example stories, two controlled experiments, and a series of interviews with 10 domain experts. Our evaluation shows that Calliope is beneficial to efficient visual data story generation.
Danqing Shi, Fuling Sun, Yang Shi 0007, Nan Cao 0001
IEEE Trans. Vis. Comput. Graph.4
2020 EmoG: Supporting the Sketching of Emotional Expressions for Storyboarding
abstract
Storyboarding is an important ideation technique that uses sequential art to depict important scenarios of user experience. Existing data-driven support for storyboarding focuses on constructing user stories, but fail to address its benefit as a graphic narrative device. Instead, we propose to develop a data-driven design support tool that increases the expressiveness of user stories by facilitating sketching storyboards. To explore this, we focus on supporting the sketching of emotional expressions of characters in storyboards. In this paper, we present EmoG, an interactive system that generates sketches of characters with emotional expressions based on input strokes from the user. We evaluated EmoG with 21 participants in a controlled user study. The results showed that our tool has significantly better performance in usefulness, ease of use, and quality of results than the baseline system.
Yang Shi 0007, Nan Cao 0001, Xiaojuan Ma, Siji Chen
CHI1
2019 AI-Sketcher : A Deep Generative Model for Producing High-Quality Sketches
abstract
Sketch drawings play an important role in assisting humans in communication and creative design since ancient period. This situation has motivated the development of artificial intelligence (AI) techniques for automatically generating sketches based on user input. Sketch-RNN, a sequence-to-sequence variational autoencoder (VAE) model, was developed for this purpose and known as a state-of-the-art technique. However, it suffers from limitations, including the generation of lowquality results and its incapability to support multi-class generations. To address these issues, we introduced AI-Sketcher, a deep generative model for generating high-quality multiclass sketches. Our model improves drawing quality by employing a CNN-based autoencoder to capture the positional information of each stroke at the pixel level. It also introduces an influence layer to more precisely guide the generation of each stroke by directly referring to the training data. To support multi-class sketch generation, we provided a conditional vector that can help differentiate sketches under various classes. The proposed technique was evaluated based on two large-scale sketch datasets, and results demonstrated its power in generating high-quality sketches.
Nan Cao 0001, Yang Shi 0007
AAAI3
2019 Interactive Context-Aware Anomaly Detection Guided by User Feedback
abstract
Automatic anomaly detection techniques have been extensively used to support decision making in abnormal situations. However, existing approaches are limited in their capacity of effectively identifying anomalies due to the complexity of the real-world environment, the uncertainty of the data input, and the unavailability of ground truth. In this paper, we propose an interactive context-aware anomaly detection algorithm framework that incorporates human judgment in searching for anomalous regions within a large geographic environment. In specific, our framework, 1) estimates a focal region and detect anomalous situations in real time, through which the user can observe and analyze suspicious entities, 2) leverages user feedback to refine results and guide further analysis, and 3) tolerates potential fault feedback provided by the users and resignal dubious anomalous points. Based on the framework, we propose two algorithm implementations, respectively, employ Bayes’ theorem and metric learning. We demonstrate the effectiveness of the proposed framework and corresponding implementations through two controlled user studies and a case study with a domain expert.
Yang Shi 0007, Maoran Xu, Rongwen Zhao, Sherry Tongshuang Wu, Nan Cao 0001
IEEE Trans. Hum. Mach. Syst.1
2018 MeetingVis: Visual Narratives to Assist in Recalling Meeting Context and Content
abstract
In team-based workplaces, reviewing and reflecting on the content from a previously held meeting can lead to better planning and preparation. However, ineffective meeting summaries can impair this process, especially when participants have difficulty remembering what was said and what its context was. To assist with this process, we introduce MeetingVis, a visual narrative-based approach to meeting summarization. MeetingVis is composed of two primary components: (1) a data pipeline that processes the spoken audio from a group discussion, and (2) a visual-based interface that efficiently displays the summarized content. To design MeetingVis, we create a taxonomy of relevant meeting data points, identifying salient elements to promote recall and reflection. These are mapped to an augmented storyline visualization, which combines the display of participant activities, topic evolutions, and task assignments. For evaluation, we conduct a qualitative user study with five groups. Feedback from the study indicates that MeetingVis effectively triggers the recall of subtle details from prior meetings: all study participants were able to remember new details, points, and tasks compared to an unaided, memory-only baseline. This visual-based approaches can also potentially enhance the productivity of both individuals and the whole team.
Yang Shi 0007, Chris Bryan, Sridatt Bhamidipati, Ying Zhao 0001, Yaoxue Zhang, Kwan-Liu Ma
IEEE Trans. Vis. Comput. Graph.1
2017 IdeaWall: Improving Creative Collaboration through Combinatorial Visual Stimuli
abstract
With the recent advances in computer-supported cooperative work systems and increasing popularization of speech-based interfaces, groupware attempting to emulate a knowledgeable participant in a collaborative environment is bound to become a reality in the near future. In this paper, we present IdeaWall, a real-time system that continuously extracts essential information from a verbal discussion and augments that information with web-search materials. IdeaWall provides combinatorial visual stimuli to the participants to facilitate their creative process. We develop three cognitive strategies, from which a prototype application with three display modes was designed, implemented, and evaluated. The results of the user study with twelve groups show that IdeaWall effectively presents visual cues to facilitate verbal creative collaboration for idea generation and sets the stage for future research on intelligent systems that assist collaborative work.
Yang Shi 0007, Ye Qi, Xiaoyao Xu, Kwan-Liu Ma
CSCW1
2016 Dimension reconstruction for visual exploration of subspace clusters in high-dimensional data
abstract
Subspace-based analysis has increasingly become the preferred method for clustering high-dimensional data. A visually interactive exploration of subspaces and clusters is a cyclic process. Every meaningful discovery will motivate users to re-search subspaces that can provide improved clustering results and reveal the relationships among clusters that can hardly coexist in the original subspaces. However, the combination of dimensions from the original subspaces is not always effective in finding the expected subspaces. In this study, we present an approach that enables users to reconstruct new dimensions from the data projections of subspaces to preserve interesting cluster information. The reconstructed dimensions are included into an analytical workflow with the original dimensions to help users construct target-oriented subspaces which clearly display informative cluster structures. We also provide a visualization tool that assists users in the exploration of subspace clusters by utilizing dimension reconstruction. Several case studies on synthetic and real-world data sets have been performed to prove the effectiveness of our approach. Lastly, further evaluation of the approach has been conducted via expert reviews.
Juncai Li, Wei Huang 0025, Ying Zhao 0001, Xiaoru Yuan, Xing Liang, Yang Shi 0007
PacificVis7
2016 IDSPlanet: A Novel Radial Visualization of Intrusion Detection Alerts
abstract
In this article, we present a novel radial visualization of IDS alerts, named IDSPlanet, which helps administrators identify false positives, analyze attack patterns, and understand evolving network conditions. Inspired by celestial bodies, IDSPlanet is composed of Chrono Rings, Alert Continents, and Interactive Core. These components correspond with temporal features of alert types, patterns of behavior in affected hosts, and correlations amongst alert types, attackers and targets. The visualization provides an informative picture for the status of the network. In addition, IDSPlanet offers different interactions and monitoring modes, which allow users to interact with high-interest individuals in detail as well as to explore overall pattern.
Yang Shi 0007, Yaoxue Zhang, Ying Zhao 0001, Guojun Wang 0001, Ronghua Shi, Xing Liang
VINCI1
2015 Extending Dimensions in Radviz based on mean shift
abstract
Radviz is a radial visualization technique which maps data from multiple dimensional space onto a planar picture. The dimensions placed on the circumference of a circle, called Dimension Anchors (DAs), can be reordered to reveal different patterns in the dataset. Extending the number of dimensions can enhance the flexibility in the placement of the DAs to explore more meaningful visualizations. In this paper, we describe a method which rationally extends a dimension to multiple new dimensions in Radviz. This method first calculates the probability distribution histogram of a dimension. The mean shift algorithm is applied to get centers of probability density to segment the histogram, and then the dimension can be extended according to the number of segments of the histogram. We also suggest using the Dunn's index to find the optimal placement of DAs, so the better effect of visual clustering could be achieved after the dimension expansion in Radviz. Finally, we demonstrate the usability of our approach on visually analysing the iris data and two other datasets.
Wei Huang 0025, Juncai Li, Yezi Huang, Yang Shi 0007, Ying Zhao 0001
PacificVis5