Deep learning models for natural language processing rely heavily on high-quality labeled datasets. However, existing labeling approaches often struggle to balance label quality with labeling cost. To address this challenge, we propose DALL, a text labeling framework that integrates data programming, active learning, and large language models. DALL introduces a structured specification that allows users and large language models to define labeling functions via configuration, rather than code. Active learning identifies informative instances for review, and the large language model analyzes these instances to help users correct labels and to refine or suggest labeling functions. We implement DALL as an interactive labeling system for text labeling tasks. Comparative, ablation, and usability studies demonstrate DALL’s efficiency, the effectiveness of its modules, and its usability.

Content-aware layout generation is crucial in poster design for automatically arranging layout elements. With the data scarcity problem, existing methods mainly employ retrieval augmentation or leverage Large Language Models (LLMs). However, these approaches still face persistent issues such as element overlap, misalignment, and high resource consumption (especially for LLMs). Additionally, these methods ignore and hardly handle layout generation based on pre-existing text canvases (i.e., text-rich images), which are common in real-world poster design. To address these issues, we propose LayoutGD, a graph-based diffusion method that aims to optimize overlap and misalignment while simultaneously enhancing performance on text-rich images. Our method represents all layout elements and image patches as independent nodes, constructs graphs based on specific topologies, and applies Graph Neural Networks (GNNs) to capture their high-dimensional spatial relationships. Furthermore, LayoutGD can process both text-clean and text-rich canvases in a unified framework, benefiting from our Enhance-Branch architecture. Extensive experiments demonstrate that our method achieves the state-of-the-art on various benchmarks. To further validate our performance and facilitate future research in text-rich canvas layout generation, we also construct a challenging text-rich dataset named TRich500, which contains a wide variety of pre-existing text images from the real-world.

Indoor localization is crucial for applications such as navigation, asset tracking, and emergency response. Fingerprint-based methods that use RSSI are widely adopted; however, they fail under large environmental changes. Unmanned Vehicles (UVs) equipped with high precision sensors are able to collect fingerprints, serving as a promising way by forming a Vehicular Crowdsensing (VCS) campaign. In this paper, we propose “BRAVE”, a Bayesian RL Approach for VCS under Environment changing, while introducing a new metric “Calibration Benefit” to explicitly quantify how effectively a learned trajectory updates those regions of the fingerprint database that have changed and matter most for localization. Specifically, we propose a spatial-temporal Bayesian Network(BN) for change detection, a region rearrangement method for fewer restarts, and an optimistic strategy to balance the exploration and exploitation trade-offs in optimizing calibration benefit. Extensive results on two real-world datasets from SML Center (Shanghai) and Haopu Fashion City (Shanghai) demonstrate that BRAVE outperforms eight baselines and the derived dataset has better localization accuracy compared with the original dataset.

Team-based combat scenarios are prevalent in various real-world applications like video gaming. Analyzing tactics in these scenarios is essential for gaining insights into game processes and improving combat behaviors. The decision-making data in team-based combat include character actions, movement trajectories, and event sequences. Existing studies face challenges in visualizing and analyzing combat tactics due to the complexity and the multifaceted characteristics of the decision-making data. To address these challenges, we introduce TactiVis, a visual analytics system designed for analyzing combat decision-making behavior. Using MOBA game as a representative case of team-based combat, TactiVis adopts a macro-to-micro tactics visual analytics framework consisting of three stages: match-level analysis, event-level understanding, and character-level comparison. In the TactiVis system, we introduce the v-storyline visualization, which encodes positions along the vertical axis to reveal tactical patterns. Case studies and a usability study demonstrate the utility and usability of TactiVis for helping analysts understand combat patterns and analyze tactics.

Insights in tabular data capture valuable patterns that help analysts understand critical information.Organizing related insights into visual data stories is crucial for in-depth analysis.However, constructing such stories is challenging because of the complexity of the inherent relations between extracted insights.Users face difficulty sifting through a vast number of discrete insights to integrate specific ones into a unified narrative that meets their analytical goals.Existing methods either heavily rely on user expertise, making the process inefficient, or employ automated approaches that cannot fully capture their evolving goals.In this paper, we introduce InRe-AcTable, a framework that enhances visual data story construction by establishing both structural and semantic connections between data insights.Each user interaction triggers the Acting module, which utilizes an insight graph for structural filtering to narrow the search space, followed by the Reasoning module using the retrievalaugmented generation method based on large language models for semantic filtering, ultimately providing insight recommendations aligned with the user's analytical intent.Based on the InReAcTable framework, we develop an interactive prototype system that guides users to construct visual data stories aligned with their analytical requirements.We conducted a case study and a user experiment to demonstrate the utility and effectiveness of the InReAcTable framework and the prototype system for interactively building visual data stories.

Media bias refers to bias in news reporting and coverage that exists pervasively. By identifying media bias, social scientists can understand the different perspectives held by media outlets in news reporting. Existing studies only focus on the analysis of media bias of isolated incidents, but neglect their sustained characteristics. Thus, they cannot provide a comprehensive understanding of specific news topics. We develop BiaSeer, a visual analytics system for identifying and understanding sustained bias of media outlets. BiaSeer employs an overview-to-detail approach for interactive identification of media bias. The overview assists users in determining the analysis scope of media outlets. In addition, it visualizes the variances in coverage patterns between selected media outlets using a matrix visualization to facilitate the identification of biased news articles. BiaSeer visualizes the sustained bias in the context of the evolution of events. It first summarizes news articles into events based on a keyword co-occurrence graph and then connects events into a narrative structure using a path-aware story tree construction method. In addition, BiaSeer integrates a sustained bias computation algorithm and enables analysts to compare the narrative structures of different media outlets using the juxtaposition-based visualization approach. We conducted a user experiment to validate the effectiveness of BiaSeer in helping social scientists understand news topics and the usability of visualization designs. To examine the effectiveness of BiaSeer, we conducted a case study with social scientists on the topics of the Russia-Ukraine conflict. The results demonstrate the utility and usability of BiaSeer in efficiently analyzing media bias and attaining a well-rounded understanding of news topics.

Embedding visual representations within original hierarchical tables can mitigate additional cognitive load stemming from the division of users' attention. The created hierarchical table visualizations can help users understand and explore complex data with multi-level attributes. However, because of many options available for transforming hierarchical tables and selecting subsets for embedding, the design space of hierarchical table visualizations becomes vast, and the construction process turns out to be tedious, hindering users from constructing hierarchical table visualizations with many data insights efficiently. We propose InsigHTable, a mixed-initiative and insight-driven hierarchical table transformation and visualization system. We first define data insights within hierarchical tables, which consider the hierarchical structure in the table headers. Since hierarchical table visualization construction is a sequential decision-making process, InsigHTable integrates a deep reinforcement learning framework incorporating an auxiliary rewards mechanism. This mechanism addresses the challenge of sparse rewards in constructing hierarchical table visualizations. Within the deep reinforcement learning framework, the agent continuously optimizes its decision-making process to create hierarchical table visualizations to uncover more insights by collaborating with analysts. We demonstrate the usability and effectiveness of InsigHTable through two case studies and sets of experiments. The results validate the effectiveness of the deep reinforcement learning framework and show that InsigHTable can facilitate users to construct hierarchical table visualizations and understand underlying data insights.

When using exploratory visual analysis to examine multivariate hierarchical data, users often need to query data to narrow down the scope of analysis. However, formulating effective query expressions remains a challenge for multivariate hierarchical data, particularly when datasets become very large. To address this issue, we develop a declarative grammar, HiRegEx (Hierarchical data Regular Expression), for querying and exploring multivariate hierarchical data. Rooted in the extended multi-level task topology framework for tree visualizations (e-MLTT), HiRegEx delineates three query targets (node, path, and subtree) and two aspects for querying these targets (features and positions), and uses operators developed based on classical regular expressions for query construction. Based on the HiRegEx grammar, we develop an exploratory framework for querying and exploring multivariate hierarchical data and integrate it into the TreeQueryER prototype system. The exploratory framework includes three major components: top-down pattern specification, bottom-up data-driven inquiry, and context-creation data overview. We validate the expressiveness of HiRegEx with the tasks from the e-MLTT framework and showcase the utility and effectiveness of TreeQueryER system through a case study involving expert users in the analysis of a citation tree dataset.

Historical visualizations are a rich resource for visualization research. While taxonomy is commonly used to structure and understand the design space of visualizations, existing taxonomies primarily focus on contemporary visualizations and largely overlook historical visualizations. To address this gap, we describe an empirical method for taxonomy development. We introduce a coding protocol and the VisTaxa system for taxonomy labeling and comparison. We demonstrate using our method to develop a historical visualization taxonomy by coding 400 images of historical visualizations. We analyze the coding result and reflect on the coding process. Our work is an initial step toward a systematic investigation of the design space of historical visualizations.

Indoor localization is drawing more and more attentions due to the growing demand of various location-based services, where fingerprinting is a popular data driven techniques that does not rely on complex measurement equipment, yet it requires site surveys which is both labor-intensive and time-consuming. Vehicular crowdsensing (VCS) with unmanned vehicles (UVs) is a novel paradigm to navigate a group of UVs to collect sensory data from certain point-of-interests periodically (PoIs, i.e., coverage holes in localization scenarios). In this paper, we formulate the multi-floor indoor fingerprint collection task with periodical PoI coverage requirements as a constrained optimization problem. Then, we propose a multi-agent deep reinforcement learning (MADRL) based solution, “MADRL-PosVCS”, which consists of a primal-dual framework to transform the above optimization problem into the unconstrained duality, with adjustable Lagrangian multipliers to ensure periodic fingerprint collection. We also propose a novel intrinsic reward mechanism consists of the mutual information between a UV’s observations and environment transition probability parameterized by a Bayesian Neural Network (BNN) for exploration, and a elevator-based reward to allow UVs to go cross different floors for collaborative fingerprint collections. Extensive simulation results on three real-world datasets in SML Center (Shanghai), Joy City (Hangzhou) and Haopu Fashion City (Shanghai) show that MADRL-PosVCS achieves better results over four baselines on fingerprint collection ratio, PoI coverage ratio for collection intervals, geographic fairness and average moving distance.

Metaverse is expected to provide mobile users with emerging applications both in regular situation like intelligent transportation services and in emergencies like wireless search and disaster response. These applications are usually associated with stringent quality-of-information (QoI) requirements like throughput and age-of-information (AoI), which can be further guaranteed by using unmanned aerial vehicles (UAVs) as aerial base stations (BSs) to compensate the existing 5G infrastructures. In this paper, we consider a new QoI-aware mobile crowdsensing (MCS) campaign by UAVs which move around and collect data from mobile users wearing metaverse devices. Specifically, we propose “MetaCS”, a multi-agent deep reinforcement learning (MADRL) framework with improvements on a Transformer-based user mobility prediction module between regions and a relational graph learning mechanism to enable the selection of most informative partners to communicate for each UAV. Extensive results and trajectory visualizations on three real mobility datasets in NCSU, KAIST and Beijing show that MetaCS consistently outperforms six baselines in terms of overall QoI index, when varying different numbers of UAVs, throughput requirement, and AoI threshold.

The integrated ground-air-space (GAS) communications system can enhance post-disaster rescue and management efforts when traditional networks fail, by navigating unmanned ground vehicles (UGVs) and unmanned arieal vehicles (UAVs) to collaboratively collect sufficient data from point-of-interests (PoIs) in a timely manner. In this paper, we consider the GAS vehicular crowdsensing (VCS) campaign, where UGVs dispatch and callback UAVs periodically across multiple stops in the workzone, to maximize the total collected amount of data, geographic fairness while minimizing the energy consumption simultaneously. Specifically, we propose an energy-efficient, go-directed hierarchical multi-agent deep reinforcement learning (MADRL) method with discrete diffusion models called “gMADRL-VCS”, to optimize the high-level goal-conditioned navigation policies of UGVs, and the low-level long-term sensing strategies of UAVs. Extensive experimental results on two real-world datasets in Roma, Italy, and Hong Kong SAR, China show that gMADRL-VCS outperforms baselines in terms of energy efficiency, data collection ratio, energy consumption, and UAV-UGV cooperation factor.

Extracting data insights and generating visual data stories from tabular data are critical parts of data analysis. However, most existing studies primarily focus on tabular data stored as flat tables, typically without leveraging the relations between cells in the headers of hierarchical tables. When properly used, rich table headers can enable the extraction of many additional data stories. To assist analysts in visual data storytelling, an approach is needed to organize these data insights efficiently. In this work, we propose CoInsight, a system to facilitate visual storytelling for hierarchical tables by connecting insights. CoInsight extracts data insights from hierarchical tables and builds insight relations according to the structure of table headers. It further visualizes related data insights using a nested graph with edge bundling. We evaluate the CoInsight system through a usage scenario and a user experiment. The results demonstrate the utility and usability of CoInsight for converting data insights in hierarchical tables into visual data stories.

Visually encoding quantitative information associated with graph links is an important problem in graph visualization. A conventional approach is to vary the thickness of lines to encode the strength of connections in node-link diagrams. In this paper, we present Sticky Links, a novel visual encoding method that draws graph links with stickiness. Taking the metaphor of links with glues, sticky links represent connection strength using spiky shapes, ranging from two broken spikes for weak connections to connected lines for strong connections. We conducted a controlled user study to compare the efficiency and aesthetic appeal of stickiness with conventional thickness encoding. Our results show that stickiness enables more effective and expressive quantitative encoding while maintaining the perception of node connectivity. Participants also found sticky links to be more aesthetic and less visually cluttering than conventional thickness encoding. Overall, our findings suggest that sticky links offer a promising alternative to conventional methods for encoding quantitative information in graphs.

Spatial Crowdsourcing (SC) has been proved as an effective paradigm for data acquisition in urban environments. Apart from using human participants, with the rapid development of unmanned vehicles (UVs) technologies, unmanned aerial or ground vehicles (UAVs, UGVs) are equipped with various high-precision sensors, enabling them to become new types of data collectors. However, UGVs’ operational range is constrained by the road network, and UAVs are limited by power supply, it is thus natural to use UGVs and UAVs together as a coalition, and more precisely, UGVs behave as the UAV carriers for range extensions to achieve complicated air-ground SC tasks. In this paper, we propose a novel communication-based multi-agent deep reinforcement learning method called "GARL", which consists of a multi-center attention-based graph convolutional network (GCN) to accurately extract UGV specific features from UGV stop network called "MC-GCN", and a novel GNN-based communication mechanism called "E-Comm" to make the cooperation among UGVs adaptive to constant changing of geometric shapes formed by UGVs. Extensive simulation results on two campuses of KAIST and UCLA campuses show that GARL consistently outperforms eight other baselines in terms of overall efficiency.

Multi-agent deep reinforcement learning (MADRL) has been widely used in many scenarios such as robotics and game AI. However, existing methods mainly focus on the optimization of agents' micro policies without considering the macro strategy. As a result, they cannot perform well in complex or sparse reward scenarios like the StarCraft Multi-Agent Challenge (SMAC) and Google Research Football (GRF). To this end, we propose a hierarchical MADRL framework called "HiMacMic" with dynamic asynchronous macro strategy. Spatially, HiMacMic determines a critical position by using a positional heat map. Temporally, the macro strategy dynamically decides its deadline and updates it asynchronously among agents. We validate HiMacMic in four widely used benchmarks, namely: Overcooked, GRF, SMAC and SMAC-v2 with nine chosen scenarios. Results show that HiMacMic not only converges faster and achieves higher results than ten existing approaches, but also shows its adaptability to different environment settings.

Recommender systems are playing a significant role in modern society to alleviate the information/choice overload problem, since Internet users may feel hard to identify the most favorite items or products from millions of candidates. Thanks to the recent successes in computer vision, auxiliary learning has become a powerful means to improve the performance of a target (primary) task. Even though helpful, the auxiliary learning scheme is still less explored in recommendation models. To integrate the auxiliary learning scheme, we propose a novel meta auxiliary learning framework to facilitate the recommendation model training, i.e., user and item latent representations. Specifically, we construct two self-supervised learning tasks, regarding both users and items, as auxiliary tasks to enhance the representation effectiveness of users and items. Then the auxiliary and primary tasks are further modeled as a meta learning paradigm to adaptively control the contribution of auxiliary tasks for improving the primary recommendation task. This is achieved by an implicit gradient method guaranteeing less time complexity compared with conventional meta learning methods. Via a comparison using four real-world datasets with a number of state-of-the-art methods, we show that the proposed model outperforms the best existing models on the Top-K recommendation by 3% to 23%.

Tabular visualization techniques integrate visual representations with tabular data to avoid additional cognitive load caused by splitting users' attention. However, most of the existing studies focus on simple flat tables instead of hierarchical tables, whose complex structure limits the expressiveness of visualization results and affects users' efficiency in visualization construction. We present HiTailor, a technique for presenting and exploring hierarchical tables. HiTailor constructs an abstract model, which defines row/column headings as biclustering and hierarchical structures. Based on our abstract model, we identify three pairs of operators, Swap/Transpose, ToStacked/ToLinear, Fold/Unfold, for transformations of hierarchical tables to support users' comprehensive explorations. After transformation, users can specify a cell or block of interest in hierarchical tables as a TableUnit for visualization, and HiTailor recommends other related TableUnits according to the abstract model using different mechanisms. We demonstrate the usability of the HiTailor system through a comparative study and a case study with domain experts, showing that HiTailor can present and explore hierarchical tables from different viewpoints. HiTailor is available at https://github.com/bitvis2021/HiTailor.

Declarative grammar is becoming an increasingly important technique for understanding visualization design spaces. The GoTreeScape system presented in the paper allows users to navigate and explore the vast design space implied by GoTree, a declarative grammar for visualizing tree structures. To provide an overview of the design space, GoTreeScape, which is based on an encoder-decoder architecture, projects the tree visualizations onto a 2D landscape. Significantly, this landscape takes the relationships between different design features into account. GoTreeScape also includes an exploratory framework that allows top-down, bottom-up, and hybrid modes of exploration to support the inherently undirected nature of exploratory searches. Two case studies demonstrate the diversity with which GoTreeScape expands the universe of designed tree visualizations for users. The source code associated with GoTreeScape is available at https://github.com/bitvis2021/gotreescape.

Creating thematic sessions based on accepted papers is important to the success of a conference. Facing a large number of papers from multiple topics, conference organizers need to identify the topics of papers and group them into sessions by considering the constraints on session numbers and paper numbers in individual sessions. In this paper, we present a system using visualization and topic modeling to help the construction of conference sessions. The system provides multiple automatically generated session schemes and allows users to create, evaluate, and manipulate paper sessions with given constraints. A case study based on our system on the VAST papers shows that our method can help users successfully construct coherent conference sessions. In addition to conference session management, our method can be extended to other tasks, such as event and class schedule.

We present GoTree, a declarative grammar allowing users to instantiate tree visualizations by specifying three aspects: visual elements, layout, and coordinate system. Within the set of all possible tree visualization techniques, we identify a subset of techniques that are both "unit-decomposable" and "axis-decomposable" (terms we define). For tree visualizations within this subset, GoTree gives the user flexible and fine-grained control over the parameters of the techniques, supporting both explicit and implicit tree visualizations. We developed Tree Illustrator, an interactive authoring tool based on GoTree grammar. Tree Illustrator allows users to create a considerable number of tree visualizations, including not only existing techniques but also undiscovered and hybrid visualizations. We demonstrate the expressiveness and generative power of GoTree with a gallery of examples and conduct a qualitative study to validate the usability of Tree Illustrator.

We propose BarcodeTree (BCT), a novel visualization technique for comparing topological structures and node attribute values of multiple trees. BCT can provide an overview of one hundred shallow and stable trees simultaneously, without aggregating individual nodes. Each BCT is shown within a single row using a style similar to a barcode, allowing trees to be stacked vertically with matching nodes aligned horizontally to ease comparison and maintain space efficiency. We design several visual cues and interactive techniques to help users understand the topological structure and compare trees. In an experiment comparing two variants of BCT with icicle plots, the results suggest that BCTs make it easier to visually compare trees by reducing the vertical distance between different trees. We also present two case studies involving a dataset of hundreds of trees to demonstrate BCT's utility.

In this work, we present Interaction+, a tool that enhances the interactive capability of existing web-based visualizations. Different from the toolkits for authoring interactions during the visualization construction, Interaction+ takes existing visualizations as input, analyzes the visual objects, and provides users with a suite of interactions to facilitate the visual exploration, including selection, aggregation, arrangement, comparison, filtering, and annotation. Without accessing the underlying data or process how the visualization is constructed, Interaction+ is application-independent and can be employed in various visualizations on the web. We demonstrate its usage in two scenarios and evaluate its effectiveness with a qualitative user study.