Music-induced painting is a unique artistic practice, where visual artworks are created under the influence of music. Evaluating whether a painting faithfully reflects the music that inspired it poses a challenging perceptual assessment task. Existing methods primarily rely on emotion recognition models to assess the similarity between music and painting, but such models introduce considerable noise and overlook broader perceptual cues beyond emotion. To address these limitations, we propose a novel framework for music-induced painting assessment that directly models perceptual coherence between music and visual art. We introduce MPD, the first large-scale dataset of music–painting pairs annotated by domain experts based on perceptual coherence. To better handle ambiguous cases, we further collect pairwise preference annotations. Building on this dataset, we present MPJudge, a model that integrates music features into a visual encoder via a modulation-based fusion mechanism. To effectively learn from ambiguous cases, we adopt Direct Preference Optimization for training. Extensive experiments demonstrate that our method outperforms existing approaches. Qualitative results further show that our model more accurately identifies music-relevant regions in paintings.

Corporate financial news is pivotal for market decisions, but often overwhelms general audiences. While data videos effectively bridge this comprehension gap, their production remains a bottleneck for journalists. We present NewsVis, an authoring tool powered by Generative Artificial Intelligence (GenAI) that automates the transformation of unstructured narratives and raw financial datasets into professional data videos. Unlike generic models, our pipeline ensures factual accuracy through a domain-specific taxonomy of financial attributes and optimizes visual information presentation via a multimodal layout algorithm. Additionally, a human-in-the-loop interface empowers journalists to audit and calibrate generative outputs. Comprehensive quantitative and qualitative evaluations demonstrate that NewsVis significantly reduces production barriers while enhancing information accessibility for viewers.

Accurate and reliable visualization of spatiotemporal sensor data such as environmental parameters and meteorological conditions is crucial for informed decision-making. Traditional spatial interpolation methods, however, often fall short of producing reliable interpolation results due to the limited and irregular sensor coverage. This paper introduces a novel spatial interpolation pipeline that achieves reliable interpolation results and produces a novel heatmap representation with uncertainty information encoded. We leverage imputation reference data from Graph Neural Networks (GNNs) to enhance visualization reliability and temporal resolution. By integrating Principal Neighborhood Aggregation (PNA) and Geographical Positional Encoding (GPE), our model effectively learns the spatiotemporal dependencies. Furthermore, we propose an extrinsic, static visualization technique for interpolation-based heatmaps that effectively communicates the uncertainties arising from various sources in the interpolated map. Through a set of use cases, extensive evaluations on real-world datasets, and user studies, we demonstrate our model's superior performance for data imputation, the improvements to the interpolant with reference data, and the effectiveness of our visualization design in communicating uncertainties.

The integrity of data visualizations is increasingly threatened by image editing techniques that enable subtle yet deceptive tampering. Through a formative study, we define this challenge and categorize tampering techniques into two primary types: data manipulation and visual encoding manipulation. To address this, we present VizDefender, a framework for tampering detection and analysis. The framework integrates two core components: 1) a semi-fragile watermark module that protects the visualization by embedding a location map to images, which allows for the precise localization of tampered regions while preserving visual quality, and 2) an intent analysis module that leverages Multimodal Large Language Models (MLLMs) to interpret manipulation, inferring the attacker's intent and misleading effects. Extensive evaluations and user studies demonstrate the effectiveness of our methods.

Text-to-3D (T23D) generation has transformed digital content creation, yet remains bottlenecked by blind trial-and-error prompting processes that yield unpredictable results. While visual prompt engineering has advanced in text-to-image domains, its application to 3D generation presents unique challenges requiring multi-view consistency evaluation and spatial understanding. We present Sel3DCraft, a visual prompt engineering system for T23D that transforms unstructured exploration into a guided visual process. Our approach introduces three key innovations: a dual-branch structure combining retrieval and generation for diverse candidate exploration; a multi-view hybrid scoring approach that leverages MLLMs with innovative high-level metrics to assess 3D models with human-expert consistency; and a prompt-driven visual analytics suite that enables intuitive defect identification and refinement. Extensive testing and a user study demonstrate that Sel3DCraft surpasses other T23D systems in supporting creativity for designers.

The dissemination of visualizations is primarily in the form of raster images, which often results in the loss of critical information such as source code, interactive features, and metadata. While previous methods have proposed embedding metadata into images to facilitate Visualization Image Data Retrieval (VIDR), most existing methods lack practicability since they are fragile to common image tampering during online distribution such as cropping and editing. To address this issue, we propose VisGuard, a tamper-resistant VIDR framework that reliably embeds metadata link into visualization images. The embedded data link remains recoverable even after substantial tampering upon images. We propose several techniques to enhance robustness, including repetitive data tiling, invertible information broadcasting, and an anchor-based scheme for crop localization. VisGuard enables various applications, including interactive chart reconstruction, tampering detection, and copyright protection. We conduct comprehensive experiments on VisGuard's superior performance in data retrieval accuracy, embedding capacity, and security against tampering and steganalysis, demonstrating VisGuard's competence in facilitating and safeguarding visualization dissemination and information conveyance.

Image steganography can hide information in a host image and obtain a stego image that is perceptually indistinguishable from the original one. This technique has tremendous potential in scenarios like copyright protection and information retrospection. Some previous studies have proposed to enhance the robustness of the methods against image disturbances to increase their applicability. However, they generally cannot achieve a satisfying balance between the steganography quality and robustness. Instead of image-in-image steganography, we focus on the issue of message-in-image embedding that is robust to various real- world image distortions. This task aims to embed information into a natural image and the decoding result is required to be completely accurate, which increases the difficulty of data concealing and revealing. Inspired by the recent developments in transformer-based vision models, we discover that the tokenized representation of image is naturally suitable for steganography task. In this paper, we propose a novel message embedding framework, called Robust Message Steganography (RMSteg), which is competent to hide message via QR Code in a host image based on an normalizing flow-based model. The stego image derived by our method has imperceptible changes and the encoded message can be accurately restored even if the image is printed out and photographed. To our best knowledge, this is the first work that integrates the advantages of transformer models into normalizing flow. The code is available at https://github.com/huayuan4396/RMSteg.

Text-to-image (T2I) generation has made remarkable progress in producing high-quality images, but a fundamental challenge remains: creating backgrounds that naturally accommodate text placement without compromising image quality. This capability is non-trivial for real-world applications like graphic design, where clear visual hierarchy between content and text is essential. Prior work has primarily focused on arranging layouts within existing static images, leaving unexplored the potential of T2I models for generating text-friendly backgrounds. We present TextCenGen, a training-free approach that actively relocates objects before optimizing text regions, rather than directly reducing cross-attention which degrades image quality. Our method introduces: (1) a force-directed graph approach that detects conflicting objects and guides them relocation using cross-attention maps, and (2) a spatial attention constraint that ensures smooth background generation in text regions. Our method is plug-and-play, requiring no additional training while well balancing both semantic fidelity and visual quality. Evaluated on our proposed text-friendly T2I benchmark of 27,000 images across three seed datasets, TextCenGen outperforms existing methods by achieving 23\% lower saliency overlap in text regions while maintaining 98\% of the original semantic fidelity measured by CLIP score and our proposed Visual-Textual Concordance Metric (VTCM).

Artistic image assessment has become a prominent research area in computer vision. In recent years, the field has witnessed a proliferation of datasets and methods designed to evaluate the aesthetic quality of paintings. However, most existing approaches focus solely on static final images, overlooking the dynamic and multi-stage nature of the artistic painting process. To address this gap, we propose a novel framework for human-aligned assessment of painting processes. Specifically, we introduce the Painting Process Assessment Dataset (PPAD)-the first large-scale dataset comprising real and synthetic painting process images, annotated by domain experts across eight detailed attributes. Furthermore, we present PPJudge (Painting Process Judge), a Transformer-based model enhanced with temporally-aware positional encoding and a heterogeneous mixture-of-experts architecture, enabling effective assessment of the painting process. Experimental results demonstrate that our method outperforms existing baselines in accuracy, robustness, and alignment with human judgment, offering new insights into computational creativity and art education.

Emotion alignment between music and palettes is crucial for effective multimedia content, yet misalignment creates confusion that weakens the intended message. However, existing methods often generate only a single dominant color, missing emotion variation. Others rely on indirect mappings through text or images, resulting in the loss of crucial emotion details. To address these challenges, we present Music2Palette, a novel method for emotion-aligned color palette generation via cross-modal representation learning. We first construct MuCED, a dataset of 2,634 expert-validated music-palette pairs aligned through Russell-based emotion vectors. To directly translate music into palettes, we propose a cross-modal representation learning framework with a music encoder and color decoder. We further propose a multi-objective optimization approach that jointly enhances emotion alignment, color diversity, and palette coherence. Extensive experiments demonstrate that our method outperforms current methods in interpreting music emotion and generating attractive and diverse color palettes. Our approach enables applications like music-driven image recoloring, video generating, and data visualization, bridging the gap between auditory and visual emotion experiences.

Scientific discoveries increasingly rely on complex multimodal reasoning based on information-intensive scientific data and domain-specific expertise. Empowered by expert-level scientific benchmarks, scientific Multimodal Large Language Models (MLLMs) hold the potential to significantly enhance this discovery process in realistic workflows. However, current scientific benchmarks mostly focus on evaluating the knowledge understanding capabilities of MLLMs, leading to an inadequate assessment of their perception and reasoning abilities. To address this gap, we present the Scientists’ First Exam (SFE) benchmark, designed to evaluate the scientific cognitive capacities of MLLMs through three interconnected levels: scientific signal perception, scientific attribute understanding, scientific comparative reasoning. Specifically, SFE comprises 830 expert-verified VQA pairs across three question types, spanning 66 multimodal tasks across five high-value disciplines. Extensive experiments reveal that current state-of-the-art GPT-o3 and InternVL-3 achieve only 34.08% and 26.52% on SFE, highlighting significant room for MLLMs to improve in scientific realms. We hope the insights obtained in SFE will facilitate further developments in AI-enhanced scientific discoveries.

Abstract As urbanization accelerates, urban performance has become a growing concern, impacting every aspect of residents' lives. However, urban performance exploration is a tedious and highly subjective process for users. Users need to manually collect and integrate various information, or spend a large amount of time and effort due to the steep learning curves of existing specialized tools. To address these challenges, we introduce SUPQA, a novel approach for urban performance exploration using natural language as input and interactive geographic visualizations as output. Our approach leverages Large Language Models (LLMs) to effectively interpret user intents and quantify various urban performance measures. We integrate progressive navigation and multi‐geographic scale analysis in our visualization system, explaining the reasoning process and streamlining users' decision‐making workflow. Two usage scenarios and evaluations demonstrate the effectiveness of SUPQA in helping residents and planners acquire desired information more efficiently and enhancing the quality of decision‐making.

Learning 3-D structures from incomplete point clouds with extreme sparsity and random distributions is a challenge since it is difficult to infer topological connectivity and structural details from fragmentary representations. Missing large portions of informative structures further aggravates this problem. To overcome this, a novel graph convolutional network (GCN) called dynamic and structure-aware NETwork (DSANet) is presented in this article. This framework is formulated based on a pyramidic auto-encoder (AE) architecture to address accurate structure reconstruction on the sparse and incomplete point clouds. A PointNet-like neural network is applied as the encoder to efficiently aggregate the global representations of coarse point clouds. On the decoder side, we design a dynamic graph learning module with a structure-aware attention (SAA) to take advantage of the topology relationships maintained in the dynamic latent graph. Relying on gradually unfolding the extracted representation into a sequence of graphs, DSANet is able to reconstruct complicated point clouds with rich and descriptive details. To associate analogous structure awareness with semantic estimation, we further propose a mechanism, called structure similarity assessment (SSA). This method allows our model to surmise semantic homogeneity in an unsupervised manner. Finally, we optimize the proposed model by minimizing a new distortion-aware objective end-to-end. Extensive qualitative and quantitative experiments demonstrate the impressive performance of our model in reconstructing unbroken 3-D shapes from deficient point clouds and preserving semantic relationships among different regional structures.

Incorporating automatic style extraction and transfer from existing well-designed graph visualizations can significantly alleviate the designer's workload. There are many types of graph visualizations. In this paper, our work focuses on node-link diagrams. We present a novel approach to streamline the design process of graph visualizations by automatically extracting visual styles from well-designed examples and applying them to other graphs. Our formative study identifies the key styles that designers consider when crafting visualizations, categorizing them into global and local styles. Leveraging deep learning techniques such as saliency detection models and multi-label classification models, we develop end-to-end pipelines for extracting both global and local styles. Global styles focus on aspects such as color scheme and layout, while local styles are concerned with the finer details of node and edge representations. Through a user study and evaluation experiment, we demonstrate the efficacy and time-saving benefits of our method, highlighting its potential to enhance the graph visualization design process.

The Aesthetics Assessment of Children's Paintings (AACP) is an important branch of the image aesthetics assessment (IAA), playing a significant role in children's education. This task presents unique challenges, such as limited available data and the requirement for evaluation metrics from multiple perspectives. However, previous approaches have relied on training large datasets and subsequently providing an aesthetics score to the image, which is not applicable to AACP. To solve this problem, we construct an aesthetics assessment dataset of children's paintings and a model based on self-supervised learning. 1) We build a novel dataset composed of two parts: the first part contains more than 20k unlabeled images of children's paintings; the second part contains 1.2k images of children's paintings, and each image contains eight attributes labeled by multiple design experts. 2) We design a pipeline that includes a feature extraction module, perception modules and a disentangled evaluation module. 3) We conduct both qualitative and quantitative experiments to compare our model's performance with five other methods using the AACP dataset. Our experiments reveal that our method can accurately capture aesthetic features and achieve state-of-the-art performance.

The voluminous nature of geospatial temporal data from physical monitors and simulation models poses challenges to efficient data access, often resulting in cumbersome temporal selection experiences in web-based data portals. Thus, selecting a subset of time steps for prioritized visualization and pre-loading is highly desirable. Addressing this issue, this paper establishes a multifaceted definition of salient time steps via extensive need-finding studies with domain experts to understand their workflows. Building on this, we propose a novel approach that leverages autoencoders and dynamic programming to facilitate user-driven temporal selections. Structural features, statistical variations, and distance penalties are incorporated to make more flexible selections. User-specified priorities, spatial regions, and aggregations are used to combine different perspectives. We design and implement a web-based interface to enable efficient and context-aware selection of time steps and evaluate its efficacy and usability through case studies, quantitative evaluations, and expert interviews.

Music and visual arts are essential in children’s arts education, and their integration has garnered significant attention. Existing data analysis methods for exploring audio-visual correlations are limited. Yet, relevant research is necessary for innovating and promoting arts integration courses. In our work, we collected substantial volumes of music-inspired doodles created by children and interviewed education experts to comprehend the challenges they encountered in the relevant analysis. Based on the insights we obtained, we designed and constructed an interactive visualization system DoodleTunes. DoodleTunes integrates deep learning-driven methods for automatically annotating several types of data features. The visual designs of the system are based on a four-level analysis structure to construct a progressive workflow, facilitating data exploration and insight discovery between doodle images and corresponding music pieces. We evaluated the accuracy of our feature prediction results and collected usage feedback on DoodleTunes from five domain experts.

In the field of financial analysis, investment research analysts depend on a detailed understanding of complex financial documents to guide their decision-making process. Charts, while providing visual insights into data, present challenges in summarization. To address this issue, we present a novel approach that leverages contextual awareness, both in terms of textual semantics and visual perception. Our method begins with object detection technology to accurately locate and identify charts. Subsequently, a pre-trained language model is employed for vectorizing text and chart captions, enabling effective correlation between charts and their textual descriptions. Utilizing a large language model and strategic prompt engineering, we generate concise yet informative chart summaries, and incorporate visual saliency to assign scores, quantifying the importance of each chart for more effective data interpretation. Our study, supported by dedicated datasets, validates efficiency and accuracy improvements in financial analysis, expediting well-informed investment decisions.

Dimensionality reduction (DR), also known as projection, is one of the most commonly used methods for visualizing high-dimensional data. Despite its effectiveness in handling large datasets with high dimensions, users often face the challenge of tuning the parameters for optimal performance. Additionally, due to the lack of intuitive standards, users often struggle to quickly identify satisfactory results from the vast number of possible outcomes. Therefore, enhancing the usability of DR algorithms is an urgent problem that needs to be addressed. In this paper, we present a method based on generative models aimed at circumventing the parameter tuning process for DR. Furthermore, to provide users with valid recommendations, we introduce mixed quality metrics based on visual saliency for visualizing DR results. These quality metrics are mapped to a continuous latent space constructed by the generative model using interpolation. We demonstrate the validity and effectiveness of our method through a series of quantitative experiments. Subsequently, we develop a visual interface that combines the proposed method and metrics. The evaluation results demonstrate that our method can quickly recommend good DR results, leading to a more user-friendly and efficient visualization analysis experience.

Visual object tracking aims to locate a targeted object in a video sequence based on an initial bounding box. Recently, Vision-Language~(VL) trackers have proposed to utilize additional natural language descriptions to enhance versatility in various applications. However, VL trackers are still inferior to State-of-The-Art (SoTA) visual trackers in terms of tracking performance. We found that this inferiority primarily results from their heavy reliance on manual textual annotations, which include the frequent provision of ambiguous language descriptions. In this paper, we propose ChatTracker to leverage the wealth of world knowledge in the Multimodal Large Language Model (MLLM) to generate high-quality language descriptions and enhance tracking performance. To this end, we propose a novel reflection-based prompt optimization module to iteratively refine the ambiguous and inaccurate descriptions of the target with tracking feedback. To further utilize semantic information produced by MLLM, a simple yet effective VL tracking framework is proposed and can be easily integrated as a plug-and-play module to boost the performance of both VL and visual trackers. Experimental results show that our proposed ChatTracker achieves a performance comparable to existing methods.

As urban populations grow, effectively accessing urban performance measures such as livability and comfort becomes increasingly important due to their significant socioeconomic impacts. While Point of Interest (POI) data has been utilized for various applications in location-based services, its potential for urban performance analytics remains unexplored. In this article, we present SenseMap, a novel approach for analyzing urban performance by leveraging POI data as a semantic representation of urban functions. We quantify the contribution of POIs to different urban performance measures by calculating semantic textual similarities on our constructed corpus. We propose Semantic-adaptive Kernel Density Estimation which takes into account POIs' influential areas across different Traffic Analysis Zones and semantic contributions to generate semantic density maps for measures. We design and implement a feature-rich, real-time visual analytics system for users to explore the urban performance of their surroundings. Evaluations with human judgment and reference data demonstrate the feasibility and validity of our method. Usage scenarios and user studies demonstrate the capability, usability and explainability of our system.

Color has been widely used to encode data in all types of visualizations. Effective color palettes contain discriminable and harmonious colors, which allow information from visualizations to be accurately and aesthetically conveyed. However, predefined color palettes not only lack the flexibility of custom color palette generation but also ignore the context in which the visualizations are used. Designing an effective color palette is a time-consuming and challenging process for users, even experts. In this work, we propose the generation of an image-based visualization color palette to exploit the human perception of visually appealing images while considering visualization cognition. By analyzing color palette constraints, including harmony, discrimination, and context, we propose an image-driven color generation method. We design a color clustering method in the saliency-hue plane based on visual importance detection and then select the palette based on the visualization color constraints. In addition, we design two color optimization and assignment strategies for visualizations of different data types. Evaluations through numeric indicators and user experiments demonstrate that the palettes predicted by our method are visually related to the original images and are aesthetically pleasing, supporting diverse visualization contexts and data types in practical applications.

DNGs are diverse network graphs with texts and different styles of nodes and edges, including mind maps, modeling graphs, and flowcharts. They are high-level visualizations that are easy for humans to understand but difficult for machines. Inspired by the process of human perception of graphs, we propose a method called GraphDecoder to extract data from raster images. Given a raster image, we extract the content based on a neural network. We built a semantic segmentation network based on U-Net. We increase the attention mechanism module, simplify the network model, and design a specific loss function to improve the model's ability to extract graph data. After this semantic segmentation network, we can extract the data of all nodes and edges. We then combine these data to obtain the topological relationship of the entire DNG. We also provide an interactive interface for users to redesign the DNGs. We verify the effectiveness of our method by evaluations and user studies on datasets collected on the internet and generated datasets.

We present InvVis, a new approach for invertible visualization, which is reconstructing or further modifying a visualization from an image. InvVis allows the embedding of a significant amount of data, such as chart data, chart information, source code, etc., into visualization images. The encoded image is perceptually indistinguishable from the original one. We propose a new method to efficiently express chart data in the form of images, enabling large-capacity data embedding. We also outline a model based on the invertible neural network to achieve high-quality data concealing and revealing. We explore and implement a variety of application scenarios of InvVis. Additionally, we conduct a series of evaluation experiments to assess our method from multiple perspectives, including data embedding quality, data restoration accuracy, data encoding capacity, etc. The result of our experiments demonstrates the great potential of InvVis in invertible visualization.

Graphs are common charts used to represent the topological relationship between nodes. It is a powerful tool for data analysis and information retrieval tasks involve asking questions about graphs. In formative study, we found that questions for graphs are not only about the relationship of nodes but also about the properties of graph elements. We propose a pipeline to answer natural language questions about graph visualizations and generate visual answers. We first extract the data from graphs and convert them into GML format. We design data structures to encode graph information and convert them into an knowledge base. We then extract topic entities from questions. We feed questions, entities and knowledge bases into our question-answer model to obtain the SPARQL queries for textual answers. Finally, we design a module to present the answers visually. A user study demonstrates that these visual and textual answers are useful, credible and and transparent.

Dynamic graph learning has received increasing attention in recent years. However, real-world graph data sets are characterized by significant structural complexity, attribute diversity, and temporal variability. Importantly, there are complex and significant influence mechanisms between them. All them pose great challenges to dynamic graph learning (DGL). To address them, we propose a novel dynamic graph learning framework, MTT-DynGL. First, graph attention networks (GAT) is used to efficiently aggregate the topology and multidimensional attribute features on each snapshot. Then, a temporal variation matrix with strength factors is designed to further measure the interaction mechanism between structures and attributes over time. Further, to effectively integrate the above results, a MTT-based dynamic graph learning network is designed. It consists of an MTT integration mechanism and a bidirectional dilated causal convolution network. The former is used to learn temporal variation features in an integrated manner, and the latter is used to improve learning quality and training efficiency. Finally, the effectiveness of our method is verified by multiple experiments.

The evaluation of a company's value can serve as a guide for investors to assess the company and make informed investment decisions. However, conventional valuation techniques are not applicable to Initial Public Offering (IPO) companies in China, mainly due to the absence of historical market performance. In contrast, a company's finance statement provides a periodic overview of the company's operational and production activities, which is linked to its market performance. Traditional methods often rely on the selection of a limited number of financial indicators from the finance statement and the application of regression analysis. These approaches fail to fully exploit the comprehensive data available in the finance statement. This study proposes a comprehensive method that leverages all relevant information contained in the finance statement, including industry interconnections, financial indices, and additional insights obtained from the report. The structured data is analyzed through tree models, while the interrelationships between different companies are modeled through graph neural networks. Our approach offers a multi-perspective evaluation of IPO companies. The results of our experiments demonstrate that our method can effectively utilize the valuable information in finance statements and improve outcomes.

PNG-8 images are commonly used on the web due to their small size, but their limited color palette often leads to dithering artifacts. Unfortunately, restoring these images using a conventional convolutional neural network (CNN) often results in suboptimal performance since the spatial distribution of dithering is not uniform across the image. This is because the convolutional operator is spatially consistent, meaning it applies the same kernel to all pixels, which we refer to as a global transformation. To address this issue, we propose PNG8IRNet, one approach that combines global and self-conditioned transformations to remove dithering artifacts. Our method incorporates a multilayer perceptron (MLP) to generate diverse kernels for each pixel, taking into account the spatial non-uniformity of dithering, which we define as a self-conditioned transformation. PNG8IRNet demonstrates its performance on multiple datasets, substantially enhancing visual comprehension through a comprehensive set of experiments.

We present iARVis, a proof-of-concept toolkit for creating, experiencing, and sharing mobile AR-based information visualization environments. Over the past years, AR has emerged as a promising medium for information and data visualization beyond the physical media and the desktop, enabling interactivity and eliminating spatial limits. However, the creation of such environments remains difficult and frequently necessitates low-level programming expertise and lengthy hand encodings. We present a declarative approach for defining the augmented reality (AR) environment, including how information is automatically positioned, laid out, and interacted with, to improve the efficiency and flexibility of constructing AR-based information visualization environments. We provide fundamental layout and visual components such as the grid, rich text, images, and charts for the development of complex visualization widgets, as well as automatic targeting methods based on image and object tracking for the development of the AR environment. To increase design efficiency, we also provide features such as hot-reload and several creation levels for both novice and advanced users. We also investigate how the augmented reality-based visualization environment could persist and be shared through the internet and provide ways for storing, sharing, and restoring the environment to give a continuous and seamless experience. To demonstrate the viability and extensibility, we evaluate iARVis using a variety of use cases along with performance evaluation and expert reviews.

Network graphs are common visualization charts. They often appear in the form of bitmaps in articles, web pages, magazine prints, and designer sketches. People often want to modify graphs because of their poor design, but it is difficult to obtain their underlying data. In this article, we present VividGraph, a pipeline for automatically extracting and redesigning graphs from static images. We propose using convolutional neural networks to solve the problem of graph data extraction. Our method is robust to hand-drawn graphs, blurred graph images, and large graph images. We also present a graph classification module to make it effective for directed graphs. We propose two evaluation methods to demonstrate the effectiveness of our approach. It can be used to quickly transform designer sketches, extract underlying data from existing graphs, and interactively redesign poorly designed graphs.

The propagation of flight delays is challenging to analyze because delay events depend on multiple variables. This phenomenon has become even worse with the increasing number of aircraft in China, and research into delay propagation has shown limited progress. In this paper, we design a visual analysis system for flight delay propagation. Unlike conventional flight delay research, this work focuses on the flight delay propagation trends in one region and representing the relationship of delays occurring in multiple airports. First, we construct a Bayesian network to analyze the delay parameters and select delay factors for visualization. Second, the system employs a series of visualization methods to present the propagation of flight delays, including density and flow visualizations. Third, the system combines multiple available visual representations for analyzing flight delays from different aspects. We demonstrate our methods with real data in multiple types of cases, and we evaluate our visual design through user studies. The results help identify several benefits of our system and confirm its usefulness for delay propagation analysis.

Automatic typography is important because it helps designers avoid highly repetitive tasks and amateur users achieve high-quality textual layout designs. However, there are often many parameters and complicated aesthetic rules that need to be adjusted in automatic typography work. In this paper, we propose an efficient deep aesthetics learning approach to generate harmonious textual layout over natural images, which can be decomposed into two stages, saliency-aware text region proposal and aesthetics-based textual layout selection. Our method incorporates both semantic features and visual perception principles. First, we propose a semantic visual saliency detection network combined with a text region proposal algorithm to generate candidate text anchors with various positions and sizes. Second, a discriminative deep aesthetics scoring model is developed to assess the aesthetic quality of the candidate textual layouts. We build a new Textual Layout Aesthetics dataset with dense annotations of each image and design a reasonable evaluation metric to compare our method with richer baselines. The results demonstrate that our method can generate harmonious textual layouts in various actual scenarios with better performance.

Visual query of spatiotemporal data is becoming an increasingly important function in visual analytics applications. Various works have been presented for querying large spatiotemporal data in real time. However, the real-time query of spatiotemporal data distribution is still an open challenge. As spatiotemporal data become larger, methods of aggregation, storage and querying become critical. We propose a new visual query system that creates a low-memory storage component and provides real-time visual interactions of spatiotemporal data. We first present a peak-based kernel density estimation method to produce the data distribution for the spatiotemporal data. Then a novel density dictionary learning approach is proposed to compress temporal density maps and accelerate the query calculation. Moreover, various intuitive query interactions are presented to interactively gain patterns. The experimental results obtained on three datasets demonstrate that the presented system offers an effective query for visual analytics of spatiotemporal data.

The current volume of data in the field of securities investment is increasing dramatically. Simultaneously, the linkage of data from multiple parties makes investment reasoning decisions more challenging than ever. In response to this problem, the financial field's knowledge graph can improve the efficiency, depth, and breadth of financial practitioners' information analysis. Some existing financial knowledge graphs analyze the shareholding relationship between companies. Still, because they are limited to observing data from the company's perspective, users without professional industry background cannot quickly find the industry factors of stock market changes. This paper proposes a financial knowledge graph from the industry chain's perspective. This paper builds upstream and downstream relationships between industries through Transformer-based bidirectional encoder to mine potential industry chain associations from text data and completes the long industry chain of the stock market. This paper also builds a visualization system to display and explore the connection between listed companies and industries. Users can inspect the industry chain's composition and each company's revenue status and stock market conditions in the industry chain. The experiment shows that when the market price fluctuation is detected, the stock price fluctuation can be traced back to its origin in the knowledge graph.

Ultrasound images often contain much noise, and the examination process is easily affected by many factors. Therefore, it is often necessary for ultrasound surgeons to have rich experience in accurately identifying neck vascular from ultrasound images. The NVNet proposed in this paper can accurately segment neck vessels and accurately segment carotid intima-media from ultrasound images. We use an improved full-scale skip connection to obtain richer feature information from the encoder and introduce enhanced attention mechanism, making it possible for NVNet to identify neck vascular from ultrasound images containing much noise accurately. Due to the lack of available datasets, we collate an entirely new carotid longitudinal sectional ultrasound dataset and carry out data annotation under ultrasound surgeons' guidance. The experiment is carried out on the collated dataset and another public dataset of cross-sectional ultrasound images, including carotid artery and internal jugular vein. The final experimental results prove that the segmentation accuracy of NVNet exceeds that of many well-known models in recent years.

User-generated online reviews are critical in P2P accommodations. They contain a wealth of information about the opinions and experiences of users, which help better understand consumer decisions and improve products and services. However, the huge volume of reviews makes it difficult for potential customers to gain useful insights and for managers to track customer opinions. To address these problems, we first use topic modeling techniques for customer opinion mining. Then, we build a deep learning network for sentiment analysis. Finally, we perform sentiment analysis of the reviews at the aspect level to obtain the sentiment vector representation of the accommodation. Moreover, we design a visual analytic system with a user-friendly interface to facilitate interactive analysis. Evaluation including user and case studies demonstrates the usefulness and effectiveness of this system.

Connectivity analysis between the components of large evolving systems can reveal significant patterns of interaction. The systems can be simulated by topological graph structures. However, such analysis becomes challenging on large and complex graphs. Tasks such as comparing, searching, and summarizing structures, are difficult due to the enormous number of calculations required. For time-varying graphs, the temporal dimension even intensifies the difficulty. In this article, we propose to reduce the complexity of analysis by focusing on subgraphs that are induced by closely related entities. To summarize the diverse structures of subgraphs, we build a supervised layout-based classification model. The main premise is that the graph structures can induce a unique appearance of the layout. In contrast to traditional graph theory-based and contemporary neural network-based methods of graph classification, our approach generates low costs and there is no need to learn informative graph representations. Combined with temporally stable visualizations, we can also facilitate the understanding of sub-structures and the tracking of graph evolution. The method is evaluated on two real-world datasets. The results show that our system is highly effective in carrying out visual-based analytics of large graphs.

Reference-based color transfer between images has been a fundamental function in image editing. However, existing approaches pay less attention to high-dynamic-range (HDR) images. It is worth noting that designing an appropriate representation for HDR images to achieve satisfying color transfer is challenging. In this paper, we propose an innovative high-dynamic-range image color transfer generative adversarial network (HDRCTGAN) to encode the original image into fine representations that allow transfer of the color of the reference image to the target image. We propose to learn fine representations through a generative adversarial network (GAN) in a self-supervised way. Particularly, the proposed method is self-supervised learning that requires only unlabeled HDR images instead of supervised learning that requires lots of ground truth pairs. HDRCTGAN consists of a generator to transfer the color of the reference image to the target image over the feature domain and a discriminator to suppress the artifacts caused by the generator. We also design a loss function to ensure that HDRCTGAN possesses two required properties: (a) high fidelity and (b) self-identity. The proposed approach yields a pleasing visual result. We have carried out HDR specific evaluations including both objective quantitative experiments with HDR metrics and subjective user studies operated on HDR display devices to demonstrate the effectiveness of our method. Furthermore, we have verified the applicability of the proposed approach to several applications, such as color transfer of HDR images captured by smartphones, color transfer of fabric images, and reference-based grayscale image colorization.

We present an approach called VisCode for embedding information into visualization images. This technology can implicitly embed data information specified by the user into a visualization while ensuring that the encoded visualization image is not distorted. The VisCode framework is based on a deep neural network. We propose to use visualization images and QR codes data as training data and design a robust deep encoder-decoder network. The designed model considers the salient features of visualization images to reduce the explicit visual loss caused by encoding. To further support large-scale encoding and decoding, we consider the characteristics of information visualization and propose a saliency-based QR code layout algorithm. We present a variety of practical applications of VisCode in the context of information visualization and conduct a comprehensive evaluation of the perceptual quality of encoding, decoding success rate, anti-attack capability, time performance, etc. The evaluation results demonstrate the effectiveness of VisCode.

Automatic typography is important because it helps designers avoid highly repetitive tasks and amateur users achieve high-quality textual layout designs. However, there are often many parameters that need to be adjusted in automatic typography work. In this paper, we propose an efficient content-aware learning-based framework to generate harmonious textual layout over natural image. Our method incorporates both semantic features and visual perception principles. First, we combine a semantic visual saliency detection network with diffusion equations and a text-region proposal algorithm to generate candidate text anchors with various positions and sizes. Second, we develop a deep scoring network to assess the aesthetic quality of the candidate results. We design multiple evaluations to compare our method with several baselines and a commercial poster design tool. The results demonstrate that our method can generate harmonious textual layout in various actual scenarios with better performance.

Recently, generative adversarial networks (GAN) have been widely used to solve image-to-image translation problems such as edges to photos, labels to scenes, and colorizing grayscale images. However, how to recover details of smoothed images is still unexplored. Naively training a GAN like pix2pix causes insufficiently perfect results due to the fact that we ignore two main characteristics including spatial variability and spatial correlation as for this problem. In this work, we propose DeSmoothGAN to utilize both characteristics specifically. The spatial variability indicates that the details of different areas of smoothed images are distinct and they are supposed to be recovered differently. Therefore, we propose to perform spatial feature-wise transformation to recover individual areas differently. The spatial correlation represents that the details of different areas are related to each other. Thus, we propose to apply full attention to consider the relations between them. The proposed method generates satisfying results on several real-world datasets. We have conducted quantitative experiments including smooth consistency and image similarity to demonstrate the effectiveness of DeSmoothGAN. Furthermore, ablation studies are performed to illustrate the usefulness of our proposed feature-wise transformation and full attention.

Abstract In urban analysis, it is desirable to find regions where a primary socio‐economic activity dominates as a key endeavour. This can be accomplished by aggregating neighbouring locations where similar activities take place. However, people move and their activities change over time. Furthermore, the boundaries of regions are not stationary. Thus, it is challenging to update region divisions and track their evolution. Geo‐textual data embody geographical information and activity descriptions. We obtain changes in regional boundaries by iteratively applying a community detection process to a sequence of latent graphs that are constructed from geo‐textual data. Region characteristics are interpreted by topics learned by the latent Dirichlet allocation model. We also propose a matching algorithm to expose region transformations between different timestamps. Interesting patterns of evolution emerge after clustering the migration trajectories of region centroids. In our visual system, users can explore the evolution of regions through animations and linked snapshots. To facilitate visual comparisons, we represent regions by hexagonal tiling that better construct arbitrary regional shapes. The effectiveness of our method is evaluated on two case studies using real‐world datasets, and a user study shows that our visual analytics system is highly effective in performing studies on such regional maps.

The density map is widely used for data sampling, time-varying detection, ensemble representation, etc. The visualization of dynamic evolution is a challenging task when exploring spatiotemporal data. Many approaches have been provided to explore the variation of data patterns over time, which commonly need multiple parameters and preprocessing works. Image generation is a well-known topic in deep learning, and a variety of generating models have been promoted in recent years. In this paper, we introduce a general pipeline called GenerativeMap to extract dynamics of density maps by generating interpolation information. First, a trained generative model comprises an important part of our approach, which can generate nonlinear and natural results by implementing a few parameters. Second, a visual presentation is proposed to show the density change, which is combined with the level of detail and blue noise sampling for a better visual effect. Third, for dynamic visualization of large-scale density maps, we extend this approach to show the evolution in regions of interest, which costs less to overcome the drawback of the learning-based generative model. We demonstrate our method on different types of cases, and we evaluate and compare the approach from multiple aspects. The results help identify the effectiveness of our approach and confirm its applicability in different scenarios.

Abstract Feature matching is the most basic and pervasive problem in computer vision and it has become a primary component in big data analytics. Many tools have been developed for extracting and matching features in video streams and image frames. However, one of the most basic tools, that is, a tool for simply visualizing matched features for the comparison and evaluation of computer vision algorithms is not generally available, especially when dealing with a large number of matching lines. We introduce VisFM, an integrated visual analysis system for comprehending and exploring image feature matchings. VisFM presents a matching view with an intuitive line bundling to provide useful insights regarding the quality of matched features. VisFM is capable of showing a summarization of the features and matchings through group view to assist domain experts in observing the feature matching patterns from multiple perspectives. VisFM incorporates a series of interactions for exploring the feature data. We demonstrate the visual efficacy of VisFM by applying it to three scenarios. An informal expert feedback, conducted by our collaborator in computer vision, demonstrates how VisFM can be used for comparing and analysing feature matchings when the goal is to improve an image retrieval algorithm.

Visualization of customer behavior is urgently needed for an increasing number of customer orders on O2O (online to offline) platform. Although many works have been done on visualizing customer opinion or customer click events of one store, visualizing customer switching behavior among stores is still challenging. The challenge is to show customer order records over time and structure the inter-connection among different stores when customer switching behavior happens. In this work, we focus on Takeout O2O service to present a novel visual analysis system for retailers focusing on customer switching behavior patterns. Firstly we define five customer segments based on switching behavior. Then this system enables temporal-spatial driver exploration for different segments through several interactive views. Moreover, in order to visualize inter-connection sequences, augmented streamgraph with the bundled parallel coordinates is proposed as one alternative technique to visualize temporal event sequences. Case studies through collaboration with domain experts also demonstrate the usefulness and effectiveness of this system in helping customer relationship management.

Abstract Image color editing techniques are of great significance for users who wish to adjust the image color. However, previous works paid less attention to the translucent images. In this paper, we propose a new method to recolor the translucent images while preserving detailed information and color relationships of the source image. We consider the recolor problem as a location transformation problem and solve it in two steps: automatic palette extraction and homography estimation. First, we propose the Hmeans method to extract the dominant colors of the source image based on histogram statistics and clustering. Then, we propose homography estimation to map the source colors to desired colors in the CIE‐LAB color space. Further, we adopt a non‐linear optimization approach to improve the result generated by the last step. The proposed method maintains high fidelity of the source image. Experiments have shown that our method generates a state‐of‐the‐art visual result, in particular in the shadow areas. The source images with ground truth generated by a ray tracer further verify the effectiveness of our method.

Interactive visualization of streaming points for real-time scatterplots and linear blending of correlation patterns is increasingly becoming the dominant mode of visual analytics for both big data and streaming data from active sensors and broadcasting media. To better visualize and interact with inter-stream patterns, it is generally necessary to smooth out gaps or distortions in the streaming data. Previous approaches either animate the points directly or present a sampled static heat-map. We propose a new approach, called StreamMap, to smoothly blend high-density streaming points and create a visual flow that emphasizes the density pattern distributions. In essence, we present three new contributions for the visualization of high-density streaming points. The first contribution is a density-based method called super kernel density estimation that aggregates streaming points using an adaptive kernel to solve the overlapping problem. The second contribution is a robust density morphing algorithm that generates several smooth intermediate frames for a given pair of frames. The third contribution is a trend representation design that can help convey the flow directions of the streaming points. The experimental results on three datasets demonstrate the effectiveness of StreamMap when dynamic visualization and visual analysis of trend patterns on streaming points are required.

Visualization of high density streaming points has become a challenge in information exploration. In this paper, we present a new pipeline for the interactive visualization of large points set. The pipeline is based on the idea that heat-map can overcome the overlapping problem in visualization of high density streaming points. Thus, we firstly define a regular streaming format for large point set which can be updated or changed continually. Based on streaming points, we use kernel density estimation to estimate the point distribution and visualize the density image. Perceptive and interactive features are also considered in our visualization. To our knowledge, our pipeline is the first work that focuses on perceptive visualization of high density streaming points. The main step of our pipeline is accelerated via GPU rendering in order to make scene of real-time interaction in visualization. We demonstrate the visual effectiveness of our pipeline on a geographical dataset of high-density streaming points.

Visual analytics of increasingly large data sets has become a challenge for traditional in-memory and off-line algorithms as well as in the cognitive process of understanding features at various scales of resolution. In this paper, we attempt a new web-based framework for the dynamic visualization of large data. The framework is based on the idea that no physical device can ever catch up to the analytical demand and the physical requirements of large data. Thus, we adopt a data streaming generator model that serializes the original data into multiple streams of data that can be contained on current hardware. Thus, the scalability of the visual analytics of large data is inherent in the streaming architecture supported by our platform. The platform is based on the traditional server-client model. However, the platform is enhanced by effective analytical methods that operate on data streams, such as binned points and bundling lines that reduce and enhance large streams of data for effective interactive visualization. We demonstrate the effectiveness of our framework on different types of large datasets.

Emergency evacuation has many applications in computer animation, virtual reality, architecture planning, safety science, etc. However, current methods most focus on the agent-based modeling and simulation. These simulation results can not consider the human behavior fully and their reliabilities are doubtable. This paper presents a new method to simulate the large-scale crowds in real-time and verify the evacuation data in complex environment. Through analyzing the characteristics of human behavior in emergent condition, a mixed geometry-based ant colony evacuation model is firstly proposed. Then, many behaviors of human are considered to calculate the best evacuation path, including autonomous avoidance, human's warning time, and preferential path selecting. The experimental results show that it is an effective method to simulate large-scale crowds in real time, because the verification makes the simulation more reliable as well as making human behavior logical and the virtual scene realistic.

ABSTRACT The visual world around us displays a rich set of light effects because of translucent and participating media. It is hard and time consuming to render these effects with scattering, caustic, and shaft because of the complex interaction between light and different media. This paper presents a new rendering method based on adaptive lattice for lighting participating media of translucent materials such as marble, wax, and shaft light. Firstly, on the basis of the lattice‐based photon tracing model, multi‐scale hierarchical lattice was constructed by mixed lattice types sampling combined cubic Cartesian and face‐centered cubic with view‐dependent adaptive resolution. Then, an adaptive method to trace diffuse photons and marked specular photons with different phase functions was suggested. Multiple lights and heterogeneous materials were also considered here. Further, the mixed rendering method and GPU accelerate technology were introduced to render different light effects under different participating media. Copyright © 2011 John Wiley & Sons, Ltd.