In this paper, we use data visualization and visual analytics to tackle the IEEE VAST Challenge 2024 Mini Challenge 3. We use TimeArcs, measures of centrality, and multiple anomaly detection algorithms to visualize the evolution of corporate structure and identify suspicious behavior in the Oceanus Fishing Market. We then create an anomaly detection algorithm combining 3 measures of centrality and isolation forest and local outlier factor algorithms to analyze transactional networks over time and rank nodes by an anomaly score. The algorithm is tested on the VAST Challenge data as well as the 1999 DARPA Intrusion Detection Dataset. The algorithm succeeds in ranking many nodes labeled as suspicious during the VAST Challenge with high anomaly scores. It achieves a peak 97.31 percent accuracy on the DARPA dataset. This demonstrates the power of representing different types of data as transactional networks and shows the potential of this algorithm to be used as a versatile tool for other datasets.

This paper proposes a temporal graph approach to visualize and analyze the evolution of gene relationships and nomenclature over time.An interactive web-based tool implements this temporal graph, enabling researchers to traverse a timeline and observe coupled dynamics in network topology and naming conventions.Analysis of a real human genomic dataset reveals emergence of densely interconnected functional modules over time, representing groups of genes involved in key biological processes.For example, the antimicrobial peptide DEFA1A3 shows increased connections to related alpha defensins involved in infection response.Tracking degree and betweenness centrality shifts over time also quantitatively highlights reprioritization of certain genes' topological importance as knowledge advances.Examination of the CNR1 gene encoding the cannabinoid receptor CB1 demonstrates changing synonymous relationships and consolidating naming patterns over time reflecting its unique functional role discovery.The integrated framework interconnecting these topological and nominclature dynamics provides richer contextual insights compared to isolated analysis methods.Overall, this temporal graph approach enables more a holistic study of knowledge evolution to elucidate complex biology.

Failed workloads that consumed significant computational resources in time and space affect the efficiency of HPC data centers significantly and thus limit the amount of scientific work that can be achieved. While the computational power has increased significantly over the years, detection and prediction of workload failures have lagged far behind and will become increasingly critical as the system scale and complexity further increase. In this study, we analyze workload traces collected from a production cluster and train machine learning models on a large amount of data sets to predict workload failures. Our prediction models consist of a queue-time model that estimates the probability of workload failures before execution and a runtime model that predicts failures at runtime. Evaluation results show that the queue-time model and runtime model can predict workload failures with a maximum precision score of 90.61% and 97.75%, respectively. By integrating the runtime model with the job scheduler, it helps reduce CPU time, and memory usage by up to 16.7% and 14.53%, respectively.

Visualization aims to strengthen data exploration and analysis, especially for complex and high-dimensional data. High-performance computing (HPC) systems are typically large and complicated instruments that generate massive performance and operation time series. Monitoring HPC systems’ performance is a daunting task for HPC admins and researchers due to their dynamic natures. This work proposes a visual design using the bipartite graph’s idea to visualize HPC clusters’ structure, metrics, and job scheduling data. We built a web-based prototype, called JobViewer, that integrates advanced methods in visualization and human-computer interaction (HCI) to demonstrate the benefits of visualization in real-time monitoring HPC centers. We also showed real use cases and a user study to validate the efficiency and highlight the current approach’s drawbacks.

The ability to capture common characteristics among complex multi-variate time series variables can profoundly impact big data analytics in uncovering valuable insights into the relationships among them and enabling a dimensionality reduction technique. Two widely used data displays include time series and scatter plots. While the former focuses on the dynamics over time, the latter deals with static relationships among variables. Motivated by these distinctive perspectives, our research aims to maximally utilize the information captured by both at the same time. This paper presents NetScatter, a visual analytic approach to characterizing changes of pairwise relationships in a high-dimensional time series. Unlike most traditional techniques that employ a single perspective of the visual display, our approach combines static perspectives of two variables in multi-variate time series into a single representation by comparing all data instances over two different time steps. The paper also introduces a list of visual features of the representation to capture how overall data evolve. We have implemented a web-based prototype that supports a full range of operations, such as ranking, filtering, and details on demand. The paper illustrates the proposed approach on data of various sizes in different domains to demonstrate its benefits.

Soil properties are vital to profiling and utilizing soil resources. Conventional approaches to measurements of soil properties often involve costly, environmental-unfriendly, and time-consuming laboratory procedures. Conversely, machine learning (ML) and deep learning (DL) are gaining traction in giving rapid, non-destructive, and cost-saving alternatives to predictions of soil properties. These ML/DL models are convenient and fast because they utilize spectral data, such as visible and near-infrared (Vis-NIR) spectra, that can be easily collected using proximal sensors for their training and prediction purposes. However, existing ML/DL approaches to this problem pose several limitations, such as having small sample sizes, needing to divide the sample data into local areas to increase accuracy, and having relatively low accuracy. Therefore, this work experiments various ML/DL methods that leverage Vis-NIR spectra collected from a rather large number of soil samples distributed all over the world to predict $p{H_{{H_2}O}}$ and pHKCl. We then propose a DL method, called RDNet, that outperforms the other existing approaches. We also utilize visualizations to verify if the proposed model learns legitimate information from the training data.

Analyzing temporal event sequences play an important role in many application domains, such as workload behavior analysis, hardware fault diagnosis, and natural disaster resilience. As data volume keeps growing, real-world temporal event sequences are often noisy, missing, and complex, thus making it a daunting task to convey much of the information from a comprehensive overview for analysts. This work proposes a visual approach based on clustering and superimposing to construct a high-level overview of sequential event data while balancing the amount of information and its cardinality. We also implement an interactive prototype, called TimeRadar, that allows domain analysts to simultaneously analyze sequence clustering, extract distribution patterns, drill multiple levels of detail to accelerate the analysis. This work aims to provide an abstracted view of temporal event sequences where significant events are highlighted. The TimeRadar is demonstrated through case studies with real-world temporal datasets of various sizes.

This paper details the methodologies and decisions making processes used while developing the attacking and defending models for the Graph Adversarial Attacks and Defense applied to a large citation graph. To handle the large graphs, our attack strategy is twofold: 1) randomly attack the structure first, 2) keep the structure unchanged, then continue the attack on the features using the gradient-based method. On the other hand, the defender is based on 1) filtering and normalizing the feature data, 2) applying the Graph Convolutional Network model, and 3) selecting the models with the highest accuracy and robustness based on our own attacking data. We applied these strategies in KDD Cup 2020 on Graph Adversarial Attacks and Defense dataset. The attacker can drop the accuracy of a surrogate 2-layer Graph Convolutional Network model from 60% to 30% on the test set. Our defending model has 68% accuracy on the validated data and has 89% of the target labels remained the same while adding fake nodes, generated by our attacking method, to the graph.

The road is vital for many aspects of life, and road maintenance is crucial for human safety. One of the critical tasks to allow timely repair of road damages is to quickly and efficiently detect and classify them. This work details the strategies and experiments evaluated for these tasks. Specifically, we evaluate Detectron2's implementation of Faster R-CNN using different base models and configurations. We also experiment with these approaches using the Global Road Damage Detection Challenge 2020, A Track in the IEEE Big Data 2020 Big Data Cup Challenge dataset. The results show that the X101-FPN base model for Faster R-CNN with Detectron2's default configurations is efficient and general enough to be transferable to different countries in this challenge. This approach results in F1 scores of 51.0% and 51.4% for the test1 and test2 sets of the challenge, respectively. Though the visualizations show good prediction results, the F1 scores are low. Therefore, we also evaluate the prediction results against the existing annotations and discover some discrepancies. Thus, we also suggest strategies to improve the labeling process for this dataset.

Understanding the status of high-performance computing platforms and correlating applications to resource usage provide insight into the interactions among platform components. A lot of efforts have been devoted into developing monitoring solutions; however, a large-scale HPC system usually requires a combination of methods/tools to successfully monitor all metrics, which will lead to a huge effort in configuration and monitoring. Besides, monitoring tools are often left behind in the procurement of large-scale HPC systems. These challenges have motivated the development of a next-generation out-of-the-box monitoring tool that can be easily deployed without losing informative metrics. In this work, we introduce MonSTer, an “out-of-the-box” monitoring tool for high-performance computing platforms. MonSTer uses the evolving specification Redfish to retrieve sensor data from Baseboard Management Controller (BMC), and resource management tools such as Univa Grid Engine (UGE) or Slurm to obtain application information and resource usage data. Additionally, it also uses a time-series database (e.g. InfluxDB) for data storage. MonSTer correlates applications to resource usage and reveals insightful knowledge without having additional overhead on the application and computing nodes. This paper presents the design and implementation of MonSTer, as well as experiences gained through real-world deployment on the 467-node Quanah cluster at Texas Tech University's High Performance Computing Center (HPCC) over the past year. In this work, we introduce MonSTer, an “out-of-the-box” monitoring tool for high-performance computing platforms. MonSTer uses the evolving specification Redfish to retrieve sensor data from Baseboard Management Controller (BMC), and resource management tools such as Univa Grid Engine (UGE) or Slurm to obtain application information and resource usage data. Additionally, it also uses a time-series database (e.g. InfluxDB) for data storage. MonSTer correlates applications to resource usage and reveals insightful knowledge without having additional overhead on the application and computing nodes. This paper presents the design and implementation of MonSTer, as well as experiences gained through real-world deployment on the 467-node Quanah cluster at Texas Tech University's High Performance Computing Center (HPCC) over the past year.

Data visualization blends art and science to convey stories from data via graphical representations. Considering different problems, applications, requirements, and design goals, it is challenging to combine these two components at their full force. While the art component involves creating visually appealing and easily interpreted graphics for users, the science component requires accurate representations of a large amount of input data. With a lack of the science component, visualization cannot serve its role of creating correct representations of the actual data, thus leading to wrong perception, interpretation, and decision. It might be even worse if incorrect visual representations were intentionally produced to deceive the viewers. This work-in-progress paper focuses on misinformation in graphical representations utilizing a case-based learning approach. The misleading data visualization examples are surveyed and projected onto fundamental units of visual communication, such as size, value, shape, size, and position. This work aims at helping viewers understand the root causes of the misuse, as well as provide basic principles for making more effective visualizations.

Visual analytics studies the integration of human perceptional ability and the data exploration process. Besides, high dimensional time series has become more and more popular data in daily applications and various domains. It is essential to study visual techniques for this type of data set. This paper is an applied work that extends successes of the scatterplot matrix (SPLOM) for cross-sectional data to the high dimensional temporal data sets. We have built a web-based application that contains various visual techniques to display the temporal data sets by SPLOM at each snapshot and animate the SPLOM to illustrate the evolution of the data sets. Two real data sets in different domains are utilized to demonstrate the advances of this approach.

Detecting outliers is one of the fundamental tasks in visual analytics and valuable in many application domains, such as suspicious network cyberattack recognition. This paper introduces an approach to analyzing and visualizing high-dimensional time series, focusing on identifying multivariate observations that are significantly different from the others. We also propose a prototype, called MTSAD, to guide users when interactively exploring abnormalities in large time series. The prototype contains two views: the main window provides an overview of identified outliers overtime, the detail window investigates and explores the ranked temporal data entries based on their outlying contributions to the overall plots. The visual interface supports a full range of interactions, such as lensing, brushing and linking, ranking, and filtering. To validate the benefits and usefulness of our approach, we demonstrate MTSAD on real-world datasets of different numbers of attributes.

This paper describes a machine learning approach to the solar flare prediction competition, a track in IEEE Big Data 2019 Big Data Cup. The competition task is to predict whether or not there is a solar flare event basing on a given time series of solar magnetic field parameters. Our method involves exploring and constructing data-driven machine learning models for the classification task of two imbalanced class labels from time series. Specifically, the investigated models include boosting, logistic regression, multilayer perceptron neural network, and long short-term memory neural network. These models have been successfully deployed and combined in an ensemble framework with two tiers in our final proposed solution for this competition. Our proposed approach ranked at the second place in the competition (the first on the private board and the eleventh on the public board).

The giant network of Internet of Things establishes connections between smart devices and people, with protocols to collect and share data. While the data is expanding at a fast pace in this era of Big Data, there are growing concerns about security and privacy policies. In the current Internet of Things ecosystems, at the intersection of the Internet of Things, Big Data, and Cybersecurity lies the subject that attracts the most attention. In aiding users in getting an adequate understanding, this paper introduces HackerNets, an interactive visualization for emerging topics in the crossing of IoT, Big Data, and Cybersecurity over time. To demonstrate the effectiveness and usefulness of HackerNets, we apply and evaluate the technique on the dataset from the social media platform.

In the paper, we focus on the statistical detection and understanding of signaling behavior via various media outlets that are associated with a violent hate crime. Following a body of theory that identifies the links between hate rhetoric with violence, we explore textual patterns of hate speech in traditional and non-traditional media outlets, including television news shows, social media, political blogs, and political forums, to detect an association with hate crime events per time and place. Applying the dynamical system theory, we can characterize the dynamic behaviors of essential keywords. Grouping keyword with similar temporal signatures enables us to highlight important information/topics corresponding to emerging events. We also introduce SpacePhaser, an interactive visual tool for textual pattern analysis. To show the usefulness of SpacePhaser, we demonstrate the results of its applications on various real-world datasets.

Analyzing public user posts and shared information on social media can assist us in measuring various population characteristics, patterns, movements, and as well as the public health conditions. In recent years, researchers have been analyzing social media (such as Facebook or Twitter feeds) to detect and predict various emerging events and market trends. Fewer attentions have been paid to the epidemic of the diseases. In this paper, we present a social media analytics tool, called HealthTvizer, for exploring health awareness using Twitter data through interactive and interconnected multiple views. We use topic modeling to pick the relevant and meaningful terms from more than 57 million tweets. We detect the disease name and related contents which are shared by the users of different geographical locations (mostly in the United States). We believe that the collected geolocations from the users' tweets can reveal the patterns of diseases for a given term which allows a researcher to detect, analyze, and explore information about the diseases and hence take necessary steps to improve public health awareness. We validate the effectiveness of HealthTvizer through an informal user study. The feedback from this study also motivates us on interesting future extensions of the tool.

The explosion of social media has paved a way for big data in which entrepreneurs use this data to find out potential customers, market demands, individual behavior, thereby to improve existing products, to create new products according to users' need, or to analyze and evaluate financial risks. The challenges of the heterogeneity and fragmentation of data make it difficult for analysts to fully exploit the benefit of deluge information. Available statistical software lacks customization and address unknown research questions. This paper proposes FinanViz, a visual analytics tool for analyzing financial news on social media. The principal aim of FinanViz is to observe the dynamic behavior of terms/words over time along with their proximity to other terms/words. The tool provides an intuitive, interactive exploration of the financial topics and what events are emerging in which we would argue that it will give hints for financial marketers in the decision making process.

Visualizing and analyzing geospatial and temporal observations are common tasks for many application domains. In this paper, we introduce UFO Tracker, a visual analytic tool for analyzing unidentified flying object sightings from the National UFO Reporting Center. The goal here is to give the user a higher level view of where different types of sightings occur, to investigate whether sightings are increasing or decreasing over time, to discover the connections between different events which might happen at different geographic areas, and to quickly identify typical incidents at a given period of time without reading the whole sightings through topic modelling. Multiple visualization and data mining techniques are combined to make sense the increasingly large UFO reports which get updated hourly. The usefulness of the application is evaluated through a case study where anon-expert in ufology can find some typical interesting sightings. Our application can also be able to detect some misleading events such as missile launch or fireworks on a specific day through keywords and topic extraction. One limitation of our application is the data which is not up-todate when new sightings are posted since the application pulled and processed data locally. Our initial application targets UFO sighting reports. However, we believe our approach has wider applications in other research domains, such as analyzing text corpus obtained from social media.

Cyber attacks cause great damage to our national security, ranging from individual internet user to biggest governmental/industrial organizations, such as Equifax (Data Breach 145.5 Million Accounts, reported in July 2017) or Uber (Data Breach 57 Million Records, reported in November 2017). The cyber assault has significantly increased in breadth and depth. This paper introduces CVExplorer, a novel interactive system for visualizing cybersecurity threats reported in the National Vulnerability Database. The proposed system aims to work as a reporting and alerting tool that can help enhance the security against cyber attacks can potentially reduce network vulnerabilities. The CVExplorer system containing multiple linked views allows users to visualize the relationships of various dimensions in the large number of vulnerability reports, such as types and levels of vulnerability, vendors, and products. The CVExplorer provides an intuitive interface and supports a range of interactive features, such as filtering and ordering by vulnerability severity ratings, allowing users to narrow down topics of interest quickly. To demonstrate the effectiveness of the proposed system, we demonstrate the CVExplorer on two case studies of Common Vulnerabilities and Exposures retrieved from the National Vulnerability Database.

The "Internet of Things" is changing the way companies operate and consumers behave. Therefore, it is essential to capture trends in "Internet of Things". This paper proposes IoTViz, a visual analytics tool for analyzing "Internet of Things" news on social media. The principal aim of IoTViz is to observe the dynamic behavior of topics along with their proximity to other dimensions such as user comments and ratings in multiple coordinated views. IoTViz provides an interactive exploration of the IoT topics and supports of a range of interactive features, such as linking and filtering, allowing users to narrow down events of interest quickly. It is interesting to filter and visualize IoT news regarding the individual organization, e.g., user opinions/ratings regarding a company or its products.

This demonstration paper introduces MalViz, a visual analytic tool for analyzing malware behavioral patterns through process monitoring events. The goals of this tool are: 1) to investigate the relationship and dependencies among processes interacted with a running malware over a certain period of time, 2) to support professional security experts in detecting and recognizing unusual signature-based patterns exhibited by a running malware, and 3) to help users identify infected system and users' libraries that the malware has reached and possibly tampered. A case study is conducted in a virtual machine environment with a sample of four malware programs. The result of the case study shows that the visualization tool offers a great support for experts in software and system analysis and digital forensics to profile and observe malicious behavior and further identify the traces of affected software artifacts.

This paper introduces CactusTree, a novel visualization technique for representing hierarchical datasets. We introduce details about the construction of CactusTrees and describe how they can be used to represent nested data and relationships between elements in the data. We explain how our design decisions were informed by tasks common to a range of scientific domains. A key contribution of this article is the introduction of descriptive features that can be used to characterize trees in terms of their structural and connective qualities.

Systems biologists and cancer researchers require interactive visualization tools that enable them to more easily navigate and discover patterns at different levels of the biological hierarchy of signaling pathways. Furthermore, biologists are often interested in understanding and exploring the causal biochemical links between processes. When exploring the literature of particular biological pathways or specific proteins within those pathways, biologists find it useful to know the contexts in which biochemical links are active and, importantly, to be aware of potential conflicts when different experiments introduce alternative interpretations of the function of a pathway or biochemical reaction. We introduce BioLinker, a interactive visualization system that helps users to perform bottom-up exploration of complex protein interaction networks. Five interconnected views provide the user with a range of ways to explore pathway data, including views that show potential conflicts within pathway databases and publications and that highlight contextual information about individual proteins. Additionally, we discuss system details to show how our system manages the large amount of protein interactions extracted from the literature of biological pathways.

Abstract In this paper we introduce TimeArcs, a novel visualization technique for representing dynamic relationships between entities in a network. Force‐directed layouts provide a way to highlight related entities by positioning them near to each other Entities are brought closer to each other (forming clusters) by forces applied on nodes and connections between nodes. In many application domains, relationships between entities are not temporally stable, which means that cluster structures and cluster memberships also may vary across time. Our approach merges multiple force‐directed layouts at different time points into a single comprehensive visualization that provides a big picture overview of the most significant clusters within a user‐defined period of time. TimeArcs also supports a range of interactive features, such as allowing users to drill‐down in order to see details about a particular cluster. To highlight the benefits of this technique, we demonstrate its application to various datasets, including the IMDB co‐star network, a dataset showing conflicting evidences within biomedical literature of protein interactions, and collocated popular phrases obtained from political blogs.

A scatter plot displays a relation between a pair of variables. Given a set of v variables, there are v(v- 1)/2 pairs of variables, and thus the same number of possible pair wise scatter plots. Therefore for even small sets of variables, the number of scatter plots can be large. Scatter plot matrices (SPLOMs) can easily run out of pixels when presenting high-dimensional data. We introduce a theoretical method and a testbed for assessing whether our method can be used to guide interactive exploration of high-dimensional data. The method is based on nine characterizations of the 2D distributions of orthogonal pair wise projections on a set of points in multidimensional Euclidean space. Working directly with these characterizations, we can locate anomalies for further analysis or search for similar distributions in a large SPLOM with more than a hundred dimensions. Our testbed, ScagExplorer, is developed in order to evaluate the feasibility of handling huge collections of scatter plots.

Scagnostics (Scatterplot Diagnostics) were developed by Wilkinson et al. based on an idea of Paul and John Tukey, in order to discern meaningful patterns in large collections of scatterplots. The Tukeys' original idea was intended to overcome the impediments involved in examining large scatterplot matrices (multiplicity of plots and lack of detail). Wilkinson's implementation enabled for the first time scagnostics computations on many points as well as many plots. Unfortunately, scagnostics are sensitive to scale transformations. We illustrate the extent of this sensitivity and show how it is possible to pair statistical transformations with scagnostics to enable discovery of hidden structures in data that are not discernible in untransformed visualizations.

We introduce a method (Scagnostic time series) and an application (TimeSeer) for organizing multivariate time series and for guiding interactive exploration through high-dimensional data. The method is based on nine characterizations of the 2D distributions of orthogonal pairwise projections on a set of points in multidimensional euclidean space. These characterizations include measures, such as, density, skewness, shape, outliers, and texture. Working directly with these Scagnostic measures, we can locate anomalous or interesting subseries for further analysis. Our application is designed to handle the types of doubly multivariate data series that are often found in security, financial, social, and other sectors.

Widespread interest in features and trends in time series has generated a need for interactive tools that support discovering unusual events in time series. In this paper, we introduce an application (TimeSeer) for guiding interactive exploration through high-dimensional data. Our application is designed to handle the types of doubly-multivariate data series by working directly on noteworthy features such as density, skewness, shape, outliers, and texture.

In Wilkinson et al. [2011] we introduced a new set-covering random projection classifier that achieved average error lower than that of other classifiers in the Weka platform. This classifier was based on an L ∞ norm distance function and exploited an iterative sequence of three stages (projecting, binning, and covering) to deal with the curse of dimensionality, computational complexity, and nonlinear separability. We now present substantial changes that improve robustness and reduce training and testing time by almost an order of magnitude without jeopardizing CHIRP's outstanding error performance.

We introduce a classifier based on the L-infinity norm. This classifier, called CHIRP, is an iterative sequence of three stages (projecting, binning, and covering) that are designed to deal with the curse of dimensionality, computational complexity, and nonlinear separability. CHIRP is not a hybrid or modification of existing classifiers; it employs a new covering algorithm. The accuracy of CHIRP on widely-used benchmark datasets exceeds the accuracy of competitors. Its computational complexity is sub-linear in number of instances and number of variables and subquadratic in number of classes.

An ongoing challenge for information visualization is how to deal with over-plotting forced by ties or the relatively limited visual field of display devices. A popular solution is to represent local data density with area (bubble plots, treemaps), color (heatmaps), or aggregation (histograms, kernel densities, pixel displays). All of these methods have at least one of three deficiencies:1) magnitude judgments are biased because area and color have convex downward perceptual functions, 2) area, hue, and brightness have relatively restricted ranges of perceptual intensity compared to length representations, and/or 3) it is difficult to brush or link to individual cases when viewing aggregations. In this paper, we introduce a new technique for visualizing and interacting with datasets that preserves density information by stacking overlapping cases. The overlapping data can be points or lines or other geometric elements, depending on the type of plot. We show real-dataset applications of this stacking paradigm and compare them to other techniques that deal with over-plotting in high-dimensional displays.

We introduce a mathematical framework, based on the L∞norm distance metric, to describe human interactions in a visual data mining environment. We use the framework to build a classifier that involves an algebra on hyper-rectangles. Our classifier, called VisClassifier, generates set-wise rules from simple gestures in an exploratory visual GUI. Logging these rules allows us to apply our analysis to a new sample or batch of data so that we can assess the predictive power of our visual-processing motivated classifier. The accuracy of this classifier on widely-used benchmark datasets rivals the accuracy of competitive classifiers.