Although HCI research indicates that lived experiences should be considered in technologies for type 1 diabetes (T1D) management, technologies and prevailing concepts for “successful” management continue to focus narrowly on metrics for improving blood glucose, rather than a holistic understanding of life with T1D. T1D shapes every aspect of life, including personal priorities, values, and needs, making it highly personal and manifesting in different practices. We conducted semi-structured interviews, a 5-day diary study, and a workshop with 10 individuals with T1D to elicit and reflect on different ways of living with T1D. Based on these accounts, we conceptualize self-management styles as a multidimensional construct spanning seven dimensions: values and prioritization, planning, risk-taking, rhythm of care, openness to experiment and learn, engagement in tracking, and success definitions. We contribute an account of these dimensions that designers can use as a lens for developing T1D technologies that support personal ways of living with T1D, foregrounding lived experience and outlining implications for HCI research.

Group accommodation booking forces travelers to coordinate externally through messaging apps and informal voting, missing opportunities for transparent preference alignment. We present PRISM, an interactive group recommender system that transforms opaque recommendation processes into transparent collaborative visual experiences. PRISM employs a two-phase interaction paradigm: individual preference elicitation through conversational AI, followed by collaborative decision-making via bivariate map preference visualization. A controlled user study with 6 pairs shows PRISM enhances transparency (+1.83 on 5-point scale), consensus building (+2.0), and reduces conformity pressure compared to traditional approaches and interfaces.

Music streaming platforms reduce rich listening experiences to algorithmic black boxes, overlooking personal narratives that make music meaningful. We present Blooming Beats, an explainable recommender system that transforms Spotify listening data into visual narratives using Data Humanism principles. The system embodies TRACE principles: Transparency through visual explanations, Context-awareness by integrating personal context, and Empathy by matching listening stories rather than user profiles. A user study with 8 participants exploring a decade of listening data shows that narrative-driven visualization suggests potential for enhancing transparency and engagement.

Current music recommendation systems often lack transparency, preventing users from understanding the recommendations or effectively steering the algorithm. We present HUMMUS, an interactive collaborative music sequential recommender system that applies Giorgia Lupi’s Data Humanism principles to combine algorithmic transparency with human-centered design. HUMMUS visualizes songs as flowers, where petals represent audio features, and connecting lines reveal recommendation relationships. Real-time voting mechanisms during natural pauses in social interaction enable collaborative decision-making between humans and recommendation algorithms. Our mixed-methods evaluation, involving 19 participants, demonstrates that humanistic design principles enhance transparency, user engagement, and collaborative decision-making while maintaining the quality of recommendations. This work contributes to the intersection of critical visualization and explainable AI by demonstrating how Data Humanism can guide human-centered recommendation systems.

With the growing abundance of time series data and anomaly detection algorithms, selecting appropriate algorithm configurations for a given dataset has become increasingly complex. We introduce AnoScout, a Visual Analytics approach to explore anomalies obtained from an algorithm ensemble with the overall goal of acquiring insights into the diversity of anomalies and identifying appropriate algorithms for each anomaly pattern. We employ unsupervised methods1 to address scenarios in which normal behavior is difficult to define, and integrate semi-supervised approaches with projection-based visualizations to support user labeling when normal behavior can be more clearly delineated. Our approach considers ensembles of algorithms, enabling robust coverage across multiple anomaly categories. AnoScout visualizes each algorithm’s contribution to the ensemble to address the challenge of differing detection behaviors across anomaly types. To support analysis in large datasets with potentially many anomalies, we integrate a recommender system that facilitates the identification of relevant anomalies. To acquire insights into recurring anomalies, users can explore anomalies through a clustering view. We demonstrate the practical utility of AnoScout using case studies from two domains: EEG measurements and industrial data analysis, which are known for containing diverse anomalies.

Analyzing tabular data by leveraging hierarchical structures for its items and attributes is a promising approach to scale for dataset sizes that make per-item and per-attribute analysis impractical. Existing approaches face limitations in supporting both item and attribute hierarchies, enabling user-controlled hierarchy creation, and ensuring visual scalability and interaction utility. We present HAXplorer , a visual analytics approach that enables users to create both item and attribute hierarchies, and to explore the resulting tabular data space by leveraging item and attribute aggregates. We demonstrate the generalizability of HAXplorer through usage scenarios across three diverse domains and evaluate its usefulness in a task-based user study. Usability is assessed through a perceived readability questionnaire and qualitative feedback. In addition to introducing a novel visual analytics system, our work offers insights into visual literacy, design validation methodologies, the positioning of HAXplorer within the broader landscape of biclustering techniques, and highlights the generative power of abstraction.

Large unlabeled datasets demand efficient and scalable data labeling solutions, in particular when the number of instances and classes is large. This leads to significant visual scalability challenges and imposes a high cognitive load on the users. Traditional instance-centric labeling methods, where (single) instances are labeled in each iteration struggle to scale effectively in these scenarios. To address these challenges, we introduce cVIL, a Class-Centric Visual Interactive Labeling methodology designed for interactive visual data labeling. By shifting the paradigm from assigning-classes-to-instances to assigning-instances-to-classes , cVIL reduces labeling effort and enhances efficiency for annotators working with large, complex and class-rich datasets. We propose a novel visual analytics labeling interface built on top of the conceptual cVIL workflow, enabling improved scalability over traditional visual labeling. In a user study, we demonstrate that cVIL can improve labeling efficiency and user satisfaction over instance-centric interfaces. The effectiveness of cVIL is further demonstrated through a usage scenario, showcasing its potential to alleviate cognitive load and support experts in managing extensive labeling tasks efficiently.

Recommender systems (RecSys) have become ubiquitous in users’ daily digital interactions, significantly influencing decision-making processes. As these systems grow in algorithmic complexity, effective explanations for non-expert users become essential to fostering understanding and trust. While academic research explores diverse explanation methods, commercial applications predominantly employ textual explanations due to their implementation efficiency and user familiarity. However, the effectiveness of these textual explanations is often compromised by suboptimal presentation within RecSys user interfaces (UIs), leading to reduced user engagement and comprehension. This issue is particularly relevant given the recent emergence of large language models (LLMs) for generating RecSys explanations. We introduce f-RecX, a conceptual framework for characterizing and designing effective textual explanations in RecSys UIs. Based on a two-phase methodology combining qualitative user studies and quantitative evaluations, f-RecX maps four input dimensions (Explanation Style, Goals, Domain Dynamics, and Recommender Systems Technique) to an output dimension focused on visual representation. The framework aims to enhance the’consumability’ of textual explanations by making them easier to locate and comprehend, and more valuable for non-expert users. We demonstrate f-RecX’s applicability through a usage scenario and analysis of existing RecSys UIs, offering valuable insights for enhancing explainability and user experience. • f-RecX uniquely bridges algorithmic and human-centered design by integrating four input dimensions (Explanation Style, Goals, Domain Dynamics, and Recommender Techniques) with visual presentation parameters, addressing the gap between explanation content generation and effective UI implementation. • The framework introduces the concept of explanation ”consumability” - how easily users can locate, understand, and derive value from explanations - providing empirically validated visual design guidelines including optimal font sizes (24-32px), positioning (top-left), and explanation length ( 20 words). • Through a comprehensive two-phase methodology combining empathy workshops and quantitative surveys, f-RecX provides design guidance that demonstrates how visual characteristics significantly impact explanation effectiveness, particularly relevant for LLM-generated explanations in commercial applications.

Time-stamped event sequences (TSEQs) are time-oriented data without value information, shifting the focus of users to the exploration of temporal event occurrences. TSEQs exist in application domains, such as sleeping behavior, earthquake aftershocks, and stock market crashes. Domain experts face four challenges, for which they could use interactive and visual data analysis methods. First, TSEQs can be large with respect to both the number of sequences and events, often leading to millions of events. Second, domain experts need validated metrics and features to identify interesting patterns. Third, after identifying interesting patterns, domain experts contextualize the patterns to foster sensemaking. Finally, domain experts seek to reduce data complexity by data simplification and machine learning support. We present IVESA, a visual analytics approach for TSEQs. It supports the analysis of TSEQs at the granularities of sequences and events, supported with metrics and feature analysis tools. IVESA has multiple linked views that support overview, sort+filter, comparison, details-on-demand, and metadata relation-seeking tasks, as well as data simplification through feature analysis, interactive clustering, filtering, and motif detection and simplification. We evaluated IVESA with three case studies and a user study with six domain experts working with six different datasets and applications. Results demonstrate the usability and generalizability of IVESA across applications and cases that had up to 1,000,000 events.

At the beginning of the COVID-19 pandemic, HealthLink BC (HLBC) rapidly integrated physicians into the triage process of their virtual healthcare service to improve patient outcomes and satisfaction with this service and preserve health care system capacity. We present the design and implementation of a visual analytics tool, VIVA (Virtual healthcare Interactions using Visual Analytics), to support HLBC in analysing various forms of usage data from the service. We abstract HLBC's data and data analysis tasks, which we use to inform our design of VIVA. We also present the interactive workflow abstraction of Scan, Act, Adapt. We validate VIVA's design through three case studies with stakeholder domain experts. We also propose the Controllability Through Configuration model to conduct and analyze design studies, and discuss architectural evolution of VIVA through that lens. It articulates configuration, both that specified by a developer or technical power user and that constructed automatically through log data from previous interactive sessions, as a bridge between the rigidity of hardwired programming and the time-consuming implementation of full end-user interactivity.

In medical diagnostics of both early disease detection and routine patient care, particle-based contamination of in-vitro diagnostics consumables poses a significant threat to patients. Objective data-driven decision-making on the severity of contamination is key for reducing patient risk, while saving time and cost in quality assessment. Our collaborators introduced us to their quality control process, including particle data acquisition through image recognition, feature extraction, and attributes reflecting the production context of particles. Shortcomings in the current process are limitations in exploring thousands of images, data-driven decision making, and ineffective knowledge externalization. Following the design study methodology, our contributions are a characterization of the problem space and requirements, the development and validation of DaedalusData, a comprehensive discussion of our study's learnings, and a generalizable framework for knowledge externalization. DaedalusData is a visual analytics system that enables domain experts to explore particle contamination patterns, label particles in label alphabets, and externalize knowledge through semi-supervised label-informed data projections. The results of our case study and user study show high usability of DaedalusData and its efficient support of experts in generating comprehensive overviews of thousands of particles, labeling of large quantities of particles, and externalizing knowledge to augment the dataset further. Reflecting on our approach, we discuss insights on dataset augmentation via human knowledge externalization, and on the scalability and trade-offs that come with the adoption of this approach in practice.

Self-management of type 1 diabetes (T1D) involves multiple factors, frequent anticipation of changes in blood glucose, and complex decision-making. ML-based blood glucose predictions (BGP) may be valuable in supporting T1D management. However, it may be difficult for people with T1D to integrate BGP into their decision-making due to prediction uncertainty and interpretation. In this study, we investigate the lived experience of people with T1D focusing on their needs and expectations in using apps that provide BGP. We designed MOON-T1D, an app that shows simulated BGP and conducted a five-day study using the Experience Sampling Method coupled with semi-structured interviews with 15 individuals with T1D who used MOON-T1D. A reflexive thematic analysis of our data revealed implications for the design and use of BGP, including the complex role of emotions and trust surrounding predictions, and ways in which BGP may ease or complicate T1D management.

In the early 2000s, the transition to paperless documentation of patients’ health data begun at large scale, with the introduction of Electronic Health and Medical Records (EHR and EMR, respectively). This constituted a paradigm shift in how patient data was stored and exchanged among institutions. The impact of the so-called “Electronic Health Revolution”1 was significant. Standardization of personal health data allowed for a more uniform definition of diagnoses and their ensuing clinical process, with fewer mistakes in diagnosis and treatment, and a more reliable application of medical guidelines.2 For instance, in the United States (US), patients now have control over their information, with more mandated electronic access.3 Recent studies showed that online medical records by US adults doubled over the last 8 years.4 Simultaneously, a new generation of smart, affordable, and wearable devices, such as smartwatches, has emerged. These devices generate fine-grained and continuous data about the health status of their users, with minimal discomfort, eliminating the need for specialized equipment. The rapid evolution of Artificial Intelligence (AI) technologies is about to significantly impact healthcare as well. AI technologies present opportunities and challenges for both physicians and patients.5 AI models recognize patterns in complex datasets, potentially identifying a broader range of disease progression patterns that might not be immediately apparent to clinicians or patients. However, the inherent “black-box” nature of AI has slowed its adoption, as healthcare professionals often struggle to evaluate the underlying process that led to the AI recommendations. In essence, while it can be impressive what AI models predict, concerns remain about why the AI produces a particular output, and how. The considerable lack of transparency impedes trust-building, such that “the doctor just won’t accept that,”6 calling for explainable AI output.

OBJECTIVES: This article describes the design and evaluation of MS Pattern Explorer, a novel visual tool that uses interactive machine learning to analyze fitness wearables' data. Applied to a clinical study of multiple sclerosis (MS) patients, the tool addresses key challenges: managing activity signals, accelerating insight generation, and rapidly contextualizing identified patterns. By analyzing sensor measurements, it aims to enhance understanding of MS symptomatology and improve the broader problem of clinical exploratory sensor data analysis. MATERIALS AND METHODS: Following a user-centered design approach, we learned that clinicians have 3 priorities for generating insights for the Barka-MS study data: exploration and search for, and contextualization of, sequences and patterns in patient sleep and activity. We compute meaningful sequences for patients using clustering and proximity search, displaying these with an interactive visual interface composed of coordinated views. Our evaluation posed both closed and open-ended tasks to participants, utilizing a scoring system to gauge the tool's usability, and effectiveness in supporting insight generation across 15 clinicians, data scientists, and non-experts. RESULTS AND DISCUSSION: We present MS Pattern Explorer, a visual analytics system that helps clinicians better address complex data-centric challenges by facilitating the understanding of activity patterns. It enables innovative analysis that leads to rapid insight generation and contextualization of temporal activity data, both within and between patients of a cohort. Our evaluation results indicate consistent performance across participant groups and effective support for insight generation in MS patient fitness tracker data. Our implementation offers broad applicability in clinical research, allowing for potential expansion into cohort-wide comparisons or studies of other chronic conditions. CONCLUSION: MS Pattern Explorer successfully reduces the signal overload clinicians currently experience with activity data, introducing novel opportunities for data exploration, sense-making, and hypothesis generation.

Item rankings are useful when a decision needs to be made, especially if there are multiple attributes to be considered. However, existing tools do not support both categorical and numerical attributes, require programming expertise for expressing preferences on attributes, do not offer instant feedback, lack flexibility in expressing various types of user preferences, or do not support all mandatory steps in the ranking-creation workflow. In this work, we present RankASco: a human-centered visual analytics approach that supports the interactive and visual creation of rankings. The iterative design process resulted in different visual interfaces that enable users to formalize their preferences based on a taxonomy of attribute scoring functions. RankASco enables broad user groups to (a) select attributes of interest, (b) express preferences on attributes through interactively tailored scoring functions, and (c) analyze and refine item ranking results. We validate RankASco in a user study with 24 participants in comparison to a general purpose tool. We report on commonalities and differences with respect to usefulness and usability and ultimately present three personas that characterize common user behavior in ranking-creation. On the human factors side, we have also identified a series of interesting behavioral variables that have an influence on the task performance and may shape the design of human-centered ranking solutions in the future.

We present ManuKnowVis, the result of a design study, in which we contextualize data from multiple knowledge repositories of a manufacturing process for battery modules used in electric vehicles. In data-driven analyses of manufacturing data, we observed a discrepancy between two stakeholder groups involved in serial manufacturing processes: Knowledge providers (e.g., engineers) have domain knowledge about the manufacturing process but have difficulties in implementing data-driven analyses. Knowledge consumers (e.g., data scientists) have no first-hand domain knowledge but are highly skilled in performing data-driven analyses. ManuKnowVis bridges the gap between providers and consumers and enables the creation and completion of manufacturing knowledge. We contribute a multi-stakeholder design study, where we developed ManuKnowVis in three main iterations with consumers and providers from an automotive company. The iterative development led us to a multiple linked view tool, in which, on the one hand, providers can describe and connect individual entities (e.g., stations or produced parts) of the manufacturing process based on their domain knowledge. On the other hand, consumers can leverage this enhanced data to better understand complex domain problems, thus, performing data analyses more efficiently. As such, our approach directly impacts the success of data-driven analyses from manufacturing data. To demonstrate the usefulness of our approach, we carried out a case study with seven domain experts, which demonstrates how providers can externalize their knowledge and consumers can implement data-driven analyses more efficiently.

This Computers and Graphics Special Section is composed of the three significantly extended papers following the 12th International EuroVis Workshop on Visual Analytics (EuroVA) 2021.EuroVA is a premier forum for Visual Analytics research in Europe and in the world alike.2021 was the 12th annual workshop with a broad spectrum of strong Visual Analytics submissions.We particularly encouraged the submission in current topics of Visual Analytics, including but not limited to Visual Analytics for social good, Visual Analytics of sets, human factors in decision making through Visual Analytics, mixed-initiative approaches and learning from user interaction, COVID-19.EuroVA 2021 took place on Monday, June 14, 2021 (https:// www.eurova.org/eurova-2021/program-2021) in the online world around Zürich, Switzerland.The workshop accepted 15 papers (of four pages) each reviewed by four members of the international program committee.After the workshop, based on the reviews and authors' presentations of their work, the authors of the four highest scoring paper were invited to submit an extended version of their papers to the Computers and Graphics Special Section on EuroVA.Following a full review cycle by three reviewers each, three papers were finally included in this special section.The three extended versions allow uncovering chains of infections through spatio-temporal Covid-19 contact traces [1], the coordination of independent Visual Analytics tools through a data-driven platform [2], and gaining an understanding of multimodal brain network data through an immersive 3D visualization approach [3].We would like to thank the authors for their work, the international program committee of EuroVA 2021, as well as the reviewers of Computers and Graphics who have significantly improved these selected paper with their recommendations.We hope the readers of this special section will enjoy these significantly enhanced versions of the selected papers from EuroVA 2021.

In this design study, we present IRVINE, a Visual Analytics (VA) system, which facilitates the analysis of acoustic data to detect and understand previously unknown errors in the manufacturing of electrical engines. In serial manufacturing processes, signatures from acoustic data provide valuable information on how the relationship between multiple produced engines serves to detect and understand previously unknown errors. To analyze such signatures, IRVINE leverages interactive clustering and data labeling techniques, allowing users to analyze clusters of engines with similar signatures, drill down to groups of engines, and select an engine of interest. Furthermore, IRVINE allows to assign labels to engines and clusters and annotate the cause of an error in the acoustic raw measurement of an engine. Since labels and annotations represent valuable knowledge, they are conserved in a knowledge database to be available for other stakeholders. We contribute a design study, where we developed IRVINE in four main iterations with engineers from a company in the automotive sector. To validate IRVINE, we conducted a field study with six domain experts. Our results suggest a high usability and usefulness of IRVINE as part of the improvement of a real-world manufacturing process. Specifically, with IRVINE domain experts were able to label and annotate produced electrical engines more than 30% faster.

Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data scientists need to assess classifiers' performances, evaluate their learning behavior over time, and compare different models. Typically, this analysis is based on single-number performance measures such as accuracy. A more detailed evaluation of classifiers is possible by inspecting class errors. The confusion matrix is an established way for visualizing these class errors, but it was not designed with temporal or comparative analysis in mind. More generally, established performance analysis systems do not allow a combined temporal and comparative analysis of class-level information. To address this issue, we propose ConfusionFlow, an interactive, comparative visualization tool that combines the benefits of class confusion matrices with the visualization of performance characteristics over time. ConfusionFlow is model-agnostic and can be used to compare performances for different model types, model architectures, and/or training and test datasets. We demonstrate the usefulness of ConfusionFlow in a case study on instance selection strategies in active learning. We further assess the scalability of ConfusionFlow and present a use case in the context of neural network pruning.

Graph neural networks (GNNs) are a class of powerful machine learning tools that model node relations for making predictions of nodes or links. GNN developers rely on quantitative metrics of the predictions to evaluate a GNN, but similar to many other neural networks, it is difficult for them to understand if the GNN truly learns characteristics of a graph as expected. We propose an approach to corresponding an input graph to its node embedding (aka latent space), a common component of GNNs that is later used for prediction. We abstract the data and tasks, and develop an interactive multi-view interface called CorGIE to instantiate the abstraction. As the key function in CorGIE, we propose the K-hop graph layout to show topological neighbors in hops and their clustering structure. To evaluate the functionality and usability of CorGIE, we present how to use CorGIE in two usage scenarios, and conduct a case study with five GNN experts. Availability: Open-source code at https://github.com/zipengliu/corgie-ui/, supplemental materials & video at https://osf.io/tr3sb/.

Event sequences are central to the analysis of data in domains that range from biology and health, to logfile analysis and people's everyday behavior. Many visualization tools have been created for such data, but people are error-prone when asked to judge the similarity of event sequences with basic presentation methods. This article describes an experiment that investigates whether local and global alignment techniques improve people's performance when judging sequence similarity. Participants were divided into three groups (basic versus local versus global alignment), and each participant judged the similarity of 180 sets of pseudo-randomly generated sequences. Each set comprised a target, a correct choice and a wrong choice. After training, the global alignment group was more accurate than the local alignment group (98 versus 93 percent correct), with the basic group getting 95 percent correct. Participants' response times were primarily affected by the number of event types, the similarity of sequences (measured by the Levenshtein distance) and the edit types (nine combinations of deletion, insertion and substitution). In summary, global alignment is superior and people's performance could be further improved by choosing alignment parameters that explicitly penalize sequence mismatches.

Class separation is an important concept in machine learning and visual analytics. We address the visual analysis of class separation measures for both high-dimensional data and its corresponding projections into 2D through dimensionality reduction (DR) methods. Although a plethora of separation measures have been proposed, it is difficult to compare class separation between multiple datasets with different characteristics, multiple separation measures, and multiple DR methods. We present ProSeCo, an interactive visualization approach to support comparison between up to 20 class separation measures and up to 4 DR methods, with respect to any of 7 dataset characteristics: dataset size, dataset dimensions, class counts, class size variability, class size skewness, outlieriness, and real-world vs. synthetically generated data. ProSeCo supports (1) comparing across measures, (2) comparing high-dimensional to dimensionally-reduced 2D data across measures, (3) comparing between different DR methods across measures, (4) partitioning with respect to a dataset characteristic, (5) comparing partitions for a selected characteristic across measures, and (6) inspecting individual datasets in detail. We demonstrate the utility of ProSeCo in two usage scenarios, using datasets [1] posted at https://osf.io/epcf9/.

Strategies for selecting the next data instance to label, in service of generating labeled data for machine learning, have been considered separately in the machine learning literature on active learning and in the visual analytics literature on human-centered approaches. We propose a unified design space for instance selection strategies to support detailed and fine-grained analysis covering both of these perspectives. We identify a concise set of 15 properties, namely measureable characteristics of datasets or of machine learning models applied to them, that cover most of the strategies in these literatures. To quantify these properties, we introduce Property Measures (PM) as fine-grained building blocks that can be used to formalize instance selection strategies. In addition, we present a taxonomy of PMs to support the description, evaluation, and generation of PMs across four dimensions: machine learning (ML) Model Output , Instance Relations , Measure Functionality , and Measure Valence . We also create computational infrastructure to support qualitative visual data analysis: a visual analytics explainer for PMs built around an implementation of PMs using cascades of eight atomic functions. It supports eight analysis tasks, covering the analysis of datasets and ML models using visual comparison within and between PMs and groups of PMs, and over time during the interactive labeling process. We iteratively refined the PM taxonomy, the explainer, and the task abstraction in parallel with each other during a two-year formative process, and show evidence of their utility through a summative evaluation with the same infrastructure. This research builds a formal baseline for the better understanding of the commonalities and differences of instance selection strategies, which can serve as the stepping stone for the synthesis of novel strategies in future work.

Linguistic insight in the form of high-level relationships and rules in text builds the basis of our understanding of language. However, the data-driven generation of such structures often lacks labeled resources that can be used as training data for supervised machine learning. The creation of such ground-truth data is a time-consuming process that often requires domain expertise to resolve text ambiguities and characterize linguistic phenomena. Furthermore, the creation and refinement of machine learning models is often challenging for linguists as the models are often complex, in-transparent, and difficult to understand. To tackle these challenges, we present a visual analytics technique for interactive data labeling that applies concepts from gamification and explainable Artificial Intelligence (XAI) to support complex classification tasks. The visual-interactive labeling interface promotes the creation of effective training data. Visual explanations of learned rules unveil the decisions of the machine learning model and support iterative and interactive optimization. The gamification-inspired design guides the user through the labeling process and provides feedback on the model performance. As an instance of the proposed technique, we present QuestionComb , a workspace tailored to the task of question classification (i.e., in information-seeking vs. non-information-seeking questions). Our evaluation studies confirm that gamification concepts are beneficial to engage users through continuous feedback, offering an effective visual analytics technique when combined with active learning and XAI.

Methods from supervised machine learning allow the classification of new data automatically and are tremendously helpful for data analysis. The quality of supervised maching learning depends not only on the type of algorithm used, but also on the quality of the labelled dataset used to train the classifier. Labelling instances in a training dataset is often done manually relying on selections and annotations by expert analysts, and is often a tedious and time-consuming process. Active learning algorithms can automatically determine a subset of data instances for which labels would provide useful input to the learning process. Interactive visual labelling techniques are a promising alternative, providing effective visual overviews from which an analyst can simultaneously explore data records and select items to a label. By putting the analyst in the loop, higher accuracy can be achieved in the resulting classifier. While initial results of interactive visual labelling techniques are promising in the sense that user labelling can improve supervised learning, many aspects of these techniques are still largely unexplored. This paper presents a study conducted using the mVis tool to compare three interactive visualisations, similarity map, scatterplot matrix (SPLOM), and parallel coordinates, with each other and with active learning for the purpose of labelling a multivariate dataset. The results show that all three interactive visual labelling techniques surpass active learning algorithms in terms of classifier accuracy, and that users subjectively prefer the similarity map over SPLOM and parallel coordinates for labelling. Users also employ different labelling strategies depending on the visualisation used.

Abstract Pre‐processing is a prerequisite to conduct effective and efficient downstream data analysis. Pre‐processing pipelines often require multiple routines to address data quality challenges and to bring the data into a usable form. For both the construction and the refinement of pre‐processing pipelines, human‐in‐the‐loop approaches are highly beneficial. This particularly applies to multivariate time series, a complex data type with multiple values developing over time. Due to the high specificity of this domain, it has not been subject to in‐depth research in visual analytics. We present a visual‐interactive approach for preprocessing multivariate time series data with the following aspects. Our approach supports analysts to carry out six core analysis tasks related to pre‐processing of multivariate time series. To support these tasks, we identify requirements to baseline toolkits that may help practitioners in their choice. We characterize the space of visualization designs for uncertainty‐aware pre‐processing and justify our decisions. Two usage scenarios demonstrate applicability of our approach, design choices, and uncertainty visualizations for the six analysis tasks. This work is one step towards strengthening the visual analytics support for data pre‐processing in general and for uncertainty‐aware pre‐processing of multivariate time series in particular.

In this design study, we present a visualization technique that segments patients' histories instead of treating them as raw event sequences, aggregates the segments using criteria such as the whole history or treatment combinations, and then visualizes the aggregated segments as static dashboards that are arranged in a dashboard network to show longitudinal changes. The static dashboards were developed in nine iterations, to show 15 important attributes from the patients' histories. The final design was evaluated with five non-experts, five visualization experts and four medical experts, who successfully used it to gain an overview of a 2,000 patient dataset, and to make observations about longitudinal changes and differences between two cohorts. The research represents a step-change in the detail of large-scale data that may be successfully visualized using dashboards, and provides guidance about how the approach may be generalized.

Supervised machine learning techniques require labelled multivariate training datasets. Many approaches address the issue of unlabelled datasets by tightly coupling machine learning algorithms with interactive visualisations. Using appropriate techniques, analysts can play an active role in a highly interactive and iterative machine learning process to label the dataset and create meaningful partitions. While this principle has been implemented either for unsupervised, semi-supervised, or supervised machine learning tasks, the combination of all three methodologies remains challenging. In this paper, a visual analytics approach is presented, combining a variety of machine learning capabilities with four linked visualisation views, all integrated within the mVis (multivariate Visualiser) system. The available palette of techniques allows an analyst to perform exploratory data analysis on a multivariate dataset and divide it into meaningful labelled partitions, from which a classifier can be built. In the workflow, the analyst can label interesting patterns or outliers in a semi-supervised process supported by active learning. Once a dataset has been interactively labelled, the analyst can continue the workflow with supervised machine learning to assess to what degree the subsequent classifier has effectively learned the concepts expressed in the labelled training dataset. Using a novel technique called automatic dimension selection, interactions the analyst had with dimensions of the multivariate dataset are used to steer the machine learning algorithms. A real-world football dataset is used to show the utility of mVis for a series of analysis and labelling tasks, from initial labelling through iterations of data exploration, clustering, classification, and active learning to refine the named partitions, to finally producing a high-quality labelled training dataset suitable for training a classifier. The tool empowers the analyst with interactive visualisations including scatterplots, parallel coordinates, similarity maps for records, and a new similarity map for partitions.

Abstract The labeling of data sets is a time‐consuming task, which is, however, an important prerequisite for machine learning and visual analytics. Visual‐interactive labeling (VIAL) provides users an active role in the process of labeling, with the goal to combine the potentials of humans and machines to make labeling more efficient. Recent experiments showed that users apply different strategies when selecting instances for labeling with visual‐interactive interfaces. In this paper, we contribute a systematic quantitative analysis of such user strategies. We identify computational building blocks of user strategies, formalize them, and investigate their potentials for different machine learning tasks in systematic experiments. The core insights of our experiments are as follows. First, we identified that particular user strategies can be used to considerably mitigate the bootstrap (cold start) problem in early labeling phases. Second, we observed that they have the potential to outperform existing active learning strategies in later phases. Third, we analyzed the identified core building blocks, which can serve as the basis for novel selection strategies. Overall, we observed that data‐based user strategies (clusters, dense areas) work considerably well in early phases, while model‐based user strategies (e.g., class separation) perform better during later phases. The insights gained from this work can be applied to develop novel active learning approaches as well as to better guide users in visual interactive labeling.

Labeling data instances is an important task in machine learning and visual analytics. Both fields provide a broad set of labeling strategies, whereby machine learning (and in particular active learning) follows a rather model-centered approach and visual analytics employs rather user-centered approaches (visual-interactive labeling). Both approaches have individual strengths and weaknesses. In this work, we conduct an experiment with three parts to assess and compare the performance of these different labeling strategies. In our study, we (1) identify different visual labeling strategies for user-centered labeling, (2) investigate strengths and weaknesses of labeling strategies for different labeling tasks and task complexities, and (3) shed light on the effect of using different visual encodings to guide the visual-interactive labeling process. We further compare labeling of single versus multiple instances at a time, and quantify the impact on efficiency. We systematically compare the performance of visual interactive labeling with that of active learning. Our main findings are that visual-interactive labeling can outperform active learning, given the condition that dimension reduction separates well the class distributions. Moreover, using dimension reduction in combination with additional visual encodings that expose the internal state of the learning model turns out to improve the performance of visual-interactive labeling.

Clustering is a core building block for data analysis, aiming to extract otherwise hidden structures and relations from raw datasets, such as particular groups that can be effectively related, compared, and interpreted. A plethora of visual-interactive cluster analysis techniques has been proposed to date, however, arriving at useful clusterings often requires several rounds of user interactions to fine-tune the data preprocessing and algorithms. We present a multi-stage Visual Analytics (VA) approach for iterative cluster refinement together with an implementation (SOMFlow) that uses Self-Organizing Maps (SOM) to analyze time series data. It supports exploration by offering the analyst a visual platform to analyze intermediate results, adapt the underlying computations, iteratively partition the data, and to reflect previous analytical activities. The history of previous decisions is explicitly visualized within a flow graph, allowing to compare earlier cluster refinements and to explore relations. We further leverage quality and interestingness measures to guide the analyst in the discovery of useful patterns, relations, and data partitions. We conducted two pair analytics experiments together with a subject matter expert in speech intonation research to demonstrate that the approach is effective for interactive data analysis, supporting enhanced understanding of clustering results as well as the interactive process itself.

The process of political decision making is often complex and tedious. The policy process consists of multiple steps, most of them are highly iterative. In addition, different stakeholder groups are involved in political decision making and contribute to the process. A series of textual documents accompanies the process. Examples are official documents, discussions, scientific reports, external reviews, newspaper articles, or economic white papers. Experts from the political domain report that this plethora of textual documents often exceeds their ability to keep track of the entire policy process. We present PolicyLine, a visualization system that supports different stakeholder groups in overview-and-detail tasks for large sets of textual documents in the political decision making process. In a longitudinal design study conducted together with domain experts in political decision making, we identified missing analytical functionality on the basis of a problem and domain characterization. In an iterative design phase, we created PolicyLine in close collaboration with the domain experts. Finally, we present the results of three evaluation rounds, and reflect on our collaborative visualization system.

Abstract The analysis of research data plays a key role in data‐driven areas of science. Varieties of mixed research data sets exist and scientists aim to derive or validate hypotheses to find undiscovered knowledge. Many analysis techniques identify relations of an entire dataset only. This may level the characteristic behavior of different subgroups in the data. Like automatic subspace clustering, we aim at identifying interesting subgroups and attribute sets. We present a visual‐interactive system that supports scientists to explore interesting relations between aggregated bins of multivariate attributes in mixed data sets. The abstraction of data to bins enables the application of statistical dependency tests as the measure of interestingness. An overview matrix view shows all attributes, ranked with respect to the interestingness of bins. Complementary, a node‐link view reveals multivariate bin relations by positioning dependent bins close to each other. The system supports information drill‐down based on both expert knowledge and algorithmic support. Finally, visual‐interactive subset clustering assigns multivariate bin relations to groups. A list‐based cluster result representation enables the scientist to communicate multivariate findings at a glance. We demonstrate the applicability of the system with two case studies from the earth observation domain and the prostate cancer research domain. In both cases, the system enabled us to identify the most interesting multivariate bin relations, to validate already published results, and, moreover, to discover unexpected relations.

Abstract We present a system to analyze time‐series data in sensor networks. Our approach supports exploratory tasks for the comparison of univariate, geo‐referenced sensor data, in particular for anomaly detection. We split the recordings into fixed‐length patterns and show them in order to compare them over time and space using two linked views. Apart from geo‐based comparison across sensors we also support different temporal patterns to discover seasonal effects, anomalies and periodicities. The methods we use are best practices in the information visualization domain. They cover the daily, the weekly and seasonal and patterns of the data. Daily patterns can be analyzed in a clustering‐based view, weekly patterns in a calendar‐based view and seasonal patters in a projection‐based view. The connectivity of the sensors can be analyzed through a dedicated topological network view. We assist the domain expert with interaction techniques to make the results understandable. As a result, the user can identify and analyze erroneous and suspicious measurements in the network. A case study with a domain expert verified the usefulness of our approach.

We present MotionExplorer, an exploratory search and analysis system for sequences of human motion in large motion capture data collections. This special type of multivariate time series data is relevant in many research fields including medicine, sports and animation. Key tasks in working with motion data include analysis of motion states and transitions, and synthesis of motion vectors by interpolation and combination. In the practice of research and application of human motion data, challenges exist in providing visual summaries and drill-down functionality for handling large motion data collections. We find that this domain can benefit from appropriate visual retrieval and analysis support to handle these tasks in presence of large motion data. To address this need, we developed MotionExplorer together with domain experts as an exploratory search system based on interactive aggregation and visualization of motion states as a basis for data navigation, exploration, and search. Based on an overview-first type visualization, users are able to search for interesting sub-sequences of motion based on a query-by-example metaphor, and explore search results by details on demand. We developed MotionExplorer in close collaboration with the targeted users who are researchers working on human motion synthesis and analysis, including a summative field study. Additionally, we conducted a laboratory design study to substantially improve MotionExplorer towards an intuitive, usable and robust design. MotionExplorer enables the search in human motion capture data with only a few mouse clicks. The researchers unanimously confirm that the system can efficiently support their work.

Abstract Exploration and selection of data descriptors representing objects using a set of features are important components in many data analysis tasks. Usually, for a given dataset, an optimal data description does not exist, as the suitable data representation is strongly use case dependent. Many solutions for selecting a suitable data description have been proposed. In most instances, they require data labels and often are black box approaches. Non‐expert users have difficulties to comprehend the coherency of input, parameters, and output of these algorithms. Alternative approaches, interactive systems for visual feature selection, overburden the user with an overwhelming set of options and data views. Therefore, it is essential to offer the users a guidance in this analytical process. In this paper, we present a novel system for data description selection, which facilitates the user's access to the data analysis process. As finding of suitable data description consists of several steps, we support the user with guidance. Our system combines automatic data analysis with interactive visualizations. By this, the system provides a recommendation for suitable data descriptor selections. It supports the comparison of data descriptors with differing dimensionality for unlabeled data. We propose specialized scores and interactive views for descriptor comparison. The visualization techniques are scatterplot‐based and grid‐based. For the latter case, we apply Self‐Organizing Maps as adaptive grids which are well suited for large multi‐dimensional data sets. As an example, we demonstrate the usability of our system on a real‐world biochemical application.