We characterize 16 challenges faced by those investigating and developing remote and synchronous collaborative experiences around visualization. Our work reflects the perspectives and prior research efforts of an international group of 29 experts from across human-computer interaction and visualization sub-communities. The challenges are anchored around five collaborative activities that exhibit a centrality of visualization and multimodal communication. These activities include exploratory data analysis, creative ideation, visualization-rich presentations, joint decision making grounded in data, and real-time data monitoring. The challenges also reflect the changing dynamics of these activities in the face of recent advances in extended reality (XR) and artificial intelligence (AI). As an organizing scheme for future research at the intersection of visualization and computer-supported cooperative work, we align the challenges with a sequence of four sets of research and development activities: technological choices, social factors, AI assistance, and evaluation.

In this Research-to-practice paper, we share our experience with implementing Specifications Grading in two separate offerings of a Data Visualization course. The course is taught in a Computer Science department every semester. Based on the analysis of the student assignments, final project, and the final grade received by the students, the students in the offerings with Specifications Grading performed better or as well as without it. We found that Specifications Grading led to students taking control of their own learning and restored rigor to the course. Student feedback for Specifications Grading was mostly positive, and multiple students encouraged the instructor to keep using it in future offerings of the course.

Community-Engaged Learning (CEL) provides students with an opportunity to engage with the community and develop academic skills as they reflect on social justice issues. We present the framework of a new CEL course that enables students to reflect on the inequities in tech through the process of teaching coding to youth from difficult circumstances such as those in the foster care and juvenile justice systems.

This Research to Practice Full Paper presents the experiences and lessons learned from five programs that provide financial awards and a holistic student support structure to low-income, academically talented students in Science, Technology, Engineering, and Mathematics (STEM). This report synthesizes the experiences of a diverse set of institutions, both public and private, that vary in size and geographic location. We have experience supporting students from a range of disciplines with an emphasis on students studying Computer Science. The goals of this work are to (1) outline the decisions that must be considered when designing a financial award program; (2) describe the interventions we have implemented and underline the institutional contexts that have led to their success; (3) describe the unique challenges posed by the COVID pandemic; and (4) highlight key elements necessary for successful program implementation. We specifically discuss the challenges we have encountered when implementing existing best practices. We report observations and results, some of which buttress those reported in the literature. Our work is intended to serve as a guide for educators who wish to implement programs to support students from financially disadvantaged and/or historically marginalized groups. By sharing our experiences and pain points, we hope to make it easier for them to design and implement effective programs adapted to their institutional needs and contexts.

This Birds-of-a-Feather session is for anyone interested in the NSF Scholarships in STEM (S-STEM) program, including current and former Principal Investigators (PIs) and those planning to apply. The S-STEM program funds scholarships and activities to support low-income, academically talented students in STEM. Any institution of higher education may apply, and the program supports a variety of projects. Designing and implementing a successful S-STEM project is challenging. The goal of this session is to catalyze a community of practice for S-STEM PIs. It will provide an opportunity to discuss lessons learned and best practices for proposal writing, project implementation, and providing student support. Specific topics to be discussed include the following: (1) Understanding the solicitation requirements and common proposal mistakes; (2) Scholar recruitment and data-driven approaches for selection; (3) Cohort building including activities for students from different majors or class years and integration of new students into existing cohorts; and (4) Remediation strategies including proactive interventions and peer support. Session leaders will introduce each topic; participants will then join a breakout group discussion of one topic. Lastly, participants will be invited to join a Slack workspace dedicated to S-STEM best practices and lessons.

As worldwide capability to collect, store, and manage information continues to grow, the resulting datasets become increasingly difficult to understand and extract insights from. Interactive data visualizations offers a promising avenue to efficiently navigate and gain insights from highly complex datasets, but the velocity of modern data streams often means that precomputed representations or summarizations of the data will quickly become obsolete. Our system, Agami, provides live-updating, interactive visualizations over streaming data. We leverage in-memory data sketches to summarize and aggregate information to be visualized, and also allow users to query future feature values by leveraging online machine learning models. Our approach facilitates low-latency, iterative exploration of data streams and can scale out incrementally to handle increasing stream velocities and query loads. We provide a thorough evaluation of our data structures and system performance using a real-world meteorological dataset.

In this Research-to-Practice Full Paper, we present the results of adopting Team-Based Learning (TBL) for teaching a Sophomore-level Systems Programming course. The goal of TBL is to "provide opportunities for students to apply their knowledge in the classroom to solve problems rather than just covering content." Based on the performance of the students in the course taught with TBL, we found that TBL had a statistically significant impact on student performance in 2 of the 5 programming assignments. Additionally, the end-of-semester student survey indicated that 88% of the students said that the team-based learning activities helped them understand the material better. Students mentioned that they felt like they belonged in Computer Science (fostering a sense of community in large classrooms) and frequently studied with some of their team members for the course assignments. Compared to a previous offering of the course that was purely lecture-based, the class as a whole received higher final grades and performed better on all of the programming assignments.

In this Research to Practice paper, we present a sustainable model for teaching training in Computer Science. To address issues related to self-efficacy and teacher preparation, we started a formal program (IDoCode) that not only provides teacher training through the academic year, but also provides teachers the opportunity to obtain a Masters in STEM Education degree or a Graduate Certificate in Computer Science Teacher Endorsement.Through our program, we have shown that teachers feel more confident in their ability to teach computer science courses such as Exploring CS, AP CS Principles, and the Java-based AP CS A, as well as leading the students in a capstone project. In this paper, we present a sustainable approach to make a cultural change in the landscape of Computer Science education in the state of Idaho. We discuss various factors including working with the State Board of Education, local software companies, the university, and other invested partners to help CS courses in high school count towards graduation. We have also been active with respect to community engagement by organizing an annual meeting with counselors and principals to encourage women and minorities to take computer science courses and conducting summer professional development workshops for new teachers.

The scatterplot matrix (SPLOM) is a commonly used technique for visualizing multiclass multivariate data. However, multiclass SPLOMs have issues with overdraw (overlapping points), and most existing techniques for alleviating overdraw focus on individual scatterplots with a single class. This paper explores whether animation using flickering points is an effective way to alleviate overdraw in these multiclass SPLOMs. In a user study with 69 participants, we found that users not only performed better at identifying dense regions using animated SPLOMs, but also found them easier to interpret and preferred them to static SPLOMs. These results open up new directions for future work on alleviating overdraw for multiclass SPLOMs, and provide insights for applying animation to alleviate overdraw in other settings.

As the number of teachers teaching computer science grows, it is increasingly important to be mindful of the training they receive with respect to broadening participation in computer science. Through our program, we have trained over 50 teachers in the greater Boise Metropolitan region, who have in turn taught over 1400 students computing concepts through courses such as Exploring CS, AP CS Principles, and AP CS A. These courses have an excellent curriculum that contains a mix of computational thinking concepts such as a focus on creativity, abstraction, coding, as well as increasing awareness about the cyber footprint of the students with respect to security and privacy. While the curriculum is excellent, we need to be more mindful about incorporating pedagogical strategies that promote inclusive teaching especially for women and minorities who are traditionally underrepresented in computer science.To address the challenges associated with teaching a truly inclusive course, we developed a new course titled “Inclusive Strategies for Computer Science Education” that draws attention to the strategies that have been studied over the years in STEM and CS education literature. We present the contents of the course along with a post-hoc qualitative survey on the applicability and practicality of the material discussed in the course.

BACKGROUND: Cluster heatmaps are commonly used in biology and related fields to reveal hierarchical clusters in data matrices. This visualization technique has high data density and reveal clusters better than unordered heatmaps alone. However, cluster heatmaps have known issues making them both time consuming to use and prone to error. We hypothesize that visualization techniques without the rigid grid constraint of cluster heatmaps will perform better at clustering-related tasks. RESULTS: We developed an approach to "unbox" the heatmap values and embed them directly in the hierarchical clustering results, allowing us to use standard hierarchical visualization techniques as alternatives to cluster heatmaps. We then tested our hypothesis by conducting a survey of 45 practitioners to determine how cluster heatmaps are used, prototyping alternatives to cluster heatmaps using pair analytics with a computational biologist, and evaluating those alternatives with hour-long interviews of 5 practitioners and an Amazon Mechanical Turk user study with approximately 200 participants. We found statistically significant performance differences for most clustering-related tasks, and in the number of perceived visual clusters. Visit git.io/vw0t3 for our results. CONCLUSIONS: The optimal technique varied by task. However, gapmaps were preferred by the interviewed practitioners and outperformed or performed as well as cluster heatmaps for clustering-related tasks. Gapmaps are similar to cluster heatmaps, but relax the heatmap grid constraints by introducing gaps between rows and/or columns that are not closely clustered. Based on these results, we recommend users adopt gapmaps as an alternative to cluster heatmaps.

With the increasing availability and popularity of visualization tools, it is easier than ever to create visual representations of data. The available tools and libraries work for a range of users from non-programmers to those with significant programming experience. A major challenge, however, is that a majority of users frequently stick with the default settings when using software.

We present an application of the generalized plot matrix (GPLOM) technique for visualizing a multidimensional dataset containing 202 YouTube videos from the "Statistics and Social Network of YouTube Videos" project. The web-based visualization provides users an easy to use interface to explore the data. Using our tool, we found several interesting relationships between the number of views, comments, ratings and stars. Visit git.io/vwDqw for a live demo.

CoVE is a novel, scalable, interactive tool that can be used to visualize and manage large colonies of laboratory animals. Effective management of large colonies of animals with multiple individual attributes and complicated breeding schemes represents a significant data management challenge in the biological sciences. Currently available software either provides databases for record keeping or generates basic pedigrees but not both. Thus, there is a pressing need for an integrated colony management system that provides a repository for the data and addresses the visualization challenge presented by complex genealogical data. We present CoVE, a colony visualization tool that provides an overview of the entire colony, clusters individuals based on Gender, Litter or Genotype, and provides an individual view of any animal for detailed examination. We demonstrate that CoVE provides an efficient way to manage, generate and view complex pedigree of real world genealogical data from animal colonies, annotated with details of individual attributes. It enables interactive tracing of lineages and identification of censored subjects in tumor studies.

Automakers are increasingly providing connectivity enhancements for vehicles to download navigational data, as well as to upload sensor information to the cloud. Generally, while more data may be better, for the driver on-the-go, information needs to be displayed in a manner that can be comprehended rather quickly. One of the major problems with visualizing route maps is that the amount of information visualized is always the same regardless of the fact that an individual may be more familiar with the region or whether an individual is driving at varying speeds. Research has shown that complex visualizations with visual clutter can cause cognitive overload that adversely affects the performance of a user. Additionally, the attention and interaction abilities of a driver are significantly compromised in a vehicular environment. We propose DriveSense, a context-sensitive visualization system that automatically varies the GPS updates and the corresponding visualization being displayed to the user based on the speed of the vehicle as well as the familiarity of the region that the user is driving in. Based on a user evaluation, we found that subjects preferred using the automatic visualizations of route maps generated by DriveSense than the visual representations shown by a standard GPS. We also computed visual clutter for our visualizations at varying speeds and found that the clutter was significantly less for the routes displayed by DriveSense for faster speeds as compared to slower speeds.

Certain biological factors such as genetics, physical fitness, and lifestyle have been shown to influence an individual's risk of acquiring disease. But are there are other socioeconomic factors that influence disease incidence as well? In this paper, we introduce a visualization tool called Disease Trends that explores the associations and possible correlations between specific economic (personal income per capita), educational (percentage of adult population with a four year college degree), and environmental (air pollution level) factors with diabetes prevalence and cancer incidence rates across counties throughout the United States. It is structured as an interactive geographical visualization that displays disease incidence data as an interactive choropleth map and connects it with coordinated views of the socioeconomic variables for each county as the user scrolls over it. Additionally, the ability to compare and contrast counties as well as to interactively specify a region for comparison allows further examination of the data. This results in an informative overview of disease incidence trends that allows users to spot areas of interest and potentially pursue these areas further with more scientific research.

The devastating power of hurricanes was evident during the 2005 hurricane season, the most active season on record. This has prompted increased efforts by researchers to understand the physical processes that underlie the genesis, intensification, and tracks of hurricanes. This research aims at facilitating an improved understanding into the structure of hurricanes with the aid of visualization techniques. Our approach was developed by a mixed team of visualization and domain experts. To better understand these systems, and to explore their representation in NWP models, we use a variety of illustration-inspired techniques to visualize their structure and time evolution. Illustration-inspired techniques aid in the identification of the amount of vertical wind shear in a hurricane, which can help meteorologists predict dissipation. Illustration-style visualization, in combination with standard visualization techniques, helped explore the vortex rollup phenomena and the mesovortices contained within. We evaluated the effectiveness of our visualization with the help of six hurricane experts. The expert evaluation showed that the illustration-inspired techniques were preferred over existing tools. Visualization of the evolution of structural features is a prelude to a deeper visual analysis of the underlying dynamics.

Abstract Illustration‐inspired techniques have provided alternative ways to visualize time‐varying data. Techniques such as speedlines, flow ribbons, strobe silhouettes and opacity‐based techniques provide temporal context to the current timestep being visualized. We evaluated the effectiveness of these illustrative techniques by conducting a user study. We compared the ability of subjects to visually track features using snapshots, snapshots augmented by illustration techniques, animations, and animations augmented by illustration techniques. User accuracy, time required to perform a task, and user confidence were used as measures to evaluate the techniques. The results indicate that the use of illustration‐inspired techniques provides a significant improvement in user accuracy and the time required to complete the task. Subjects performed significantly better on each metric when using augmented animations as compared to augmented snapshots.

The effective visualization of vascular structures is critical for diagnosis, surgical planning as well as treatment evaluation. In recent work, we have developed an algorithm for vessel detection that examines the intensity profile around each voxel in an angiographic image and determines the likelihood that any given voxel belongs to a vessel; we term this the "vesselness coefficient" of the voxel. Our results show that our algorithm works particularly well for visualizing branch points in vessels. Compared to standard Hessian based techniques, which are fine-tuned to identify long cylindrical structures, our technique identifies branches and connections with other vessels. Using our computed vesselness coefficient, we explore a set of techniques for visualizing vasculature. Visualizing vessels is particularly challenging because not only is their position in space important for clinicians but it is also important to be able to resolve their spatial relationship. We applied visualization techniques that provide shape cues as well as depth cues to allow the viewer to differentiate between vessels that are closer from those that are farther. We use our computed vesselness coefficient to effectively visualize vasculature in both clinical neurovascular x-ray computed tomography based angiography images, as well as images from three different animal studies. We conducted a formal user evaluation of our visualization techniques with the help of radiologists, surgeons, and other expert users. Results indicate that experts preferred distance color blending and tone shading for conveying depth over standard visualization techniques.

Neurosurgical planning and image guided neurosurgery require the visualization of multimodal data obtained from various functional and structural image modalities, such as Magnetic Resonance Imaging (MRI), Computed Tomography (CT), functional MRI, Single photon emission computed tomography (SPECT) and so on. In the case of epilepsy neurosurgery for example, these images are used to identify brain regions to guide intracranial electrode implantation and resection. Generally, such data is visualized using 2D slices and in some cases using a 3D volume rendering along with the functional imaging results. Visualizing the activation region effectively by still preserving sufficient surrounding brain regions for context is exceedingly important to neurologists and surgeons. We present novel interaction techniques for visualization of multimodal data to facilitate improved exploration and planning for neurosurgery. We extended the line widget from VTK to allow surgeons to control the shape of the region of the brain that they can visually crop away during exploration and surgery. We allow simple spherical, cubical, ellipsoidal and cylindrical (probe aligned cuts) for exploration purposes. In addition we integrate the cropping tool with the image-guided navigation system used for epilepsy neurosurgery. We are currently investigating the use of these new tools in surgical planning and based on further feedback from our neurosurgeons we will integrate them into the setup used for image-guided neurosurgery.

Analyzing, visualizing, and illustrating changes within time-varying volumetric data is challenging due to the dynamic changes occurring between timesteps. The changes and variations in computational fluid dynamic volumes and atmospheric 3D datasets do not follow any particular transformation. Features within the data move at different speeds and directions making the tracking and visualization of these features a difficult task. We introduce a texture-based feature tracking technique to overcome some of the current limitations found in the illustration and visualization of dynamic changes within time-varying volumetric data. Our texture-based technique tracks various features individually and then uses the tracked objects to better visualize structural changes. We show the effectiveness of our texture-based tracking technique with both synthetic and real world time-varying data. Furthermore, we highlight the specific visualization, annotation, registration, and feature isolation benefits of our technique. For instance, we show how our texture-based tracking can lead to insightful visualizations of time-varying data. Such visualizations, more than traditional visualization techniques, can assist domain scientists to explore and understand dynamic changes.

Traditionally, time-varying data has been visualized using snapshots of the individual time steps or an animation of the snapshots shown in a sequential manner. For larger datasets with many time-varying features, animation can be limited in its use, as an observer can only track a limited number of features over the last few frames. Visually inspecting each snapshot is not practical either for a large number of time-steps. We propose new techniques inspired from the illustration literature to convey change over time more effectively in a time-varying dataset. Speedlines are used extensively by cartoonists to convey motion, speed, or change over different panels. Flow ribbons are another technique used by cartoonists to depict motion in a single frame. Strobe silhouettes are used to depict previous positions of an object to convey the previous positions of the object to the user. These illustration-inspired techniques can be used in conjunction with animation to convey change over time.