Stress in a graph drawing has been a popular layout principle for more than two decades. Low stress drawings exhibit the property that the geometric distances between all pairs of nodes correlate with the shortest paths between them. The assumption has always been that low stress drawings are "nicer" and better support human perception and comprehension than high stress drawings. In this paper, we put these assumptions to the test. We use a normalised scale-independent and rotation-independent metric for stress; this is necessary to ensure strict controls on our experimental stimuli. We report on three experiments, exploring human perception of stress, preference for stress, and the effect of stress on a graph performance task. We conclude that people can see stress in a graph drawing, that they prefer low stress drawings, and that their performance in a shortest path task improves as stress decreases - thus empirically confirming long-standing assumptions.

Graphs are drawn for various purposes, and drawings are meant to display various features of a graph (such as planarity, Hamiltonicity). Still, there is a long history in measuring the quality of a graph drawing. Most of the metrics that have been implemented and used in large studies assume that graphs are drawn straight-line. Most of the studies use randomly generated graphs or one of very few existing benchmark sets that consist of graphs with a specific technical background (e.g., telecommunication networks). In this paper, we extend ten commonly used metrics to node-link diagrams where edges can be curves or polygonal chains. We implement these measures and use them to evaluate a new collection of graph drawings that we have extracted from 27 proceedings of the Graph Drawing conference using an automated pipeline. We compare the "metrics landscape" of our new benchmark set, the GD-collection-v1, which seems to mostly contain manually drawn graphs, to the metric landscape of a benchmark set with randomly generated graphs and computer-generated straight-line drawings that has been used in a recent study [Mooney et al.; PacificVis 2024]. Comparing the GD-collection-v1 with the Mooney at al. dataset reveals a distinct metrics landscape: GD drawings come from much smaller graphs (median vertex number 11 vs. 48) and therefore attain higher medians on most readability metrics. For example, Neighbourhood Preservation (0.5 vs. 0.239) is markedly higher in the GD-collection-v1. We also find that a large proportion of extracted drawings contain curved and/or polygonal edges (57%), motivating the extended metric definitions.

Abstract We investigate the usability of Euclidean, spherical and hyperbolic geometries for network visualization. Several techniques have been proposed for both spherical and hyperbolic network visualization tools, based on the fact that some networks admit lower embedding error (distortion) in such non‐Euclidean geometries. However, it is not yet known whether a lower embedding error translates to human subject benefits, e.g., better task accuracy or lower task completion time. We design, implement, conduct, and analyze a human subjects study to compare Euclidean, spherical and hyperbolic network visualizations using tasks that span the network task taxonomy. While in some cases accuracy and response times are negatively impacted when using non‐Euclidean visualizations, the evaluation shows that differences in accuracy for hyperbolic and spherical visualizations are not statistically significant when compared to Euclidean visualizations. Additionally, differences in response times for spherical visualizations are not statistically significant compared to Euclidean visualizations.

Most of the common graph layout principles (a.k.a. "aesthetics") on which many graph drawing algorithms are based are easy to define and to perceive. For example, the number of pairs of edges that cross each other, how symmetric a drawing looks, the aspect ratio of the bounding box, or the angular resolution at the nodes. The extent to which a graph drawing conforms to these principles can be determined by looking at how it is drawn - that is, by looking at the marks on the page - without consideration for the underlying structure of the graph. A key layout principle is that of optimising "stress", the basis for many algorithms such as the popular Kamada & Kawai algorithm and several force-directed algorithms. The stress of a graph drawing is, loosely speaking, the extent to which the geometric distance between each pair of nodes is proportional to the shortest path between them - over the whole graph drawing. The definition of stress therefore relies on the underlying structure of the graph (the "paths") in a way that other layout principles do not, making stress difficult to describe to novices unfamiliar with graph drawing principles, and, we believe, difficult to perceive. We conducted an experiment to see whether people (novices as well as experts) can see stress in graph drawings, and found that it is possible to train novices to "see" stress - even if their perception strategies are not based on the definitional concepts.

Any graph drawing can be characterised by a range of computational aesthetic metrics. For example, a given drawing might be described as having eight crossings, a mean angular resolution of 0.34, and an edge orthogonality value of 0.72. However, without knowing the distribution of these metrics it is hard to compare the quality of drawings of different graphs, nor know whether a given drawing is typical or an outlier within the space of all possible drawings. This paper explores the range and distribution of ten normalised graph drawing layout metrics, based on graphs created by six graph generation algorithms and drawings created by six popular layout algorithms. We include the "Rome" and "North" graph repositories in our analysis. Our exploration of the multi-dimensional aesthetics space allows for comparisons between the graph drawing algorithms, highlighting those that cover larger or smaller volumes of the aesthetics space. We calculate the correlation coefficients between the metrics, indicating those that may conflict with each other (negatively correlated), and those that may be redundant (positively correlated). Our results will be useful as the basis for simulated annealing or gradient descent layout algorithms, for identifying the best layout algorithms for producing a specified combination and range of aesthetics, and for informing experimental controls in human empirical studies.

Social networks are the fabric of society and the subject of frequent visual analysis. Closed triads represent triangular relationships between three people in a social network and are significant for understanding inherent interconnections and influence within the network. The most common methods for representing social networks (node-link diagrams and adjacency matrices) are not optimal for understanding triangles. We propose extending the adjacency matrix form to 3D for better visualization of network triads. We design a 3D matrix reordering technique and implement an immersive interactive system to assist in visualizing and analyzing closed triads in social networks. The evaluations demonstrate that our method provides substantial added value over node-link diagrams in improving the efficiency and accuracy of manipulating and understanding the social network triads.

This paper presents an analysis of different interaction techniques used in interactive data visualisations to support end-users in visual analytics tasks. Our selection of interaction techniques is based on prior work and consists of the interaction techniques SELECT, EXPLORE, RECONFIGURE, ENCODE, FILTER, ABSTRACT/ELABORATE, and CONNECT. Through a within-subject study, we assessed participants’ abilities to utilise these techniques when faced with three distinct types of data-driven tasks; lookup, comparison, and Relation-seeking. Our research investigates the impact of these interaction techniques on the correctness, confidence, perceived difficulty, and cognitive load of N = 80 self-identified data scientists and N = 80 non-experts. We find that interaction technique significantly impacts answer correctness and participant confidence. Participants performed best across those interaction techniques that allow for information that is deemed least relevant to be concealed, which is reflected in lower intrinsic and extraneous cognitive load. Interestingly, participants’ expertise affected their confidence but not their accuracy. Our results provide insights useful for a more targeted and informed design and usage of interactive data visualisations.

COVID-19 regulations presented clinicians with a new set of challenges that affected their ability to efficiently provide patient care and, as a result, telemedicine was rapidly adopted to deliver care remotely. However, these telemedicine platforms undermine patient care due to clinicians' inability to acquire all the relevant patient information required to diagnose and treat the patient. To explore this gap, we conducted a requirements analysis for the development of a 3D remote patient monitoring and diagnosis platform, by using a user-centric design methodology. In this requirements analysis, we elicited information about the clinical domain, identified clinicians’ requirements, elicited clinicians’ insights, and documented the clinicians' requirements. The outcome was the emergence of refined clinicians' requirements to guide the implementation of the Digital Twin concept paired with holoportation for remote 3D monitoring and diagnosis of patients. We anticipate that the application of a 3D telemedicine platform with these requirements for patient care during a pandemic could potentially enhance clinicians' efficiency and the effectiveness of remote patient care.

Multiple choice questions are a popular means of assessment for online examinations: easy to mark, but difficult to prepare in a way that makes it hard for students to gain high marks by sharing answers between them. Here we describe a systematic approach for creating multiple choice questions that can be used to test the understanding of bookwork topics, while still being challenging and mitigating against potential cheating.

The identification of groups in social networks drawn as graphs is an important task for social scientists who wish to know how a population divides with respect to relationships or attributes. Community detection algorithms identify communities (groups) in social networks by finding clusters in the graph: that is, sets of people (nodes) where the relationships (edges) between them are more numerous than their relationships with other nodes. This approach to determining communities is naturally based on the underlying structure of the network, rather than on attributes associated with nodes. In this paper, we report on an experiment that (a) compares the effectiveness of several force-directed graph layout algorithms for visually identifying communities, and (b) investigates their usefulness when group membership is based not on structure, but on attributes associated with the people in the network. We find algorithms that clearly separate communities with large distances to be most effective, while using colour to represent community membership is more successful than reliance on structural layout.

The Global Research Activity Map (GRAM) is an interactive web-based system for visualizing and analyzing worldwide scholarship activity as represented by research topics. The underlying data for GRAM is obtained from Google Scholar academic research profiles and is used to create a weighted topic graph. Nodes correspond to self-reported research topics and edges indicate co-occurring topics in the profiles. The GRAM system supports map-based interactive features, including semantic zooming, panning, and searching. Map overlays can be used to compare human resource investment, displayed as the relative number of active researchers in particular topic areas, as well scholarly output in terms of citations and normalized citation counts. Evaluation of the GRAM system, with the help of university research management stakeholders, reveals interesting patterns in research investment and output for universities across the world (USA, Europe, Asia) and for different types of universities. While some of these patterns are expected, others are surprising. Overall, GRAM can be a useful tool to visualize human resource investment and research productivity in comparison to peers at a local, regional and global scale. Such information is needed by university administrators to identify institutional strengths and weaknesses and to make strategic data-driven decisions.

Abstract When looking at drawings of graphs, questions about graph density, community structures, local clustering and other graph properties may be of critical importance for analysis. While graph layout algorithms have focused on minimizing edge crossing, symmetry, and other such layout properties, there is not much known about how these algorithms relate to a user's ability to perceive graph properties for a given graph layout. In this study, we apply previously established methodologies for perceptual analysis to identify which graph drawing layout will help the user best perceive a particular graph property. We conduct a large scale (n = 588) crowdsourced experiment to investigate whether the perception of two graph properties (graph density and average local clustering coefficient) can be modeled using Weber's law. We study three graph layout algorithms from three representative classes (Force Directed ‐ FD, Circular, and Multi‐Dimensional Scaling ‐ MDS), and the results of this experiment establish the precision of judgment for these graph layouts and properties. Our findings demonstrate that the perception of graph density can be modeled with Weber's law. Furthermore, the perception of the average clustering coefficient can be modeled as an inverse of Weber's law, and the MDS layout showed a significantly different precision of judgment than the FD layout.

For decades, researchers in information visualisation and graph drawing have focused on developing techniques for the layout and display of very large and complex networks. Experiments involving human participants have also explored the readability of different styles of layout and representations for such networks. In both bodies of literature, networks are frequently referred to as being ‘large’ or ‘complex’, yet these terms are relative. From a human-centred, experiment point-of-view, what constitutes ‘large’ (for example) depends on several factors, such as data complexity, visual complexity, and the technology used. In this paper, we survey the literature on human-centred experiments to understand how, in practice, different features and characteristics of node–link diagrams affect visual complexity.

The most common depictions of graphs are node-link diagrams (NLDs) and matrix-based diagrams (MBDs). Making valid comparisons between these two visualisation techniques is difficult because they are each subject to a variety of representation parameters with respect to graph layout (NLD) and node ordering (MBD), meaning that any given choice of layout and order (even if they fulfil some aesthetic criteria) may influence experimental results. To overcome this problem, we propose a MBD-based technique which hybridises the entity visual encoding of a MBD with the edge visual encoding of a NLD. Using a typical MBD, we propose three edge visual encoding evolutions to ultimately render edges like in a NLD while preserving nodes depiction and order. Such encoding evolutions allow us to perform an experimental evaluation of user performances for a path finding task without the above limitations. We show that for a path finding task, our edge visual encoding evolutions tend to improve the user experience when analysing and interacting with a MBD.

In dynamic graph drawing, preserving the mental map, or ensuring that the location of nodes do not change significantly as the information evolves over time is considered an important property by algorithm designers. Many prior experiments have attempted to verify this principle, with surprisingly little success. These experiments have used several different algorithmic methods, a variety of graph interpretation questions on both real and fabricated data, and different presentation methods. However, none of the results have conclusively demonstrated the importance of mental map preservation on task performance. Our experiment measures the efficacy of the dynamic graph drawing in a different manner: we look at how memorable the evolving graph is, rather than how easy it is to interpret. As observed in the previous studies, we found no significant difference in terms of response time or error rate when preserving the mental map. While preserving the mental map is a good idea in principle, we find that it may not always support performance. However, our qualitative data suggests that, in terms of the user's perception, preserving the mental map makes memorability tasks easier. Our qualitative data also suggests that there may be two features of the dynamic graph drawing that may assist in their memorability: interesting subgraphs that remain visible over time and interesting patterns in node movement. The former is supported by preserving the mental map while the latter is not.

We investigate the cognitive impact of various layout features-symmetry, alignment, collinearity, axis alignment and orthogonality - on the recall of network diagrams (graphs). This provides insight into how people internalize these diagrams and what features should or shouldn't be utilised when designing static and interactive network-based visualisations. Participants were asked to study, remember, and draw a series of small network diagrams, each drawn to emphasise a particular visual feature. The visual features were based on existing theories of perception, and the task enabled visual processing at the visceral level only. Our results strongly support the importance of visual features such as symmetry, collinearity and orthogonality, while not showing any significant impact for node-alignment or parallel edges.

Prior empirical work on layout aesthetics for graph drawing algorithms has concentrated on the interpretation of existing graph drawings. We report on experiments which focus on the creation and layout of graph drawings: participants were asked to draw graphs based on adjacency lists, and to lay them out "nicely." Two interaction methods were used for creating the drawings: a sketch interface which allows for easy, natural hand movements, and a formal point-and-click interface similar to a typical graph editing system. We find, in common with many other studies, that removing edge crossings is the most significant aesthetic, but also discover that aligning nodes and edges to an underlying grid is important. We observe that the aesthetics favored by participants during creation of a graph drawing are often not evident in the final product and that the participants did not make a clear distinction between the processes of creation and layout. Our results suggest that graph drawing systems should integrate automatic layout with the user's manual editing process, and provide facilities to support grid-based graph creation.

Euler diagrams are often used to visualize intersecting data sets in applications such as criminology; genetics, medicine, and computer file systems. One interesting aspect of these diagrams is that some data sets cannot be drawn without breaking one or more "wellformedness properties," which are considered to reduce the user comprehension of the diagrams. However, it is possible to draw the same data with different diagrams, each of which breaks different wellformedness properties. Hence, some properties are "swappable," so motivating the study of which of the alternatives would be best to use. This paper reports on the two empirical studies to determine how wellformedness properties affect comprehension. One study was with abstract data, the other was with concrete data that visualized students' enrollment on university modules. We have results from both studies that imply that diagrams with concurrency or disconnected zones perform less well than other some other properties. Further, we have no results that imply that diagrams with brushing points adversely affect performance. Our data also indicate that nonsimple curves are preferred less than diagrams with other properties. These results will inform both human diagram designers and the developers of automated drawing systems on the best way to visualize data using Euler diagrams.

In this paper, we present the results of a human-computer interaction experiment that compared the performance of the animation of dynamic graphs to the presentation of small multiples and the effect that mental map preservation had on the two conditions. Questions used in the experiment were selected to test both local and global properties of graph evolution over time. The data sets used in this experiment were derived from standard benchmark data sets of the information visualization community. We found that small multiples gave significantly faster performance than animation overall and for each of our five graph comprehension tasks. In addition, small multiples had significantly more errors than animation for the tasks of determining sets of nodes or edges added to the graph during the same timeslice, although a positive time-error correlation coefficient suggests that, in this case, faster responses did not lead to more errors. This result suggests that, for these two tasks, animation is preferable if accuracy is more important than speed. Preserving the mental map under either the animation or the small multiples condition had little influence in terms of error rate and response time.

Students in a large introductory programming course were asked twice to predict their scores on the final exam: once at the beginning of a six-week module, and once at the end. In between, students in only one of the two lecture streams recorded subjective confidence in their answers to individual questions on weekly quizzes. Students' predictions were moderately correlated with their scores. Students who attended more quizzes had not only higher exam scores, but improved their predictions more than those who attended fewer quizzes. Practice recording confidence on individual quiz questions did not yield significantly more improvement in exam predictions. Several findings from previous work are confirmed, including that women were significantly more underconfident than men.

Abstract Graph visualization systems often exploit opaque metanodes to reduce visual clutter and improve the readability of large graphs. This filtering can be done in a path‐preserving way based on attribute values associated with the nodes of the graph. Despite extensive use of these representations, as far as we know, no formal experimentation exists to evaluate if they improve the readability of graphs. In this paper, we present the results of a user study that formally evaluates how such representations affect the readability of graphs. We also explore the effect of graph size and connectivity in terms of this primary research question. Overall, for our tasks, we did not find a significant difference when this clustering is used. However, if the graph is highly connected, these clusterings can improve performance. Also, if the graph is large enough and can be simplified into a few metanodes, benefits in performance on global tasks are realized. Under these same conditions, however, performance of local attribute tasks may be reduced.

While popularity of peer assessment in Computer Science has increased in recent years, the validity of peer assessed marks remain a significant concern to instructors and source of anxiety to students. We report here on a large-scale study (1,500 students and 10,000 reviews) involving three introductory programming classes which recorded grades and feedback comments for both student and tutor reviews of novice programs. Using a paired analysis, we compare the quantitative marks given by students with those given by tutors, for both functional and non-functional aspects of the program. We also report on an analysis of the lexical sophistication of feedback comments.

A recent approach to engaging students in deep learning involves an online tool, PeerWise, through which students contribute multiple-choice questions to a shared question bank. Earlier work demonstrated a strong correlation between the use of PeerWise and student performance. In this study we investigate the quality of the MCQ repository created by students in an introductory programming course by analysing the range of topics on which students chose to write questions (i.e.~the repository coverage) without guidance from an instructor. We assess the repository coverage by comparing it with a common list of typical introductory programming topics, and by looking at its extent. We find that, despite having freedom to choose any topic, students created a repository that covered all the major topics in the curriculum.

University marketing endeavours attempt to encourage school leavers to study Computing Science; the success of such attempts are becoming increasingly crucial as we see a drop in Computing Science enrollments. However, the marketing typically focusses on presenting Computing Science from the point of view of how we academics view it, with little understanding of what school pupils know of Computing Science, or their perceptions of it. We report on an extensive study into high school students' perceptions of Computing Science as a discipline, degree programme and career, and conclude that few students have a clear notion as to what Computing Science is, and that their perceptions of the discipline develop early on in their school career.

PeerWise is a system in which students create multiple choice questions and answer those created by their peers. In this paper, we report on some quantitative results which suggest that students who use PeerWise actively perform better in final examinations than students who are not active. We note a significant correlation between performance in written (not just multiple choice) questions and PeerWise activity, suggesting that active use of the system may contribute to deep (and not just drill-and-practise) learning.

Tactons are structured vibrotactile messages which can be used for non-visual information presentation when visual displays are limited, unavailable or inappropriate, such as in mobile phones and other mobile devices. Little is yet known about how to design them effectively. Previous studies have investigated the perception of Tactons which encode two dimensions of information using two different vibrotactile parameters (rhythm and roughness) and found recognition rates of around 70. When more dimensions of information are required it may be necessary to extend the parameter-space of these Tactons. Therefore this study investigates recognition rates for Tactons which encode a third dimension of information using spatial location. The results show that identification rate for three-parameter Tactons is just 48, but that this can be increased to 81 by reducing the number of values of one of the parameters. These results will aid designers to select suitable Tactons for use when designing mobile displays.

Electronic Blocks are a new programming environment, designed specifically for children aged between three and eight years. As such, the design of the Electronic Block environment is firmly based on principles of developmentally appropriate practices in early childhood education. The Electronic Blocks are physical, stackable blocks that include sensor blocks, action blocks and logic blocks. Evaluation of the Electronic Blocks with both preschool and primary school children shows that the blocks' ease of use and power of engagement have created a compelling tool for the introduction of meaningful technology education in an early childhood setting. The key to the effectiveness of the Electronic Blocks lies in an adherence to theories of development and learning throughout the Electronic Blocks design process.

Information visualisation systems which generate diagrams representing discrete relational information must consider potential users if they are to be effective. Many algorithms which render an abstract graph structure as a diagram are valued for their conformance to aesthetic criteria (e.g. reducing the number of edge crossings, maximising symmetry), or for computational efficiency. They are not usually judged on their ability to produce diagrams that maximise human performance. This paper presents the results of experiments investigating the relative worth (from an HCI point of view) of graph drawing aesthetics and algorithms using a single graph. The results indicate that while some individual aesthetics affect human performance, it is difficult to say that one algorithm is ‘better’ than another from a relational understanding point of view. Designers of automatic layout algorithms, and the systems which embody such algorithms, can benefit from this study and this human-centred approach, by adapting their methods to focus on user concerns, rather than computational ones.