Transcription with Incremental Presentation of the Stimulus (TIPS) is an instrument to assess users’ graph comprehension through trace data. TIPS records cognitive micro-behaviours (e.g., keystrokes and pen-strokes) with millisecond accuracy across cycles of viewing and copying. These signals provide eight potential TIPS measures to study learners’ competence. In an experiment with 30 participants and visualizations of varying complexity, several TIPS measures, particularly when normalized using a test of rigid spatial transformations (Mental Rotation Test), showed moderate to strong correlations (r = 0.5–0.6) with an independent measure of graph comprehension. Results were most robust for complex stimuli. Potential practical benefits of TIPS include shorter testing time, no need to create new questions for different stimuli, and the ability to fully automate scoring, making the assessment more efficient and scalable.

In this paper we introduce MaRE, a tool designed to generate representations in multiple modalities for a given mathematical problem while ensuring the correctness and interpretability of the transformations between different representations. The theoretical foundation for this tool is Representational Systems Theory (RST), a mathematical framework for studying the structure and transformations of representations. In MaRE’s web front-end user interface, a set of probability equations in Bayesian Notation can be rigorously transformed into Area Diagrams, Contingency Tables, and Probability Trees with just one click, utilising a back-end engine based on RST. A table of cognitive costs, based on the cognitive Representational Interpretive Structure Theory (RIST), that a representation places on a particular profile of user is produced at the same time. MaRE is general and domain independent, applicable to other representations encoded in RST. It may enhance mathematical education and research, facilitating multi-modal knowledge representation and discovery.

Providing an approach to model the memory structures that humans build as they use visualizations could be useful for researchers, designers and educators in the field of information visualization. Cheng and colleagues formulated Representation Interpretive Structure Theory (RIST) for that purpose. RIST adopts a human information processing perspective in order to address the immediate, short timescale, cognitive load likely to be experienced by visualization users. RIST is operationalized in a graphical modeling notation and browser-based editor. This paper demonstrates the utility of RIST by showing that (a): RIST models are compatible with established empirical and computational cognitive findings about differences in human performance on alternative representations; (b) they can encompass existing explanations from the literature; and, (c) they provide new explanations about causes of those performance differences.

Competence Assessment by Chunk Hierarchy Evaluation with Transcription-tasks (CACHET) was proposed by Cheng [14]. It analyses micro-behaviors captured during cycles of stimulus viewing and copying in order to probe chunk structures in memory. This study extends CACHET by applying it to the domain of graphs and charts. Since drawing strategies are diverse, a new interactive stimulus presentation method is introduced: Transcription with Incremental Presentation of the Stimulus (TIPS). TIPS aims to reduce strategy variations that mask the chunking signal by giving users manual element-by-element control over the display of the stimulus. The potential of TIPS, is shown by the analysis of six participants transcriptions of stimuli of different levels of familiarity and complexity that reveal clear signals of chunking. To understand how the chunk size and individual differences drive TIPS measurements, a CPM-GOMS model was constructed to formalize the cognitive process involved in stimulus comprehension and chunk creation.

Pólya and others recognised that an appropriate representation of a problem is key for enabling us to solve it. But choosing the right representation is a problem that novice problem solvers find difficult, so must turn to experts for guidance. In this paper, we present a study that examines how human experts recommend representations. We asked high school mathematics teachers to order representational systems based on their suitability generally, and with respect to a student profile. We found the teachers updated their recommendations based on the problem and student profile, but were inconsistent with each other. This inconsistency highlights a need for more training and support in representational system selection.

Choosing an effective representation is fundamental to the ability of the representation's user to exploit it for the intended purpose. The major contribution of this paper is to provide a novel, flexible framework, rep2rep, that can be used by AI systems to recommend effective representations. What makes an effective representation is determined by whether it expresses the necessary information, supports the execution of tasks, and reflects the user's cognitive abilities. In general, there is no single `most effective' representation for every problem and every user, which makes it difficult to choose one from the plethora of possible representations. To address this, rep2rep includes: a domain-independent language for describing representations, algorithms that compute measures of informational suitability and overall cognitive cost, and uses these measures to recommend representations. We demonstrate the application of rep2rep in the probability domain. Importantly, our framework provides the foundations for personalised interaction with AI systems in the context of representation choice.

Mathematics and computing students learn new concepts and fortify their expertise by solving problems. The representation of a problem, be it through algebra, diagrams, or code, is key to understanding and solving it. Multiple-representation interactive environments are a promising approach, but the task of choosing an appropriate representation is largely placed on the user. We propose a new method to recommend representations based on correspondences: conceptual links between domains. Correspondences can be used to analyse, identify, and construct analogies even when the analogical target is unknown. This paper explains how correspondences build on probability theory and Gentner's structure-mapping framework; proposes rules for semi-automated correspondence discovery; and describes how correspondences can explain and construct analogies.