As generative artificial intelligence (genAI) tools become embedded in computing education workflows, it is essential to understand how students use such systems to learn beyond introductory programming. This work investigates the relationship between the use of genAI by students and their conceptual understanding of mathematical and algorithmic principles in an undergraduate machine learning course with 134 students. We deploy a course-specific, custom-interfaced large language model (LLM), CubBot, to examine (1) how students interact with genAI in an upper-level CS course via an analysis of anonymized chat logs and (2) how genAI usage relates to students' conceptual understanding and learning outcomes via a randomized, controlled assessment comparing performance with and without CubBot access. This research contributes to the growing body of work on genAI-supported education by providing one of the first empirical investigations into genAI's relationship with conceptual learning in an upper-level CS course.

As undergraduates studying computer science progress through the curriculum, one additional kind of academic CS experience that many undertake, often outside of formal course structures, is their first CS research project. As anyone who has conducted CS research before will know, there are many components of working on research that don't necessarily come up in traditional classes. How do we as educators and research mentors teach them these skills?

We contribute an autoethnographic reflection on the complexity of defining and measuring visualization literacy (i.e., the ability to interpret and construct visualizations) to expose our tacit thoughts that often exist in-between polished works and remain unreported in individual research papers. Our work is inspired by the growing number of empirical studies in visualization research that rely on visualization literacy as a basis for developing effective data representations or educational interventions. Researchers have already made various efforts to assess this construct, yet it is often hard to pinpoint either what we want to measure or what we are effectively measuring. In this autoethnography, we gather insights from 14 internal interviews with researchers who are users or designers of visualization literacy tests. We aim to identify what makes visualization literacy assessment a "wicked" problem. We further reflect on the fluidity of visualization literacy and discuss how this property may lead to misalignment between what the construct is and how measurements of it are used or designed. We also examine potential threats to measurement validity from conceptual, operational, and methodological perspectives. Based on our experiences and reflections, we propose several calls to action aimed at tackling the wicked problem of visualization literacy measurement, such as by broadening test scopes and modalities, improving test ecological validity, making it easier to use tests, seeking interdisciplinary collaboration, and drawing from continued dialogue on visualization literacy to expect and be more comfortable with its fluidity.

Users often have access to multiple forecasts regarding an event.Different forecasts incorporate different assumptions and epistemic information.A growing body of work argues against decisionmaking solely based on expected utility maximisation strategies in multiple forecasts scenarios, in favour of other strategies such as the maximin expected utility.In this work, we compare two different approaches for depicting epistemic uncertainty-ensembles (a direct representation of multiple forecasts) and p-boxes (a representation which only communicates the bounds of epistemic uncertainty)-in plots where individual distributions are represented as cumulative distribution plots (CDFs).We conduct three experiments to investigate the impact of the visual representation on the decision-making strategies that people adopt.Our results suggest that participants adopt conservative decision-making strategies (i.e.place greater weight on the worst-case forecast than the best-case forecast) for both p-boxes and ensembles if the set of forecasts are uniformly distributed.However, if a majority of the forecasts are clustered near one of the bounds, participants may discount the forecast which appears as a visual outlier.

As a step towards improving visualization literacy, this work investigates how students approach reading visualizations differently after taking a university-level visualization course. We asked students to verbally walk through their process of making sense of unfamiliar visualizations, and conducted a qualitative analysis of these walkthroughs. Our qualitative analysis found that after taking a visualization course, students engaged with visualizations in more sophisticated ways: they were more likely to exhibit design empathy by thinking critically about the tradeoffs behind why a chart was designed in a particular way, and were better able to deconstruct a chart to make sense of it. We also gave students a quantitative assessment of visualization literacy and found no evidence of scores improving after the class, likely because the test we used focused on a different set of skills than those emphasized in visualization classes. While current measurement instruments for visualization literacy are useful, we propose developing standardized assessments for additional aspects of visualization literacy, such as deconstruction and design empathy. We also suggest that these additional aspects could be incorporated more explicitly in visualization courses. All supplemental materials are available at https://osf.io/w5pum/.

We propose a new approach to uncertainty communication: we keep the uncertainty representation fixed, but adjust the distribution displayed to compensate for biases in people’s subjective probability in decision-making. To do so, we adopt a linear-in-probit model of subjective probability and derive two corrections to a Normal distribution based on the model’s intercept and slope: one correcting all right-tailed probabilities, and the other preserving the mode and one focal probability. We then conduct two experiments on U.S. demographically-representative samples. We show participants hypothetical U.S. Senate election forecasts as text or a histogram and elicit their subjective probabilities using a betting task. The first experiment estimates the linear-in-probit intercepts and slopes, and confirms the biases in participants’ subjective probabilities. The second, preregistered follow-up shows participants the bias-corrected forecast distributions. We find the corrections substantially improve participants’ decision quality by reducing the integrated absolute error of their subjective probabilities compared to the true probabilities. These corrections can be generalized to any univariate probability or confidence distribution, giving them broad applicability. Our preprint, code, data, and preregistration are available at https://doi.org/10.17605/osf.io/kcwxm