Human tutoring interventions play a crucial role in supporting student learning, improving academic performance, and promoting personal growth. This paper focuses on analyzing mathematics tutoring discourse using talk moves—a framework of dialogue acts grounded in Accountable Talk theory. However, scaling the collection, annotation, and analysis of extensive tutoring dialogues to develop machine learning models is a challenging and resource-intensive task. To address this, we present SAGA22, a compact dataset, and explore various modeling strategies, including dialogue context, speaker information, pretraining datasets, and further fine-tuning. By leveraging existing datasets and models designed for classroom teaching, our results demonstrate that supplementary pretraining on classroom data enhances model performance in tutoring settings, particularly when incorporating longer context and speaker information. Additionally, we conduct extensive ablation studies to underscore the challenges in talk move modeling.

Transcripts of teaching episodes can be effective tools to understand discourse patterns in classroom instruction. According to most educational experts, sustained classroom discourse is a critical component of equitable, engaging, and rich learning environments for students. This paper describes the TalkMoves dataset, composed of 567 human-annotated K-12 mathematics lesson transcripts (including entire lessons or portions of lessons) derived from video recordings. The set of transcripts primarily includes in-person lessons with whole-class discussions and/or small group work, as well as some online lessons. All of the transcripts are human-transcribed, segmented by the speaker (teacher or student), and annotated at the sentence level for ten discursive moves based on accountable talk theory. In addition, the transcripts include utterance-level information in the form of dialogue act labels based on the Switchboard Dialog Act Corpus. The dataset can be used by educators, policymakers, and researchers to understand the nature of teacher and student discourse in K-12 math classrooms. Portions of this dataset have been used to develop the TalkMoves application, which provides teachers with automated, immediate, and actionable feedback about their mathematics instruction.

With the proliferation of 3D printing technologies in schools and makerspaces, there is a need for teaching 3D modeling to students. Learning 3D modeling enhances spatial thinking skills, an essential skill for success in STEM. Creating 3D models requires students to have a deep understanding of 3D space, including rotating and scaling. In this study, we propose a framework developed through video-coding from analyzing screen recordings of middle-school students’ usage of a 3D modeling tool - Tinkercad. The proposed framework focuses on identifying challenges students encounter during 3D modeling. These challenges include spatial thinking skills, working with the Tinkercad interface, and mental model formation. We authenticated the framework by collecting and analyzing data from a 3D printing unit in three middle schools. Our results and subsequent analysis can guide educators and researchers on how to use this framework to support students in having productive learning experiences with Computer-Aided Design tools.

Engaging in physical computing activities involving both hardware and software provides a hands-on introduction to computer science. The move to remote learning for primary and secondary schools during the 2020-2021 school year due to COVID-19 made implementing physical computing activities especially challenging. However, it is important that these activities are not simply eliminated from the curriculum. This paper explores how a unit centered around students investigating how programmable sensors that can support data-driven scientific inquiry was collaboratively adapted for remote instruction. A case study of one teacher’s experience implementing the unit with a group of middle school students (ages 11 to 14) in her STEM elective class examines how her students could still engage in computational thinking practices around data and programming. The discussion includes both the challenges and unexpected affordances of engaging in physical computing activities remotely that emerged from her implementation.

This paper describes the design and classroom implementation of a week-long unit that aims to integrate computational thinking (CT) into middle school science classes using programmable sensor technology. The goals of this sensor immersion unit are to help students understand why and how to use sensor and visualization technology as a powerful data-driven tool for scientific inquiry in ways that align with modern scientific practice. The sensor immersion unit is anchored in the investigation of classroom data where students engage with the sensor technology to ask questions about and design displays of the collected data. Students first generate questions about how data data displays work and then proceed through a set of programming exercises to help them understand how to collect and display data collected from their classrooms by building their own mini data displays. Throughout the unit students draw and update their hand drawn models representing their current understanding of how the data displays work. The sensor immersion unit was implemented by ten middle school science teachers during the 2019/2020 school year. Student drawn models of the classroom data displays from four of these teachers were analyzed to examine students' understandings in four areas: function of sensor components, process models of data flow, design of data displays, and control of the display. Students showed the best understanding when describing sensor components. Students exhibited greater confusion when describing the process of how data streams moved through displays and how programming controlled the data displays.

Our work builds on advances in deep learning for natural language processing to automatically analyze transcribed classroom discourse and reliably generate information about teachers’ uses of specific discursive strategies called ”talk moves.” Talk moves can be used by both teachers and learners to construct conversations in which students share their thinking, actively consider the ideas of others, and engage in sustained reasoning. Currently, providing teachers with detailed feedback about the talk moves in their lessons requires highly trained observers to hand code transcripts of classroom recordings and analyze talk moves and/or one-on-one expert coaching, a time-consuming and expensive process that is unlikely to scale. We created a bidirectional long short-term memory (bi-LSTM) network that can automate the annotation process. We have demonstrated the feasibility of this deep learning approach to reliably identify a set of teacher talk moves at the sentence level with an F1 measure of 65%.

As robots increasingly enter our daily lives, there is a need to understand how to design robots capable of emotional interaction with humans, especially children, due to their sensitivity and vulnerability. For example, robots that provide children with social and emotional support might be more effective at also helping children develop cognitive abilities, rather than designing robots that focus solely on helping children acquire cognitive skill. In this paper, we examine the design of robots that can provide human-like hugs as a particular form of social and emotional support. We first discuss the need to design robots that can interact emotionally with children. Then, we present the development of a shirt augmented with pressure sensors used to collect data on how humans hug each other. Finally, we detail the design of "Hugbot", a soft robot that could use this data to give human-like hugs, and discuss our planned future work on this system.

This experience report describes two iterations of a curriculum development process in which middle school teachers worked with our research team to collaboratively design and enact instructional units where students used sensors to investigate scientific phenomena. In this report, we examine the affordances of using a sensor platform to support the integration of disciplinary learning and computational thinking (CT) aligned with Next Generation Science Standards and the CT in STEM Taxonomy developed by Weintrop and colleagues. In the first unit, students investigated the conditions for mold growth within their school using a custom sensor system. After analyzing implementation experiences and student interest data, our team engaged in another round of co-design to develop a second instructional unit. This unit uses a different sensor system (the micro:bit) which supports additional CT in STEM practices due to its block-based programming interface and its real time data display. For the second unit we selected a different phenomenon: understanding and designing maglev trains.

Currently, providing teachers with detailed feedback about their classroom discourse strategies requires highly trained observers to hand code transcripts of classroom recordings to identify talk moves and/or one-on-one expert coaching. Both approaches are time-consuming and expensive, require considerable human expertise, and do not scale to large numbers of teachers. We are currently developing an innovative application, the TalkBack application, a new type of teacher learning environment based on the automated analysis of classroom recordings. The TalkBack application will utilize a big data infrastructure for managing and analyzing classroom recordings, including an embedded automated talk move classifier. The application will provide teachers with a detailed record of the discourse strategies used in their lessons. A central premise of our research is that this type of personalized, automated feedback can dramatically enhance teacher learning and support improvements in their instruction.The project will exemplify how next-generation repositories of classroom recordings can be architected to support large-scale research by enabling automated analyses based on machine learning models.

Iterative design is a powerful method for developing digital classroom tools and curricula. We explore how infusing learning analytics into this process has influenced our development of EcoSurvey, a digital modeling tool for mapping the organisms and interactions in the local ecosystem. We have found that analytic techniques can help us discover areas in which students struggle to engage with scientific modeling, and we can iteratively use learning analytics to demonstrate the impact of design changes.

Modeling has a strong focus in current science learning frameworks as a critical skill for students to learn. However, understanding students' scientific models and their modeling practices at scale is a difficult task that has not been taken up by the research literature. The complex variables involved in classroom learning, such as teacher differences, increase the difficulty of understanding this problem. This work begins with an exploration of the methods used to explore students' scientific modeling in the learning sciences space and the frameworks developed to characterize student modeling practices. Learning analytics can be used to leverage these frameworks of scientific modeling practices to explore questions around students' scientific models and their modeling practices. These analyses are focused around the use of EcoSurvey, a collaborative, digital tool used in high-school biology classrooms to model the local ecosystem. This tool was deployed in ten biology classrooms and used with varying degrees of success. There are significant teacher-level differences found in the activity sequences of students using the EcoSurvey tool. The theoretical metrics around scientific modeling practices and automatically extracted feature sequences were also used in a classification task to automatically determine a particular student's teacher. These results underline the power of learning analytics methods to give insight into how modeling practices are realized in the classroom. This work also informs changes to modeling tools, associated curricula, and supporting professional development around scientific modeling.

Md Arafat Sultan, Jordan Boyd-Graber, Tamara Sumner. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.

Md Arafat Sultan, Cristobal Salazar, Tamara Sumner. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.

We show that weak ties offer a useful theoretical lens for understanding the sharing and usage of community-contributed resources amongst educators in a large urban school district. Community-contributed resources include a rich variety of teaching and learning resources such as lesson plans, presentation slides, animations and simulations. In this research, we consider whether the deduced relationships between members of the community constitute weak ties. A deduced relationship exists when two community members view or access the same resource. If these deduced relationships do constitute weak ties then other theorized network properties should also be manifest, namely homophily and triadic closures. Our findings support these theoretical conjectures. Firstly, results indicate that the strength of a tie is directly proportional to the level of similarity between users in the network (homophily property). Secondly, we found strong support for the triadic closure property as well; we developed a computational model to predict the formation of weak ties via triadic closures with an accuracy of 97.8%. Insights from our model can be used to improve a collaborative filtering approach for resource recommendation by predicting future similarity between users in the network.

Monolingual alignment is the task of pairing semantically similar units from two pieces of text.We report a top-performing supervised aligner that operates on short text snippets.We employ a large feature set to ( 1) encode similarities among semantic units (words and named entities) in context, and (2) address cooperation and competition for alignment among units in the same snippet.These features are deployed in a two-stage logistic regression framework for alignment.On two benchmark data sets, our aligner achieves F 1 scores of 92.1% and 88.5%, with statistically significant error reductions of 4.8% and 7.3% over the previous best aligner.It produces top results in extrinsic evaluation as well.

We present a simple, easy-to-replicate monolingual aligner that demonstrates state-of-the-art performance while relying on almost no supervision and a very small number of external resources. Based on the hypothesis that words with similar meanings represent potential pairs for alignment if located in similar contexts, we propose a system that operates by finding such pairs. In two intrinsic evaluations on alignment test data, our system achieves F1 scores of 88–92%, demonstrating 1–3% absolute improvement over the previous best system. Moreover, in two extrinsic evaluations our aligner outperforms existing aligners, and even a naive application of the aligner approaches state-of-the-art performance in each extrinsic task.

Efficient learning from Web resources can depend on accurately assessing the quality of each resource. We present a methodology for developing computational models of quality that can assist users in assessing Web resources. The methodology consists of four steps: 1) a meta-analysis of previous studies to decompose quality into high-level dimensions and low-level indicators, 2) an expert study to identify the key low-level indicators of quality in the target domain, 3) human annotation to provide a collection of example resources where the presence or absence of quality indicators has been tagged, and 4) training of a machine learning model to predict quality indicators based on content and link features of Web resources. We find that quality is a multifaceted construct, with different aspects that may be important to different users at different times. We show that machine learning models can predict this multifaceted nature of quality, both in the context of aiding curators as they evaluate resources submitted to digital libraries, and in the context of aiding teachers as they develop online educational resources. Finally, we demonstrate how computational models of quality can be provided as a service, and embedded into applications such as Web search.

Educative sensemaking focuses on the needs of self-directed learners, a nonexpert population of thinkers who must locate relevant information sources, evaluate the applicability and accuracy of digital resources for learning, and determine how and when to use these resources to complete educational tasks. Self-directed learners face a sensemaking paradox: They must employ deep-level thinking skills to process information sources meaningfully, but they often lack the requisite domain knowledge needed to deeply analyze information sources and to successfully integrate incoming information with their own existing knowledge. In this article, we focus on the needs of college-aged students engaged in learning about natural sciences using web-based learning resources. We explored the impact of cognitive personalization technologies on students' sensemaking processes using a controlled study in which students' cognitive and metacognitive processes were analyzed as they completed a common educational task: writing an essay. We coded students' observable on-screen behaviors, self-reported processes, final essays, and responses to domain assessments to assess benefits of personalization technologies on students' educative sensemaking. Results show that personalization supported students' analysis of knowledge representations, helped students work with their representations in meaningful ways, and supported effective encoding of new knowledge. We discuss implications for new technologies to help students overcome the educative sensemaking paradox.

A persona represents a group of target users that share common behavioral characteristics. By using a narrative, picture, and name, a persona provides HCI practitioners with a vivid and specific design target. This research develops a new methodology for the identification and creation of personas through the application of Latent Semantic Analysis (LSA). An application of the LSA methodology is provided in the context of the design of an Institutional Repository system. The LSA methodology helps overcome some of the drawbacks of current methods for the identification and creation of personas, and makes the process less subjective, more efficient, and less reliant on specialized skills.

Article Free Access Share on From documents to discourse: shifting conceptions of scholarly publishing Authors: Tamara Sumner Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K. Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K.View Profile , Simon Buckingham Shum Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K. Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K.View Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 95–102https://doi.org/10.1145/274644.274659Published:01 January 1998Publication History 22citation671DownloadsMetricsTotal Citations22Total Downloads671Last 12 Months19Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

We explore some of the complex issues surrounding the design and use of multimedia and Internet-based learning resources in distance education courses. We do so by analysing our experiences designing a diverse array of learning media for a large scale, distance learning course in introductory computing. During the project, we had to significantly rethink the design and production of our learning resources as we shifted from a paper-based teaching model to an interactive teaching model. This shift entailed changes to our design products (to promote more effective media use by learners) and changes to our design practices (to foster consistent media use and design across a large and distributed team). Course designers and course students alike needed help in breaking out of paper-based models of learning to obtain maximum benefit from the interactive teaching model.

Article Free Access Share on The cognitive ergonomics of knowledge-based design support systems Authors: Tamara Sumner Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, UK Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, UKView Profile , Nathalie Bonnardel Centre de Recherche en, Psychologie Cognitive, Université de Provence, 13621 Aix en Provence, France Centre de Recherche en, Psychologie Cognitive, Université de Provence, 13621 Aix en Provence, FranceView Profile , Benedikte Harstad Kallak Computas Expert Systems A.S., Leif Transtads Plass 6, Postboks 430, 1301 Sartdvika, Norway Computas Expert Systems A.S., Leif Transtads Plass 6, Postboks 430, 1301 Sartdvika, NorwayView Profile Authors Info & Claims CHI '97: Proceedings of the ACM SIGCHI Conference on Human factors in computing systemsMarch 1997 Pages 83–90https://doi.org/10.1145/258549.258613Published:27 March 1997Publication History 20citation1,198DownloadsMetricsTotal Citations20Total Downloads1,198Last 12 Months73Last 6 weeks14 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Article Free Access Share on The high-tech toolbelt: a study of designers in the workplace Author: Tamara Sumner Department of Computer Science and Institute of Cognitive Science, University of Colorado, Boulder, CO Department of Computer Science and Institute of Cognitive Science, University of Colorado, Boulder, COView Profile Authors Info & Claims CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMay 1995 Pages 178–185https://doi.org/10.1145/223904.223927Published:01 May 1995Publication History 5citation338DownloadsMetricsTotal Citations5Total Downloads338Last 12 Months26Last 6 weeks6 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF