This study investigates how AI feedback influences college students’ reading and writing performance within an intelligent text aligned with the Interactive Constructive Active and Passive (ICAP) framework. Using a within-subjects design, students completed two read-to-write tasks — constructed response items (CRIs) and summaries — under two conditions: (1) Strategic Thinking And Interactive Reading Support (STAIRS), which provided AI feedback to support interactive engagement, and (2) Random Reread, a control condition without AI feedback to support constructive engagement. STAIRS offered automated scoring of read-to-write tasks, targeted rereading, and a chatbot that engaged students in interactive dialogues to support revision. Results showed that STAIRS significantly improved students’ summary revisions and subsequent summary performance, suggesting that AI feedback functions not only as guidance for improving current summaries but also benefits future performance. However, students in the STAIRS condition did not show higher CRI or summative quiz scores. CRI scores may have been limited by a ceiling effect, while the lack of quiz benefits may reflect a tradeoff in the study design, where the classroom tasks did not fully match the system’s activities. These mixed outcomes highlight the need to further explore how AI feedback may support different aspects of reading with an intelligent text.

Wearable cameras provide an efficient and convenient way to record our lives, supporting real-time documentation and analysis across various domains. Recent research has explored diverse methods for temporal segmentation, which aim to transform unstructured video data into structured events. This transformation facilitates deeper video understanding, optimizes computational resources, and improves the accessibility and interpretability of video content for both machines and humans. However, unlike conventional videos, wearable camera recordings present unique challenges. These include highly unstable camera perspectives, diverse activities across various environments, and flexible duration. As a result, traditional temporal segmentation methods often fail to return effective results. This paper introduces SmartSeg, an unsupervised, non-parametric approach for segmenting wearable camera videos without labeled data. By capturing the fundamental meanings of the video, SmartSeg aggregates the video through the Temporal Self-Similarity Metric encoder and groups sequences of frames into coherent events through clustering techniques. We evaluated SmartSeg on three diverse datasets. We achieved a 50% increase in Mean-over-Frames(MoF) compared to the state-of-the-art on one egocentric dataset. We conducted a real-world case study on nursing simulations, demonstrating SmartSeg’s ability to effectively segment complex, noisy interactions with diverse activity transitions. The results highlight SmartSeg’s robustness in handling long, unstructured, and visually challenging wearable camera videos, establishing it as a promising tool for real-world video temporal segmentation tasks.

Actionability is a critical, but understudied, issue in learning analytics for driving impact on learning. This study investigated access and action-taking of 91 students in an online undergraduate statistics course who received analytics designed for actionability twice a week for five weeks in the semester. Findings showed high levels of access, but little direct action through the provided links. The major contribution of the study was the identification of unexpected indirect actions taken by students in response to the analytics which requires us to think (and look for evidence of impact) more broadly than has been done previously. The study also found that integrating analytics into existing learning tools and routines can increase access rates to the analytics, but may not guarantee meaningful engagement without better strategies to manage analytic timing. Together, this study advances an understanding of analytic actionability, calling for a broader examination of both direct and indirect actions within a larger learning ecosystem.

This study addresses the gap in knowledge about differences in how instructors use analytics to inform teaching by examining the ways that thirteen college instructors engaged with a set of university-provided analytics. Using multiple walk-through interviews with the instructors and qualitative inductive coding, two profiles of instructor analytics use were identified that were distinct from each other in terms of the goals of analytics use, how instructors made sense of and took actions upon the analytics, and the ways that ethical concerns were conceived. Specifically, one group of instructors used analytics to help students get aligned to and engaged in the course, whereas the other group used analytics to align the course to meet students’ needs. Instructors in both profiles saw ethical questions as central to their learning analytics use, with instructors in one profile focusing on transparency and the other on student privacy and agency. These findings suggest the need to view analytics use as an integrated component of instructor teaching practices and envision complementary sets of technical and pedagogical support that can best facilitate the distinct activities aligned with each profile.

Participatory Design (PD), and Co-design (Co-D), can be effective ways to improve technological innovation and to incorporate users’ needs in the development of learning analytics (LA). However, these methods can be difficult to implement and there has yet to be a synopsis of how its and techniques have been applied to the specific needs of LA. This study reviewed 90 papers that described 52 cases of PD of LA between 2010 and 2020 to address the research question “How is participatory design (PD) being used within LA?”. It focuses on examining which groups of participants are normally included in PD for LA, in what phases of the design process it is used, and what specific tools and techniques have LA designers adapted or developed to co-create with design partners. Findings show that there is a growing number of researchers using these methods in recent years, particularly in higher education and with instructor stakeholders. However, it was also found that often the literature would describe the PD activities only superficially, and that some aspects of PD, such as recruitment, were seldom considered overtly in the descriptions of these processes.

Despite growing implementation of teacher-facing analytics in higher education, relatively little is known about the detailed processes through which instructors make sense of analytics in their teaching practices beyond their initial encounters with tools. This study unpacked the sensemaking process of thirteen instructors with analytic experience, using interviews that included walkthroughs of their analytics use. Qualitative inductive analysis was used to identify themes related to (1) the questions they asked of the analytics, (2) the techniques they used to interpret them, and (3) the challenges they encountered. Findings indicated that instructors went beyond a general curiosity to develop three types of questions of the analytics (goal-oriented, problem-oriented, and instruction modification questions). Instructors also used specific techniques to read and explain data by (a) developing expectations about the answers the analytics would provide, and (b) making comparisons to reveal student diversity, identify effects of instructional revision and diagnose issues. The study found instructors faced an initial learning curve when seeking and making use of relevant information, but also continued to revisit these challenges when they were not able to develop a routine of analytics use. These findings both contribute to a conceptual understanding of instructor analytic sensemaking and have practical implications for its systematic support.

This paper puts forth the idea of a subversive stance on learning analytics as a theoretically-grounded means of engaging with issues of power and equity in education and the ways in which they interact with the usage of data on learning processes. The concept draws on efforts from fields such as socio-technical systems and critical race studies that have a long history of examining the role of data in issues of race, gender and class. To illustrate the value that such a stance offers the field of learning analytics, we provide examples of how taking a subversive perspective can help us to identify tacit assumptions-in-practice, ask generative questions about our design processes and consider new modes of creation to produce tools that operate differently in the world.

Abstract This mixed methods study examined the impact of two design strategies on interactional processes in a collaborative tangible-tabletop land-use planning simulation. Twenty pairs of fifth grade children used the simulation to create a world they would want to live in. To investigate the impact of positive interdependence half the pairs were assigned one of two roles, each with an associated set of tangible ‘land-use’ stamp tools. All pairs were given access to pause and reflect tools. Quantitative results showed that children in the positive interdependence condition gave more one-way explanations to their partners than control pairs. They also had fewer but longer instances of bilaterally resolved conflict. Qualitative findings indicated the importance of pause and reflect tools for provoking explanations and resolving conflict. This study has revealed important considerations for the instantiation of positive interdependence and reflective pauses in collaborative tabletop learning systems, showing both quantitative and qualitative differences in the interactional processes that result from these design strategies. CCS CONCEPTS. Human-centered computing → Empirical studies in collaborative and social computing.

Reflection assessment is a critical component of health professions education that can be used for personalized learning support. However, reflection assessment at scale remains a challenge due to the demanding nature of tasks and the common use of simplified criteria of quality. This study addressed this issue by developing a multi-dimensional automated assessment that uses linguistic models to classify reflections by overall quality (depth) and the presence of six constituent elements denoting quality (description, analysis, feeling, perspective, evaluation, and outcome). 1500 reflections from 369 dental students were manually coded to establish ground truth. Classifiers for each of the six elements were trained and tested based on linguistic features extracted using the LIWC tool applying both single-label and multi-label classification approaches. Classifiers for depth were built both directly from linguistic features and based on the presence of the six elements. Results showed that linguistic modeling can be used to reliably detect the presence of reflection elements and the level of depth. However, the depth classifier showed a heavy reliance on cognitive elements (description, analysis, and evaluation) rather than the others. These findings indicate the feasibility of implementing multidimensional automated assessment in health professions education and the need to reconsider how quality of reflection is conceptualized.

This study explores the application of computational techniques to extract information about dental students' developing conceptions of their profession from digital reflective journal entries. Top concept networks were created for two cohorts of students at the beginning and end of their four-year program. A shift from a collection of general notions about becoming a professional to a more integrated, patient-centered conceptualization was found for both cohorts. The two groups initially differed in their perception of dental school (a mechanism for being able to work as a dentist versus a place to learn the skills to serve patients well) and subsequently in the extent of attention they paid to the feelings of their patients and themselves, as well as the continual growth of skill after graduation. Several useful linguistic markers were identified for examining these same issues in other cohorts. The results suggest that top concept networks can offer a useful window into students' developing conceptions of their profession. This kind of information can support student success on a macro level by offering feedback on existing curricula / informing learning designs to cultivate desired conceptions, and on a micro level through identifying particular ways individuals align with and diverge from the common trajectories.

This study examined the relationship between discussion forum contributions and course assessment results in a statistics MOOC. An important feature of the study is that it distinguished between discussions that were related to the learning of course material ("content-related") and those which were not ("non-content"). Another contribution is that the study evaluated the additional usefulness of social centrality measures in predicting course grade after the quantity of forum contributions has been accounted for. Results showed that, overall, 15% of course learners contributed to the forums and these learners had a significantly higher rate of successfully passing the course than non-contributors (64% vs 32% passing). Learners who made posts to both content-related and non-content threads had a higher passing rate than those who only contributed to one type or the other. Among learners who successfully passed the course, there were no differences in course grade when comparing discussion contributors and non-contributors overall; however those who contributed to content-related threads performed slightly better than those who did not (course grade of 87% vs 85%). A predictive model based on the number of posts made to content-related threads explained a small proportion of variance in course grades; addition of social centrality measures did not significantly improve the variance explained by the model.

This paper explores the potential of using naïve topic modeling to support instructors in navigating MOOC discussion forums. Categorizing discussion threads into topics can provide an overview of the discussion, improve navigation of the forum, and support replying to a representative sample of content related posts. We investigate four different approaches to using topic models to organize and present discussion posts, highlighting the strength and weaknesses of each approach to support instructors.

This study examines the impact of content-based network partitioning and tie definition on social network structures and interpretation for MOOC discussion forums. Using dynamic interrelated post and thread categorization [5] based on a previously developed natural language model [27], 817 threads containing 3124 discussion posts from 567 learners in a MOOC on the use of statistics in medicine were characterized as either related to the learning of course content or not. Content-related, non-content, and unpartitioned interaction networks were constructed based on five different tie definitions: Direct Reply, Star, Direct Reply+Star, Limited Copresence, and Total Copresence. Results showed content-related and non-content networks to have distinct characteristics at the network, community, and individual node levels, validating the usefulness of the content/non-content distinction as an analytic tool. Network properties were less sensitive to differences in tie definition with the exception of Total Copresence, which showed distinct characteristics presenting dangers for general use, but usefulness for detecting inflated social status due to "superthread" initiation.

This paper presents a thread characterization method that compares categorization results for thread starters and replies made by a previously-developed natural language model, using human judgment to resolve discrepancies. In an example application using the complete discussion forum data from a MOOC on medical statistics, the method increased the estimation of classification accuracy from .81 to .88 with the addition of a minimal number of human hours.

Interest in temporal analytics---analytics that probe temporal aspects of learning so as to gain insights into the processes through which learning occurs---continues to grow. The relationships of temporal patterns to learning outcomes is a central area of interest. However, while the literature on temporal analyses is developing, there has been less consideration of the methods by which temporal analyses might be translated to actionable insights and thus, put into use in educational practice. Emerging temporal analysis techniques present both theoretical and practical challenges for producing and interpreting results. Synergetic actions are needed in order to support practitioners.

This study addresses the issues of overload and chaos in MOOC discussion forums by developing a model to categorize and identify threads based on whether or not they are substantially related to the course content. Content-related posts were defined as those that give/seek help for the learning of course material and share/comment on relevant resources. A linguistic model was built based on manually-coded starting posts in threads from a statistics MOOC (n=837) and tested on thread starting posts from the second offering of the same course (n=304) and a different statistics course (n=298). The number of views and votes threads received were tested to see if they helped classification. Results showed that content-related posts in the statistics MOOC had distinct linguistic features which appeared to be unrelated to the subject-matter domain; the linguistic model demonstrated good cross-course reliability (all recall and precision > .77) and was useful across all time segments of the courses; number of views and votes were not helpful for classification.

Interest in analyses that probe the temporal aspects of learning continues to grow. The study of common and consequential sequences of events (such as learners accessing resources, interacting with other learners and engaging in self-regulatory activities) and how these are associated with learning outcomes, as well as the ways in which knowledge and skills grow or evolve over time are both core areas of interest. Learning analytics datasets are replete with fine-grained temporal data: click streams; chat logs; document edit histories (e.g. wikis, etherpads); motion tracking (e.g. eye-tracking, Microsoft Kinect), and so on. However, the emerging area of temporal analysis presents both technical and theoretical challenges in appropriating suitable techniques and interpreting results in the context of learning. The learning analytics community offers a productive focal ground for exploring and furthering efforts to address these challenges. This workshop, the fourth in a series on temporal analysis of learning, provides a focal point for analytics researchers to consider issues around and approaches to temporality in learning analytics.

This study investigated the extent to which students asked and instructors answered content-related questions in MOOC discussion forums; subsequently a classification model was built to identify such questions based on extracted linguistic features. Results showed content-related threads were a minority and under-addressed by instructors. However, linguistic modeling was promising in identifying them with high reliability.

Games for Change (G4C) is a movement and community of practice dedicated to using digital games for social change. However, a common model of persuasion built into most G4C, called Information Deficit, assumes that supporting children to learn facts will result in behavior change around social issues. There is little evidence that this approach works. We propose a model of game play, called Emergent Dialogue, which encourages children to discuss their values during interaction with factual information in a G4C. We summarize a set of guidelines based on our Emergent Dialogue model and apply them to the design of Youtopia, a tangible, tabletop learning game about sustainability. Our goal was to create a game that provided opportunities for children to express and discuss their values around sustainable development tradeoffs during game play. We evaluate our design using video, survey and questionnaire data. Our results provide evidence that our model and design guidelines are effective for supporting valuebased dialogue during collaborative game play.

Many studies suggest that tangibles and digital tabletops have potential to support collaborative interaction. However, previous findings show that users often work in parallel with such systems. One design strategy that may encourage collaboration rather than parallel use involves creating a system that responds to co-dependent access points in which more than one action is required to create a successful system response. To better understand how co-dependent access points support collaboration, we designed a comparative study with 12 young adults using the same application with a co-dependent and an independent access point design. We collected and analyzed categories of both verbal and behavioural data in the two conditions. Our results show support for the co-dependent strategy and suggest ways that the co-dependent design can be used to support flexible collaboration on tangible tabletops for young adults.

This article addresses a relatively unexplored area in the emerging field of learning analytics, the design of learning analytics interventions. A learning analytics intervention is defined as the surrounding frame of activity through which analytic tools, data, and reports are taken up and used. It is a soft technology that involves the orchestration of the human process of engaging with the analytics as part of the larger teaching and learning activity. This paper first makes the case for the overall importance of intervention design, situating it within the larger landscape of the learning analytics field, and then considers the specific issues of intervention design for student use of learning analytics. Four principles of pedagogical learning analytics intervention design that can be used by teachers and course developers to support the productive use of learning analytics by students are introduced: Integration, Agency, Reference Frame and Dialogue. In addition three core processes in which to engage students are described: Grounding, Goal-Setting and Reflection. These principles and processes are united in a preliminary model of pedagogical learning analytics intervention design for students, presented as a starting point for further inquiry.

Youtopia is a hybrid tangible and multi-touch land use planning activity for elementary school aged children. It was implemented on a Microsoft Pixelsense digital tabletop. The main method of interaction is through physical stamp objects that children use to "stamp" different land use types onto an interactive map. Youtopia was developed to investigate issues surrounding how to design and evaluate children's collaborative learning applications using digital tabletops. In particular we are looking at how the interface design supports in depth discussion and negotiation between pairs of children around issues in sustainable development. Our primary concern is to investigate questions about codependent access points, which may enable positive interdependence among children. Codependent access points are characteristics that enable two or more children to participate and interact together. In Youtopia these implemented through sequences of stamps that are required for successful interaction, which can be assigned to children (codependent mode) or remain unassigned (independent mode).

This paper describes an application of learning analytics that builds on an existing research program investigating how students contribute and attend to the messages of others in online discussions. A pedagogical model that translates the concepts and findings of the research program into guidelines for practice and analytics with which students and instructors can assess their discussion participation are presented. The analytics are both embedded in the learning environment and extracted from it, allowing for integrated and reflective metacognitive activity. The pedagogical intervention is based on the principles of (1) Integration (2) Diversity (of Metrics) (3) Agency (4) Reflection (5) Parity and (6) Dialogue. Details of an initial implementation of this approach and preliminary findings are described. Initial results strongly support the value of student-teacher dialogue around the analytics. In contrast, instructor parity in analytics use did not seem as important to students as was expected. Analytics were reported as useful in validating invisible discussion activity, but at times triggered emotionally-charged responses.

Many researchers have suggested that tangible user interfaces (TUIs) have potential for supporting learning. However, the theories used to explain possible effects are often invoked at a very broad level without explication of specific mechanisms by which the affordances of TUIs may be important for learning processes. Equally problematic, we lack theoretically grounded guidance for TUI designers as to what design choices might have significant impacts on learning and how to make informed choices in this regard. In this paper, we build on previous efforts to address the need for a structure to think about TUI design for learning by constructing the Tangible Learning Design Framework. We first compile a taxonomy of five elements for thinking about the relationships between TUI features, interactions and learning. We then briefly review cognitive, constructivist, embodied, distributed and social perspectives on cognition and learning and match specific theories to the key elements in the taxonomy to determine guidelines for design. In each case, we provide examples from previous work to explicate our guidelines; where empirical work is lacking, we suggest avenues for further research. Together, the taxonomy and guidelines constitute the Tangible Learning Design Framework. The framework advances thinking in the area by highlighting decisions in TUI design important for learning, providing initial guidance for thinking about these decisions through the lenses of theories of cognition and learning, and generating a blueprint for research on testable mechanisms of action by which TUI design can affect learning.

We describe a tangible user interface-based learning environment for children called Towards Utopia. The environment was designed to enable children, aged seven to ten, to actively construct knowledge around concepts related to land use planning and sustainable development in their community. We use Towards Utopia as a research prototype to investigate how and why tangible users interfaces can be designed to support, augment, or constrain learning opportunities. We follow a design-oriented research approach that includes a theoretically grounded analysis of design features of Towards Utopia to understand how and why design choices influence the kinds of learning opportunities created. We also describe the results of our empirical evaluation of learning outcomes in order to validate the effectiveness of our design. We conclude with general guidelines for the design of tangibles for learning.