The emerging concept of data storytelling (DS) suggests that enhancing visualisations with annotations and narratives can make complex data more insightful than conventional visualisations. Previous works found that DS-enhanced visualisations are more effective than conventional visualisations for simple tasks like identifying key data points or the main message. However, no previous work has explored the extent to which DS enhancements influence task completion across different levels of cognitive complexity. We address this gap by presenting the results of a study where 128 participants completed tasks based on four visualisations (two line charts and two choropleth maps, either with or without DS elements) spanning a range of complexity based on Bloom's taxonomy, which has been applied in data visualisation to categorise tasks hierarchically from lower to higher-order thinking. Results suggest that while DS-enhanced visualisations effectively support lower-order tasks (finding data points and understanding insights), they don't necessarily aid the correct completion of higher-order tasks (application, analysis, evaluation and creation). However, DS enhancements improve how efficiently participants complete complex tasks.

High staff turnover in higher education often burdens teachers with laborious handovers of teaching tasks every semester. To boost teachers' workflow efficiency, we present an innovative knowledge management tool that allows experienced teachers to seamlessly capture task steps (i.e., denoted as ShareFlow) that can be subsequently recommended to novices via proactive push, all happening during teachers' natural workflow to minimize disruptions. We conducted a controlled experiment with 30 participants and compared our tool against a state-of-the-art baseline knowledge management system powered by a large language model (Claude 3 Haiku). We found that our knowledge management tool reduced task completion time and improved task quality (with statistical significance). Feedback from the participants also indicated the high usability of our tool, suggesting its strong potential for practical adoption for improving teacher workflows.

While the capacity to self-regulate has been found to be crucial for secondary school students, prior studies often rely on self-report surveys and think-aloud protocols that present notable limitations in capturing self-regulated learning (SRL) processes. This study advances the understanding of SRL in secondary education by using trace data to examine SRL processes during multi-source writing tasks, with higher education participants included for comparison. We collected fine-grained trace data from 66 secondary school students and 59 university students working on the same writing tasks within a shared SRL-oriented learning environment. The data were labelled using Bannert's validated SRL coding scheme to reflect specific SRL processes, and we examined the relationship between these processes, essay performance, and educational levels. Using epistemic network analysis (ENA) to model and visualise the interconnected SRL processes in Bannert's coding scheme, we found that: (a) secondary school students predominantly engaged in three SRL processes - Orientation, Re-reading, and Elaboration/Organisation; (b) high-performing secondary students engaged more in Re-reading, while low-performing students showed more Orientation process; and (c) higher education students exhibited more diverse SRL processes such as Monitoring and Evaluation than their secondary education counterparts, who heavily relied on following task instructions and rubrics to guide their writing. These findings highlight the necessity of designing scaffolding tools and developing teacher training programs to enhance awareness and development of SRL skills for secondary school learners.

In computer-based learning environments (CBLEs), adopting effective self-regulated learning (SRL) strategies requires sophisticated coordination of multiple SRL processes. While various studies have proposed adaptive SRL scaffolds (i.e. real-time advice on adopting effective SRL processes) and embedded them in CBLEs to facilitate learners' effective use of SRL strategies, two key research gaps remain. First, there is a lack of research on SRL scaffolds that are based on continuous assessment of both learners' SRL processes and learning conditions (e.g., awareness of learning resources) to provide adaptive support. Second, current analytics-based scaffolding mechanisms lack the scalability needed to effectively address multiple learning conditions. Integration of analytics of SRL with generative artificial intelligence (GenAI) can provide scalable scaffolding for real-time SRL processes and evolving conditions. Yet, empirical studies implementing and evaluating effects of this integration remain scarce. To address these limitations, we conducted a randomized control trial, assigning participants to three groups (control, process only, and process with condition groups) to investigate the effects of using GenAI to turn insights from real-time analytics about students' SRL processes and conditions into adaptive scaffolds. The results demonstrate that integrating real-time analytics with GenAI in adaptive SRL scaffolds - addressing both SRL processes and dynamic conditions - promotes more metacognitive learning patterns compared to the control and process-only groups. In addition, the learners showed varying levels of compliance with analytics-based GenAI scaffolds, and this was also reflected in how the learners coordinated their SRL processes, particularly in the performance phase of SRL. This study contributes to the literature by designing, implementing, and evaluating the impact of adaptive scaffolds on learners' SRL processes using real-time analytics with GenAI.

Knowledge management involves finding, expanding, and using knowledge in an organisation to achieve goals. Its role is crucial in higher education to improve problem-solving, research, and teaching by acquiring, sharing, and applying knowledge. Higher education institutions can promote knowledge management through Communities of Practice, but doing so remains challenging due to cultural, organisational, and technological reasons. We present findings of the first step of co-design workshops with authentic higher education teaching teams that sought to understand (a) their practices as a community and any motivators and impediments to their community development; (b) how they perceived the tools they use for knowledge management; and (c) the kinds of tools they believed could help them better conduct knowledge management and develop as Communities of Practice. Our findings suggested four essential design requirements and informed our development of a new tool to support the knowledge management needs of higher education teaching teams.

Educational feedback has been widely acknowledged as an effective approach to improving student learning. However, scaling effective practices can be laborious and costly, which motivated researchers to work on automated feedback systems (AFS). Inspired by the recent advancements in the pre-trained language models (e.g., ChatGPT), we posit that such models might advance the existing knowledge of textual feedback generation in AFS because of their capability to offer natural-sounding and detailed responses. Therefore, we aimed to investigate the feasibility of using ChatGPT to provide students with feedback to help them learn better. Our results show that i) ChatGPT is capable of generating more detailed feedback that fluently and coherently summarizes students' performance than human instructors; ii) ChatGPT achieved high agreement with the instructor when assessing the topic of students' assignments; and iii) ChatGPT could provide feedback on the process of students completing the task, which might benefit students developing learning skills.

Feedback is an effective way to assist students in achieving learning goals. The conceptualisation of feedback is gradually moving from feedback as information to feedback as a learner-centred process. To demonstrate feedback effectiveness, feedback as a learner-centred process should be designed to provide quality feedback content and promote student learning outcomes on the subsequent task. However, it remains unclear how instructors adopt the learner-centred feedback framework for feedback provision in the teaching practice. Thus, our study made use of a comprehensive learner-centred feedback framework to analyse feedback content and identify the characteristics of feedback content among student groups with different performance changes. Specifically, we collected the instructors’ feedback on two consecutive assignments offered by an introductory to data science course at the postgraduate level. On the basis of the first assignment, we used the status of student grade changes (i.e., students whose performance increased and those whose performance did not increase on the second assignment) as the proxy of the student learning outcomes. Then, we engineered and extracted features from the feedback content on the first assignment using a learner-centred feedback framework and further examined the differences of these features between different groups of student learning outcomes. Lastly, we used the features to predict student learning outcomes by using widely-used machine learning models and provided the interpretation of predicted results by using the SHapley Additive exPlanations (SHAP) framework. We found that 1) most features from the feedback content presented significant differences between the groups of student learning outcomes, 2) the gradient boost tree model could effectively predict student learning outcomes, and 3) SHAP could transparently interpret the feature importance on predictions.

There is a proliferation of learning analytics (LA) dashboards aimed at supporting teachers. Yet, teachers still find it challenging to make sense of LA dashboards, thereby making informed decisions. Two main strategies to address this are emerging: i) upskilling teachers’ data literacy; ii) improving the explanatory design features of current dashboards (e.g., adding visual cues or text) to minimise the skills required by teachers to effectively use dashboards. While each approach has its own trade-offs, no previous work has explored the interplay between the dashboard design and such "data skills". In this paper, we explore how teachers with varying visualisation literacy (VL) skills use LA dashboards enhanced with (explanatory) data storytelling elements. We conducted a quasi-experimental study with 23 teachers of varied VL inspecting two versions of an authentic multichannel dashboard enhanced with data storytelling elements. We used an eye-tracking device while teachers inspected the students’ data captured from Zoom and Google Docs, followed by interviews. Results suggest that high VL teachers adopted complex exploratory strategies and were more sensitive to subtle inconsistencies in the design; while low VL teachers benefited the most from more explicit data storytelling guidance such as accompanying complex graphs with narrative and semantic colour encoding.

Learning analytics (LA) employs educational data to improve the timeliness of support for learners. Apart from technical aspects, there is a need to understand social complexities brought about by different stakeholders, so as to systematise the adoption of LA in Higher Education (HE). We present an analysis of the situation, needs and challenges of LA in the context of Spanish HE, considering managers’, teachers’ and students’ perspectives. Qualitative research is employed using recursive abstraction. Results reveal that the level of institutional adoption is low and none of the analysed institutions had an LA policy. Furthermore, only two of these institutions had an initial LA strategy. While the institutions shared some commonalities in their objectives for LA, chosen tools and adoption challenges, the distinct differences in the political contexts and institutional practices among the institutions reaffirmed that LA solutions and services cannot be implemented in the same manner. Moreover, different needs for LA and concerns are identified about its adoption among managers, students and teachers. These observations lead to our conclusion that the main challenges to implement LA in Spain are not related to technological issues but to the social and cultural issues rooted in institutions and those associated with different stakeholders.

One of the ultimate goals of several learning analytics (LA) initiatives is to close the loop and support students’ and teachers’ reflective practices. Although there has been a proliferation of end-user interfaces (often in the form of dashboards), various limitations have already been identified in the literature such as key stakeholders not being involved in their design, little or no account for sense-making needs, and unclear effects on teaching and learning. There has been a recent call for human-centred design practices to create LA interfaces in close collaboration with educational stakeholders to consider the learning design, and their authentic needs and pedagogical intentions. This paper addresses the call by proposing a question-driven LA design approach to ensure that end-user LA interfaces explicitly address teachers’ questions. We illustrate the approach in the context of synchronous online activities, orchestrated by pairs of teachers using audio-visual and text-based tools (namely Zoom and Google Docs). This study led to the design and deployment of an open-source monitoring tool to be used in real-time by teachers when students work collaboratively in breakout rooms, and across learning spaces.

Academic analytics focuses on collecting, analysing and visualising educational data to generate institutional insights and improve decision-making for academic purposes. However, challenges that arise from navigating a complex organisational structure when introducing analytics systems have called for the need to engage key stakeholders widely to cultivate a shared vision and ensure that implemented systems create desired value. This paper presents a study that takes co-design steps to identify design needs and strategic approaches for the adoption of academic analytics, which serves the purpose of enhancing the measurement of educational quality utilising institutional data. Through semi-structured interviews with 54 educational stakeholders at a large research university, we identified particular interest in measuring student engagement and the performance of courses and programmes. Based on the observed perceptions and concerns regarding data use to measure or evaluate these areas, implications for adoption strategy of academic analytics, such as leadership involvement, communication, and training, are discussed.

Feedback plays a crucial role in student learning. Learning analytics (LA) has demonstrated potential in addressing prominent challenges with feedback practice, such as enabling timely feedback based on insights obtained from large data sets. However, there is insufficient research looking into relations between student expectations of feedback and their experience with LA-based feedback. This paper presents a pilot study that examined students’ experience of LA-based feedback, offered with the OnTask system, taking into consideration the factors of students’self-efficacy and self-regulation skills. Two surveys were carried out at a Brazilian university, and the results highlighted important implications for LA-based feedback practice, including leveraging the ‘partnership’ between the human teacher and the computer, and developing feedback literacy among learners.

Several tools to support learning processes based on educational data have emerged from research on Learning Analytics (LA) in the last few years. These tools aim to support students and instructors in daily activities, and academic managers in making institutional decisions. Although the adoption of LA tools is spreading, the field still needs to deepen the understanding of the contexts where learning takes place, and of the views of the stakeholders involved in implementing and using these tools. In this sense, the SHEILA framework proposes a set of instruments to perform a detailed analysis of the expectations and needs of different stakeholders in higher education institutions, regarding the adoption of LA. Moreover, there is a lacuna in research on stakeholders' expectations from LA outside the Global North. Therefore, this paper reports on the findings of the application of interviews and focus groups, based on the SHEILA framework, with students and teaching staff from a Brazilian public university, to investigate their perceptions of the potential benefits and risks of using LA in higher education in the country. Findings indicate that there is a high interest in using LA for improving the learning experience, in particular, being able to provide personalized feedback, to adapt teaching practices to students' needs, and to make evidence-based pedagogical decisions. From the analysis of these perspectives, we point to opportunities for using LA in Brazilian higher education.

Learning analytics promises to support adaptive learning in higher education. However, the associated issues around privacy protection, especially their implications for students as data subjects, has been a hurdle to wide-scale adoption. In light of this, we set out to understand student expectations of privacy issues related to learning analytics and to identify gaps between what students desire and what they expect to happen or choose to do in reality when it comes to privacy protection. To this end, an investigation was carried out in a UK higher education institution using a survey (N=674) and six focus groups (26 students). The study highlight a number of key implications for learning analytics research and practice: (1) purpose, access, and anonymity are key benchmarks of ethics and privacy integrity; (2) transparency and communication are key levers for learning analytics adoption; and (3) information asymmetry can impede active participation of students in learning analytics.

Learning design research has predominately relied upon survey- and interview-based methodologies, both of which are subject to limitations of social desirability and recall. An alternative approach is offered in this manuscript, whereby physical and online learning activity data is analysed using Epistemic Network Analysis. Using a sample of 6,040 course offerings from 10 faculties across a four year period (2016--2019), the utility of networks to understand learning design is illustrated. Specifically, through the adoption of a network analytic approach, the following was found: universities are clearly committed to blended learning, but there are considerable differences both between and within disciplines.

Interest in learning analytics (LA) has grown rapidly among higher education institutions (HEIs). However, the maturity levels of HEIs in terms of being ‘student data-informed’ are only at early stages. There often are barriers that prevent data from being used systematically and effectively. To assist higher education institutions to become more mature users and custodians of digital data collected from students during their online learning activities, the SHEILA framework, a policy development framework that supports systematic, sustainable and responsible adoption of LA at an institutional level, was recently built. This paper presents a mix-method study using a group concept mapping (GCM) approach that was conducted with LA experts to explore essential features of LA policy in HEI in contribution the development of the framework. The study identified six clusters of features that an LA policy should include, provided ratings based on ease of implementation and importance for each of the six themes, and offered suggestions to HEIs how they can proceed with the development of LA policies.

This paper introduces a learning analytics policy development framework developed by a cross-European research project team - SHEILA (Supporting Higher Education to Integrate Learning Analytics), based on interviews with 78 senior managers from 51 European higher education institutions across 16 countries. The framework was developed using the RAPID Outcome Mapping Approach (ROMA), which is designed to develop effective strategies and evidence-based policy in complex environments. This paper presents three case studies to illustrate the development process of the SHEILA policy framework, which can be used to inform strategic planning and policy processes in real world environments, particularly for large-scale implementation in higher education contexts.

This paper presents the results of a review of eight policies for learning analytics of relevance for higher education, and discusses how these policies have tried to address prominent challenges in the adoption of learning analytics, as identified in the literature. The results show that more considerations need to be given to establishing communication channels among stakeholders and adopting pedagogy-based approaches to learning analytics. It also reveals the shortage of guidance for developing data literacy among end-users and evaluating the progress and impact of learning analytics. Moreover, the review highlights the need to establish formalised guidelines to monitor the soundness, effectiveness, and legitimacy of learning analytics. As interest in learning analytics among higher education institutions continues to grow, this review will provide insights into policy and strategic planning for the adoption of learning analytics.

This workshop aims to promote strategic planning for learning analytics in higher education through developing institutional policies. While adoption of learning analytics is predominantly seen in small-scale and bottom-up patterns, it is believed that a systemic implementation can bring the widest impact to the education system and lasting benefits to learners. However, the success of it highly depends on the adopted strategy that meets the needs of various stakeholders and systematically pushes the institution towards achieving its targets. It is imperative to develop a learning analytics policy that ensures a practice that is valid, effective and ethical.