Since 2013, the Learning Analytics and Knowledge (LAK) conference has included a Doctoral Consortium (DC), supported by the Society for Learning Analytics Research (SoLAR). Given the LAK25 conference theme of ‘expanding the horizons of learning analytics’, it is timely to reflect on how well the LAK DC is meeting its objectives of building capacity in the field and developing the next generation. We frame the DC as a structured entry into the LAK community of practice (CoP), familiarising students with the domain of learning analytics, understanding its practices, and building connections with other members. CoPs generate five types of value for their members: immediate, potential, applied, realised and reframing. This study used a survey of the 92 DC students from the first decade (2013–22), supplemented with scientometric analysis of LAK publications, to address the questions: What value do students gain from attending the LAK doctoral consortium? and Do students gain the same value from face-to-face and virtual doctoral consortia? Thematic analysis of responses (N=37, a 40% response rate) showed that students gained a wide range of immediate and potential value from the DC, which in many cases also prompted changes in practice, performance improvement or redefinition of success. However, the value reported by a third of respondents who had attended virtually was more limited. We note that, despite the value already offered, the DC could provide clearer routes both to future community engagement and to extending goals that expand the horizons of learning analytics. This paper's contributions are (i) the first systematic documentation of student perceptions of LAK DCs, (ii) identification of ways in which doctoral consortia can be developed in the future, and (iii) specific attention to how virtual DCs can offer greater value for both participants and the host community of practice. The findings related to future development and value do not apply only to LAK but can be generalised to other doctoral consortia.

With the emergence of generative AI, the field of Learning Analytics (LA) has increasingly embraced the use of Large Language Models (LLMs) to automate qualitative analysis. Deductive analysis requires theoretical or other conceptual grounding to inform coding. However, few studies detail the process of translating the literature into a codebook, and then into an effective LLM prompt. In this paper, we introduce Grounded Prompt Engineering (GROPROE) as a systematic process to develop a literature-grounded prompt for deductive analysis. We demonstrate our GROPROE process on a dataset of 860 written reflections, coding for students’ affective engagement and sense of belonging. To evaluate the quality of the coding we demonstrate substantial human/LLM Inter-Annotator Reliability (IAR). To evaluate the consistency of LLM coding, a subset of the data was analysed 60 times using the LLM Quotient showing how this stabilized for most codes. We discuss the dynamics of human-AI interaction when following GROPROE, foregrounding how the prompt took over as the iteratively revised codebook, and how the LLM provoked codebook revision. The contributions to the LA field are threefold: (i) GROPROE as a systematic prompt-design process for deductive coding grounded in literature, (ii) a detailed worked example showing its application to Belonging Analytics, and (iii) implications for human-AI interaction in automated deductive analysis.

Queer young men (similar to others in the LGBTQ+ community) depend heavily on social platforms but their use can often be problematic. Their needs are often not adequately considered in the design of general platforms and they can be exposed to intra-community harms on LGBTQ+ specific platforms such as dating apps. To explore how social platform design could be improved to better support the needs of queer young men, we conducted a co-design study. We recruited 13 queer men working in technology design to generate new concepts for social platform features. We then refined these concepts and evaluated them in group sessions with end users, a different cohort of 15 queer young men. Here we present mockups of the concepts and findings from evaluations. Our findings show specific ways that providing more agency to social platform users could improve their experiences and we discuss implications for design.

In our era of rapid technological advancement, the research landscape for writing assistants has become increasingly fragmented across various research communities. We seek to address this challenge by proposing a design space as a structured way to examine and explore the multidimensional space of intelligent and interactive writing assistants. Through community collaboration, we explore five aspects of writing assistants: task, user, technology, interaction, and ecosystem. Within each aspect, we define dimensions and codes by systematically reviewing 115 papers, while leveraging the expertise of researchers in various disciplines. Our design space aims to offer researchers and designers a practical tool to navigate, comprehend, and compare the various possibilities of writing assistants, and aid in the design of new writing assistants.

Advancements in Multimodal Learning Analytics (MMLA) have the potential to enhance the development of effective teamwork skills and foster reflection on collaboration dynamics in physical learning environments. Yet, only a few MMLA studies have closed the learning analytics loop by making MMLA solutions immediately accessible to educators to support reflective practices, especially in authentic settings. Moreover, deploying MMLA solutions in authentic settings can bring new challenges beyond logistic and privacy issues. This paper reports the design and use of TeamSlides, a multimodal teamwork analytics dashboard to support teacher-guided reflection. We conducted an in-the-wild classroom study involving 11 teachers and 138 students. Multimodal data were collected from students working in team healthcare simulations. We examined how teachers used the dashboard in 22 debrief sessions to aid their reflective practices. We also interviewed teachers to discuss their perceptions of the dashboard’s value and the challenges faced during its use. Our results suggest that the dashboard effectively reinforced discussions and augmented teacher-guided reflection practices. However, teachers encountered interpretation conflicts, sometimes leading to mistrust or misrepresenting the information. We discuss the considerations needed to overcome these challenges in MMLA research.

Dashboards are increasingly used in education to provide teachers and students with insights into learning. Yet, existing dashboards are often criticised for their failure to provide the contextual information or explanations necessary to help students interpret these data. Data Storytelling (DS) is emerging as an alternative way to communicate insights providing guidance and context to facilitate students’ interpretations. However, while data stories have proven effective in prompting students’ reflections, to date, it has been necessary for researchers to craft the stories rather than enabling teachers to do this by themselves. This can make this approach more feasible and scalable while also respecting teachers’ agency. Based on the notion of DS, this paper presents a DS editor for teachers. A study was conducted in two universities to examine whether the editor could enable teachers to create stories adapted to their learning designs. Results showed that teachers appreciated how the tool enabled them to contextualise automated feedback to their teaching needs, generating data stories to support student reflection.

A student's “sense of belonging” is critical to retention and success in higher education. However, belonging is a multifaceted and dynamic concept, making monitoring and supporting it with timely action challenging. Conventional approaches to researching belonging depend on lengthy surveys and/or focus groups, and while often insightful, these are resource-intensive, slow, and cannot be repeated too often. “Belonging Analytics” is an emerging concept pointing to the potential of learning analytics to address this challenge, and to illustrate this concept, this paper investigates the feasibility of asking students a single question about what promotes their sense of belonging. To validate this, responses were analysed using a form of topic modelling, and these were triangulated by examining alignment with (i) students’ responses to Likert scale items in a belonging scale and (ii) the literature on the drivers of belonging. These alignments support our proposal that this is a practical tool to gain timely insight into a cohort's sense of belonging. Reflecting our focus on practical tools, the approach is implemented using analytics products readily available to educational institutions — Linguistic Inquiry Word Count (LIWC) and Statistical Program for Social Sciences (SPSS).

Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations “in-the-wild”. These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers’ tasks. These practicalities have been rarely investigated. This article addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators in the context of nursing education. The lessons learnt were synthesised into topics related to (i) technological/physical aspects of the deployment; (ii) multimodal data and interfaces; (iii) the design process; (iv) participation, ethics and privacy; and (v) sustainability of the deployment.

It is not overstating matters to say that humanity finds itself at an inflection point. The interlocking crises can feel overwhelming (ecological; political; financial; medical; technological; educational...), with recent leaps in AI closing the gap between human and machine cognition, raising many issues, including of course, educational questions. If a plausible diagnosis for our predicament as a species is "failure to learn", praxis questions assail us. What qualities should we cultivate most urgently, in which contexts? What literacies equip teachers and learners to engage critically with AI? How do we track progress meaningfully, at scale? And very pragmatically, when and why do people deem our tools trustworthy enough to trial, and if robust, embed sustainably into their teaching and learning practices?

Despite the digital revolution, physical space remains the site for teaching and learning embodied knowledge and skills. Both teachers and students must develop spatial competencies to effectively use classroom spaces, enabling fluid verbal and non-verbal interaction. While video permits rich activity capture, it provides no support for quickly seeing activity patterns that can assist learning. In contrast, position tracking systems permit the automated modelling of spatial behaviour, opening new possibilities for feedback. This paper introduces the design rationale for ”Dandelion Diagrams” that integrate participant location, trajectory and body orientation over a variable period. Applied in two authentic teaching contexts (a science laboratory, and a nursing simulation) we show how heatmaps showing only teacher/student location led to misinterpretations that were resolved by overlaying Dandelion Diagrams. Teachers also identified a variety of ways they could aid professional development. We conclude Dandelion Diagrams assisted sensemaking, but discuss the ethical risks of over-interpretation.

Learning Analytics (LA) is a bricolage field that requires a concerted effort to ensure that all stakeholders it affects are able to contribute to its development in a meaningful manner. We need mechanisms that support collaborative sense-making. This paper argues that graphical causal models can help us to span the disciplinary divide, providing a new apparatus to help educators understand, and potentially challenge, the technical models developed by LA practitioners as they form. We briefly introduce causal modelling, highlighting its potential benefits in helping the field to move from associations to causal claims, and illustrate how graphical causal models can help us to reason about complex statistical models. The approach is illustrated by applying it to the well known problem of at-risk modelling.

Learning Analytics (LA) dashboards have become a popular medium for communicating to teachers analytical insights obtained from student data. However, recent research indicates that LA dashboards can be complex to interpret, are often not grounded in educational theory, and frequently provide little or no guidance on how to interpret them. Despite these acknowledged problems, few suggestions have been made as to how we might improve the visual design of LA tools to support richer and alternative ways to communicate student data insights. In this paper, we explore three design alternatives to represent student multimodal data insights by combining data visualisation, narratives and storytelling principles. Based on foundations in data storytelling, three visual-narrative interfaces were designed with teachers: i) visual data slices, ii) a tabular visualisation, and iii) a written report. These were validated as a part of an authentic study where teachers explored activity logs and physiological data from co-located collaborative learning classes in the context of healthcare education. Results suggest that alternatives to LA dashboards can be considered as effective tools to support teachers’ reflection, and that LA designers should identify the representation type that best fits teachers’ needs.

research-article Share on Questioning learning analytics? Cultivating critical engagement as student automated feedback literacy Authors: Antonette Shibani University of Technology, Sydney, Australia University of Technology, Sydney, AustraliaView Profile , Simon Knight University of Technology, Sydney, Australia University of Technology, Sydney, AustraliaView Profile , Simon Buckingham Shum University of Technology, Sydney, Australia University of Technology, Sydney, AustraliaView Profile Authors Info & Claims LAK22: LAK22: 12th International Learning Analytics and Knowledge ConferenceMarch 2022 Pages 326–335https://doi.org/10.1145/3506860.3506912Online:21 March 2022Publication History 0citation152DownloadsMetricsTotal Citations0Total Downloads152Last 12 Months152Last 6 weeks26 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Teachers’ spatial behaviours in the classroom can strongly influence students’ engagement, motivation and other behaviours that shape their learning. However, classroom teaching behav-iour is ephemeral, and has largely remained opaque to computational analysis. This paper presents a library called ‘Moodoo’ that can serve to automatically model how teachers make use of the classroom space by analysing indoor positioning traces. The system automatically ex-tracts spatial metrics (e.g. teacher-student ratios, frequency of visits to students’ personal spaces, presence in classroom spaces of interest, index of dispersion and entropy), mapping from the teachers’ low-level positioning data to higher-order spatial constructs.

In nursing education through team simulations, students must learn to position themselves correctly in coordination with colleagues. However, with multiple student teams in action, it is difficult for teachers to give detailed, timely feedback on these spatial behaviours to each team. Indoor-positioning technologies can now capture student spatial behaviours, but relatively little work has focused on giving meaning to student activity traces, transforming low-level x/y coordinates into language that makes sense to teachers. Even less research has investigated if teachers can make sense of that feedback. This paper therefore makes two contributions. (1) Methodologically, we document the use of Epistemic Network Analysis (ENA) as an approach to model and visualise students’ movements. To our knowledge, this is the first application of ENA to analyse human movement. (2) We evaluated teachers’ responses to ENA diagrams through qualitative analysis of video-recorded sessions. Teachers constructed consistent narratives about ENA diagrams’ meaning, and valued the new insights ENA offered. However, ENA’s abstract visualisation of spatial behaviours was not intuitive, and caused some confusions. We propose, therefore, that the power of ENA modelling can be combined with other spatial representations such as a classroom map, by overlaying annotations to create a more intuitive user experience.

Effective teamwork is critical to improve patient outcomes in healthcare. However, achieving this capabilityrequires that pre-service nurses develop the spatial abilities they will require in their clinical placements, suchas: learning when to remain close to the patient and to other team members; positioning themselves correctlyat the right time; and deciding on specific team formations (e.g. face-to-face or side-by-side) to enable effectiveinteraction or avoid disrupting clinical procedures. However, positioning dynamics are ephemeral and caneasily become occluded by the multiple tasks nurses have to accomplish. Digital traces automatically capturedby indoor positioning sensors can be used to address this problem for the purpose of improving nurses' reflection, learning and professional development. This paper presents; i) a qualitative study that illustrateshow to elicit spatial behaviours from educators' pedagogical expectations, and ii) a modelling approachthat transforms nurses' low-level position traces into higher-order proxemics constructs, informed by sucheducatos' expectations, in the context of simulation-based teamwork training. To illustrate our modellingapproach, we conducted an in-the-wild study with 55 undergraduate students and five educators from whompositioning traces were captured in eleven authentic nursing education classes. Low-levelx-ydata was usedto model three proxemic constructs: i) co-presence in interactional spaces, ii) socio-spatial formations (i.e.f-formations), and ii) presence in spaces of interest. Through a number of vignettes, we illustrate how indoorpositioning analytics can be used to address questions that educators and researchers have about teamwork inhealthcare simulation settings.

Significant progress to integrate and analyse multimodal data has been carried out in the last years. Yet, little research has tackled the challenge of visualising and supporting the sensemaking of multimodal data to inform teaching and learning. It is naïve to expect that simply by rendering multiple data streams visually, a teacher or learner will be able to make sense of them. This paper introduces an approach to unravel the complexity of multimodal data by organising it into meaningful layers that explain critical insights to teachers and students. The approach is illustrated through the design of two data storytelling prototypes in the context of nursing simulation. Two authentic studies with educators and students identified the potential of the approach to create learning analytics interfaces that communicate insights on team performance, as well as concerns in terms of accountability and automated insights discovery.

In an era that will increasingly depend upon lifelong learning, the LA community will need to facilitate the movement and sharing of data and information across institutional and geographic boundaries. This will help us to recognise prior learning (RPL) and to personalise the learner experience. Here, we explore the utility of skills-based curriculum analytics and how it might facilitate the process of awarding RPL between two institutions. We explore the potential utility of combining natural language processing and skills taxonomies to map between subject descriptions for these two different institutions, presenting two algorithms we have developed to facilitate RPL and evaluating their performance. We draw attention to some of the issues that arise, listing areas that we consider ripe for future work in a surprisingly underexplored area.

Human-centred software design gives all stakeholders an active voice in the design of the systems that they are expected to use. However, this is not yet commonplace in Learning Analytics (LA). Co-design techniques from other domains therefore have much to offer to LA, in principle, but there are few detailed accounts of exactly how such sessions unfold. This paper presents the rationale driving a card-based co-design tool specifically tuned for LA, called LA-DECK. In the context of a pilot study with students, educators, LA researchers and developers, we provide qualitative and quantitative accounts of how participants used the cards. Using three different forms of analysis (transcript-centric design vignettes, card-graphs and time-on-topic), we characterise in what ways the sessions were "participatory" in nature, and argue that the cards succeeded in playing very similar roles to those documented in the literature on successful card-based design tools.

Collocated, face-to-face teamwork remains a pervasive mode of working, which is hard to replicate online. Team members' embodied, multimodal interaction with each other and artefacts has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. However, the ready availability of sensors makes it increasingly affordable to instrument work spaces to study teamwork and groupwork. The possibility of visualising key aspects of a collaboration has huge potential for both academic and professional learning, but a frontline challenge is the enrichment of quantitative data streams with the qualitative insights needed to make sense of them. In response, we introduce the concept of collaboration translucence, an approach to make visible selected features of group activity. This is grounded both theoretically (in the physical, epistemic, social and affective dimensions of group activity), and contextually (using domain-specific concepts). We illustrate the approach from the automated analysis of healthcare simulations to train nurses, generating four visual proxies that fuse multimodal data into higher order patterns.

A major promise of learning analytics is that through the collection of large amounts of data we can derive insights from authentic learning environments, and impact many learners at scale. However, the context in which the learning occurs is important for educational innovations to impact student learning. In particular, for student-facing learning analytics systems like feedback tools to work effectively, they have to be integrated with pedagogical approaches and the learning design. This paper proposes a conceptual model to strike a balance between the concepts of generalizable scalable support and contextualized specific support by clarifying key elements that help to contextualize student-facing learning analytics tools. We demonstrate an implementation of the model using a writing analytics example, where the features, feedback and learning activities around the automated writing feedback tool are tuned for the pedagogical context and the assessment regime in hand, by co-designing them with the subject experts. The model can be employed for learning analytics to move from generalized support to meaningful contextualized support for enhancing learning.

Learning to collaborate effectively requires practice, awareness of group dynamics, and reflection; often it benefits from coaching by an expert facilitator. However, in physical spaces it is not always easy to provide teams with evidence to support collaboration. Emerging technology provides a promising opportunity to make collocated collaboration visible by harnessing data about interactions and then mining and visualizing it. These collocated collaboration analytics can help researchers, designers, and users to understand the complexity of collaboration and to find ways they can support collaboration. This article introduces and motivates a set of principles for mining collocated collaboration data and draws attention to trade-offs that may need to be negotiated en route. We integrate Data Science principles and techniques with the advances in interactive surface devices and sensing technologies. We draw on a 7-year research program that has involved the analysis of six group situations in collocated settings with more than 500 users and a variety of surface technologies, tasks, grouping structures, and domains. The contribution of the article includes the key insights and themes that we have identified and summarized in a set of principles and dilemmas that can inform design of future collocated collaboration analytics innovations.

Data science is now impacting the education sector, with a growing number of commercial products and research prototypes providing learning dashboards. From a human-centred computing perspective, the end-user's interpretation of these visualisations is a critical challenge to design for, with empirical evidence already showing that `usable' visualisations are not necessarily effective from a learning perspective. Since an educator's interpretation of visualised data is essentially the construction of a narrative about student progress, we draw on the growing body of work on Data Storytelling (DS) as the inspiration for a set of enhancements that could be applied to data visualisations to improve their communicative power. We present a pilot study that explores the effectiveness of these DS elements based on educators' responses to paper prototypes. The dual purpose is understanding the contribution of each visual element for data storytelling, and the effectiveness of the enhancements when combined.

Learning Analytics (LA) sits at the confluence of many contributing disciplines, which brings the risk of hidden assumptions inherited from those fields. Here, we consider a hidden assumption derived from computer science, namely, that improving computational accuracy in classification is always a worthy goal. We demonstrate that this assumption is unlikely to hold in some important educational contexts, and argue that embracing computational "imperfection" can improve outcomes for those scenarios. Specifically, we show that learner-facing approaches aimed at "learning how to learn" require more holistic validation strategies. We consider what information must be provided in order to reasonably evaluate algorithmic tools in LA, to facilitate transparency and realistic performance comparisons.

A large number of learning tools offering some sort of personalisation features rely mainly on the analysis of logged interactions between students and particular user interfaces. Much less attention has been given to the analysis of physical aspects so often present in 'traditional' intellectual tasks, although these are both important in the full development of a life-long learner. This paper (1) discusses existing literature focused on supporting learning using proximity and location analytics and sensors, and, based on this, (2) illustrates the feasibility and potential of these analytics for teaching and learning through an study in the context of proximity and location analytics in a team-based health simulation classroom.

Reflective writing can provide a powerful way for students to integrate professional experience and academic learning. However, writing reflectively requires high quality actionable feedback, which is time-consuming to provide at scale. This paper reports progress on the design, implementation, and validation of a Reflective Writing Analytics platform to provide actionable feedback within a tertiary authentic assessment context. The contributions are: (1) a new conceptual framework for reflective writing; (2) a computational approach to modelling reflective writing, deriving analytics, and providing feedback; (3) the pedagogical and user experience rationale for platform design decisions; and (4) a pilot in a student learning context, with preliminary data on educator and student acceptance, and the extent to which we can evidence that the software provided actionable feedback for reflective writing.

There is untapped potential in achieving the full impact of learning analytics through the integration of tools into practical pedagogic contexts. To meet this potential, more work must be conducted to support educators in developing learning analytics literacy. The proposed workshop addresses this need by building capacity in the learning analytics community and developing an approach to resourcing for building 'writing analytics literacy'.

There is an increasing interest in the analysis of both student's writing and the temporal aspects of learning data. The analysis of higher-level learning features in writing contexts requires analyses of data that could be characterised in terms of the sequences and processes of textual features present. This paper (1) discusses the extant literature on sequential and process analyses of writing; and, based on this and our own first-hand experience on sequential analysis, (2) proposes a number of approaches to both pre-process and analyse sequences in whole-texts. We illustrate how the approaches could be applied to examples drawn from our own datasets of 'rhetorical moves' in written texts, and the potential each approach holds for providing insight into that data. Work is in progress to apply this model to provide empirical insights. Although, similar sequence or process mining techniques have not yet been applied to student writing, techniques applied to event data could readily be operationalised to undercover patterns in texts.

Healthcare simulations are hands-on learning experiences aimed at allowing students to practice essential skills that they may need when working with real patients in clinical workplaces. Some clinical classrooms are equipped with patient manikins that can respond to actions or that can be programmed to deteriorate over time. Students can perform assessments and interventions, and enhance their critical thinking and communication skills. There is an opportunity to exploit the students' digital traces that these manikins can pervasively capture to make key aspects of the learning process visible. The setting can be augmented with sensors to capture traces of group interaction. These multimodal data can be used to generate visualisations or feedback for students or teachers. This paper reports on an authentic classroom study using analytics to integrate multimodal data of students' interactions with the manikins and their peers in simulation scenarios. We report on the challenges encountered in deploying such analytics 'in the wild', using an analysis framework that considers the social, epistemic and physical dimensions of collocated group activity.

The amount of data extracted from learning experiences has grown at an astonishing pace both in depth due to the increasing variety of data sources, and in breath with courses now being offered to massive student cohorts. However, in this emerging scenario instructors are now facing the challenge of connecting the knowledge emerging from data analysis with the provision of meaningful support actions to students within the context of an instructional design.

For students, in particular freshmen, the degree pathway from semester to semester is not that transparent, although students have a reasonable idea what courses are expected to be taken each semester. An often-pondered question by students is: "what can I expect in the next semester?" More precisely, given the commitment and engagement I presented in this particular course and the respective performance I achieved, can I expect a similar outcome in the next semester in the particular course I selected? Are the demands and expectations in this course much higher so that I need to adjust my commitment and engagement and overall workload if I expect a similar outcome? Is it better to drop a course to manage expectations rather than to (predictably) fail, and perhaps have to leave the degree altogether? Degree and course advisors and student support units find it challenging to provide evidence based advise to students. This paper presents research into educational process mining and student data analytics in a whole university scale approach with the aim of providing insight into the degree pathway questions raised above. The beta-version of our course level degree pathway tool has been used to shed light for university staff and students alike into our university's 1,300 degrees and associated 6 million course enrolments over the past 20 years.

In many situations, it remains critical for team members to develop strategies to effectively use the space and tools available to complete demanding tasks. However, despite the availability of sensors and analytics for instrumenting physical space, relatively little progress has been made in modelling the embodied dimensions of co-located teamwork. This paper explores an in-the-wild pilot study through which we explore a methodology to model embodied mobility teamwork strategies in the context of healthcare education. We developed the means for tracking, clustering and processing student-nurses' mobility data around a patient manikin. We illustrate the feasibility of our approach by discussing ways to make sense of these data to uncover meaningful trends, and the inherent challenges of applying physical space analytics in authentic settings.

This demonstration introduces the Connected Learning Analytics (CLA) Toolkit. The CLA toolkit harvests data about student participation in specified learning activities across standard social media environments, and presents information about the nature and quality of the learning interactions.

The proliferation of varied types of multi-user interactive surfaces (such as digital whiteboards, tabletops and tangible interfaces) is opening a new range of applications in face-to-face (f2f) contexts. They offer unique opportunities for Learning Analytics (LA) by facilitating multi-user sensemaking of automatically captured digital footprints of students' f2f interactions. This paper presents an analysis of current research exploring learning analytics associated with the use of surface devices. We use a framework to analyse our first-hand experiences, and the small number of related deployments according to four dimensions: the orchestration aspects involved; the phases of the pedagogical practice that are supported; the target actors; and the levels of iteration of the LA process. The contribution of the paper is twofold: 1) a synthesis of conclusions that identify the degree of maturity, challenges and pedagogical opportunities of the existing applications of learning analytics and interactive surfaces; and 2) an analysis framework that can be used to characterise the design space of similar areas and LA applications.

Writing Analytics focuses on the measurement and analysis of written texts for the purpose of understanding writing processes and products, in their educational contexts, and improving the teaching and learning of writing. This workshop adopts a critical, holistic perspective in which the definition of "the system" and "success" is not restricted to IR metrics such as precision and recall, but recognizes the many wider issues that aid or obstruct analytics adoption in educational settings, such as theoretical and pedagogical grounding, usability, user experience, stakeholder design engagement, practitioner development, organizational infrastructure, policy and ethics.

When used effectively, reflective writing tasks can deepen learners' understanding of key concepts, help them critically appraise their developing professional identity, and build qualities for lifelong learning. As such, reflecting writing is attracting substantial interest from universities concerned with experiential learning, reflective practice, and developing a holistic conception of the learner. However, reflective writing is for many students a novel genre to compose in, and tutors may be inexperienced in its assessment. While these conditions set a challenging context for automated solutions, natural language processing may also help address the challenge of providing real time, formative feedback on draft writing. This paper reports progress in designing a writing analytics application, detailing the methodology by which informally expressed rubrics are modelled as formal rhetorical patterns, a capability delivered by a novel web application. This has been through iterative evaluation on an independently human-annotated corpus, showing improvements from the first to second version. We conclude by discussing the reasons why classifying reflective writing has proven complex, and reflect on the design processes enabling work across disciplinary boundaries to develop the prototype to its current state.

C ollaborative C omputer‐ S upported A rgument V isualization ( CCSAV ) has often been proposed as an alternative over more conventional, mainstream platforms for online discussion (e.g., online forums and wikis). CCSAV tools require users to contribute to the creation of a joint artifact (argument map) instead of contributing to a conversation. In this paper we assess empirically the effects of this fundamental design choice and show that the absence of conversational affordances and socially salient information in representation‐centric tools is detrimental to the users' collaboration experience. We report empirical findings from a study in which subjects using different collaborative platforms (a forum, an argumentation platform, and a socially augmented argumentation tool) were asked to discuss and predict the price of a commodity. By comparing users' experience across several metrics we found evidence that the collaborative performance decreases gradually when we remove conversational interaction and other types of socially salient information. We interpret these findings through theories developed in conversational analysis (common ground theory) and communities of practice and discuss design implications. In particular, we propose balancing the trade‐off between knowledge reification and participation in representation‐centric tools with the provision of social feedback and functionalities supporting meaning negotiation.

When assessing student essays, educators look for the students' ability to present and pursue well-reasoned and strong arguments. Such scholarly argumentation is often articulated by rhetorical metadiscourse. Educators will be necessarily examining metadiscourse in students' writing as signals of the intellectual moves that make their reasoning visible. Therefore students and educators could benefit from available powerful automated textual analysis that is able to detect rhetorical metadiscourse. However, there is a need to validate such technologies in higher education contexts, since they were originally developed in non-educational applications. This paper describes an evaluation study of a particular language analysis tool, the Xerox Incremental Parser (XIP), on undergraduate social science student essays, using the mark awarded as a measure of the quality of the writing. As part of this exploration, the study presented in this paper seeks to assess the quality of the XIP through correlational studies and multiple regression analysis.

The first international workshop on discourse-centric learning analytics (DCLA) took place at LAK13 in Leuven, Belgium. That workshop succeeded in its aim of catalysing ideas and building community connections between those working in this field of social learning analytics. It also proposed a mission statement for DCLA: to devise and validate analytics that look beyond surface measures in order to quantify linguistic proxies for deeper learning. This year, the focus of the second international DCLA workshop, like that of LAK14, is on the intersection of learning analytics research, theory and practice. Once researchers have developed and validated discourse-centric analytics, how can these be successfully deployed at scale to support learning?

This paper describes a novel analytics dashboard which visualises the key features of scholarly documents. The Dashboard aggregates the salient sentences of scholarly papers, their rhetorical types and the key concepts mentioned within these sentences. These features are extracted from papers through a Natural Language Processing (NLP) technology, called Xerox Incremental Parser (XIP). The XIP Dashboard is a set of visual analytics modules based on the XIP output. In this paper, we briefly introduce the XIP technology and demonstrate an example visualisation of the XIP Dashboard.

Social learning analytics are concerned with the process of knowledge construction as learners build knowledge together in their social and cultural environments. One of the most important tools employed during this process is language. In this paper we take exploratory dialogue, a joint form of co-reasoning, to be an external indicator that learning is taking place. Using techniques developed within the field of computational linguistics, we build on previous work using cue phrases to identify exploratory dialogue within online discussion. Automatic detection of this type of dialogue is framed as a binary classification task that labels each contribution to an online discussion as exploratory or non-exploratory. We describe the development of a self-training framework that employs discourse features and topical features for classification by integrating both cue-phrase matching and k-nearest neighbour classification. Experiments with a corpus constructed from the archive of a two-day online conference show that our proposed framework outperforms other approaches. A classifier developed using the self-training framework is able to make useful distinctions between the learning dialogue taking place at different times within an online conference as well as between the contributions of individual participants.

There is a well-established literature examining the relationships between epistemology (the nature of knowledge), pedagogy (the nature of learning and teaching), and assessment. Learning Analytics (LA) is a new assessment technology and should engage with this literature since it has implications for when and why different LA tools might be deployed. This paper discusses these issues, relating them to an example construct, epistemic beliefs -- beliefs about the nature of knowledge -- for which analytics grounded in pragmatic, sociocultural theory might be well placed to explore. This example is particularly interesting given the role of epistemic beliefs in the everyday knowledge judgements students make in their information processing. Traditional psychological approaches to measuring epistemic beliefs have parallels with high stakes testing regimes; this paper outlines an alternative LA for epistemic beliefs which might be readily applied to other areas of interest. Such sociocultural approaches afford opportunity for engaging LA directly in high quality pedagogy.

Social Learning Analytics (SLA) are designed to support students learning through social networks, and reflective practitioners engage in informal learning through a community of practice. This short paper reports work in progress to develop SLA motivated specifically by Networked Learning Theory, drawing on the related concepts and tools of Social Network Analytics and Social Capital Theory, which provide complementary perspectives onto the structure and content of such networks. We propose that SLA based on these perspectives needs to devise models and visualizations capable of showing not only the usual SNA metrics, but the types of social tie forged between actors, and topic-specific subnetworks. We describe a technical implementation demonstrating this approach, which extends the Network Awareness Tool by automatically populating it with data from a social learning platform SocialLearn. The result is the ability to visualize relationships between people who interact around the same topics.

The Educational Data Scientist is currently a poorly understood, rarely sighted breed. Reports vary: some are known to be largely nocturnal, solitary creatures, while others have been reported to display highly social behaviour in broad daylight. What are their primary habits? How do they see the world? What ecological niches do they occupy now, and will predicted seismic shifts transform the landscape in their favour? What survival skills do they need when running into other breeds? Will their numbers grow, and how might they evolve? In this panel, the conference will hear and debate not only broad perspectives on the terrain, but will have been exposed to some real life specimens, and caught glimpses of the future ecosystem.

This workshop anticipates that an important class of learning analytic will emerge at the intersection of research into learning dynamics, online discussion platforms, and computational linguistics. Written discourse is arguably the primary class of data that can give us insights into deeper learning and higher order qualities such as critical thinking, argumentation, mastery of complex ideas, empathy, collaboration and interpersonal skills. Moreover, the ability to write in a scholarly manner is a core competence, often taking the form of discourse with oneself and the literature. Computational linguistics research has developed a rich array of tools for machine interpretation of human discourse, but work to develop these tools in the context of learning is at a relatively early stage. Moreover, there is a significant difference between designing tools to assist researchers in discourse analysis, and their deployment on platforms to provide meaningful analytics for the learners and educators who are conducting that discourse. This workshop aims to catalyse ideas and build community connections among those who want to shape this field.

W This panel is proposed as a means of promoting mutual learning and continued dialogue between the Educational Data Mining and Learning Analytics communities. EDM has been developing as a community for longer than the LAK conference, so what if anything makes the LAK community different, and where is the common ground?

This paper proposes that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. The paper introduces the broad rationale for SLA by reviewing some of the key drivers that make social learning so important today. Five forms of SLA are identified, including those which are inherently social, and others which have social dimensions. The paper goes on to describe early work towards implementing these analytics on SocialLearn, an online learning space in use at the UK's Open University, and the challenges that this is raising. This work takes an iterative approach to analytics, encouraging learners to respond to and help to shape not only the analytics but also their associated recommendations.

Theoretical and empirical evidence in the learning sciences substantiates the view that deep engagement in learning is a function of a complex combination of learners' identities, dispositions, values, attitudes and skills. When these are fragile, learners struggle to achieve their potential in conventional assessments, and critically, are not prepared for the novelty and complexity of the challenges they will meet in the workplace, and the many other spheres of life which require personal qualities such as resilience, critical thinking and collaboration skills. To date, the learning analytics research and development communities have not addressed how these complex concepts can be modelled and analysed, and how more traditional social science data analysis can support and be enhanced by learning analytics. We report progress in the design and implementation of learning analytics based on a research validated multidimensional construct termed "learning power". We describe, for the first time, a learning analytics infrastructure for gathering data at scale, managing stakeholder permissions, the range of analytics that it supports from real time summaries to exploratory research, and a particular visual analytic which has been shown to have demonstrable impact on learners. We conclude by summarising the ongoing research and development programme and identifying the challenges of integrating traditional social science research, with learning analytics and modelling.

This paper describes the rationale behind a workshop on using data-intensive computational methods of analysis for empirical-analytical studies of collaborative and networked learning, with a particular focus on how learning takes place in the technically-mediated interplay between individual, small group and collective levels of agency. This workshop is primarily designed for researchers interested in empirical-analytical studies using data-intensive computational methods of analysis (including social-network analysis, log-file analysis, data mining, video analysis).

While generic web analytics tend to focus on easily harvested quantitative data, Learning Analytics will often seek qualitative understanding of the context and meaning of this information. This is critical in the case of dialogue, which may be employed to share knowledge and jointly construct understandings, but which also involves many superficial exchanges. Previous studies have validated a particular pattern of 'exploratory dialogue' in learning environments to signify sharing, challenge, evaluation and careful consideration by participants. This study investigates the use of sociocultural discourse analysis to analyse synchronous text chat during an online conference. Key words and phrases indicative of exploratory dialogue were identified in these exchanges, and peaks of exploratory dialogue were associated with periods set aside for discussion and keynote speakers. Fewer individuals posted at these times, but meaningful discussion outweighed trivial exchanges. If further analysis confirms the validity of these markers as learning analytics, they could be used by recommendation engines to support learners and teachers in locating dialogue exchanges where deeper learning appears to be taking place.

Drawing on sociocultural discourse analysis and argumentation theory, we motivate a focus on learners' discourse as a promising site for identifying patterns of activity which correspond to meaningful learning and knowledge construction. However, software platforms must gain access to qualitative information about the rhetorical dimensions to discourse contributions to enable such analytics. This is difficult to extract from naturally occurring text, but the emergence of more-structured annotation and deliberation platforms for learning makes such information available. Using the Cohere web application as a research vehicle, we present examples of analytics at the level of individual learners and groups, showing conceptual and social network patterns, which we propose as indicators of meaningful learning.

This paper reports progress in realizing human-agent argumentation, which we argue will be part of future Computer-Supported Collaborative Argumentation (CSCA) tools. With a particular interest in argument mapping, we present two investigations demonstrating how a particular agent-oriented language and architecture can augment CSCA: (i) the use of the IBIS formalism enabling Brahms agents to simulate argumentation, and (ii) the extension of the Compendium tool by integrating it with Brahms agents tasked with detecting related discourse elsewhere.

This article characterizes key weaknesses in the ability of current digital libraries to support scholarly inquiry, and as a way to address these, proposes computational services grounded in semiformal models of the naturalistic argumentation commonly found in research literatures. It is argued that a design priority is to balance formal expressiveness with usability, making it critical to coevolve the modeling scheme with appropriate user interfaces for argument construction and analysis. We specify the requirements for an argument modeling scheme for use by untrained researchers and describe the resulting ontology, contrasting it with other domain modeling and semantic web approaches, before discussing passive and intelligent user interfaces designed to support analysts in the construction, navigation, and analysis of scholarly argument structures in a Web-based environment. © 2007 Wiley Periodicals, Inc. Int J Int Syst 22: 17–47, 2007.

Annotating a document with an interpretation of its contents raises a number of challenges that we are hoping to address via the creation of a supporting environment. We present these challenges and motivate an approach based on the notion of suggestions to support document annotation, hoping these suggestions would act as leads to follow for annotators, therefore reducing some of the difficulties inherent to the task. The environment resulting from this approach, ClaimSpotter, is presented. Aspects of its evaluation are also given, using the findings of a study involving a group of participants faced with a document annotation task.

Formative work by Lave and Wenger has articulated how practices emerge through the interplay of informal processes with symbolic codifications and artifacts. In this paper, we describe how ontologies can serve as symbolic tools within a community of practice supporting communication and knowledge sharing. We show that when a community's perspective on an issue is stable, it opens the possibility for introducing knowledge services, based on an ontology co-constructed by knowledge engineers with stakeholders. Using a case study we describe our approach, ontology driven document enrichment, looking at how ontology construction and population can be supported by web based technologies.

Article Free Access Share on From documents to discourse: shifting conceptions of scholarly publishing Authors: Tamara Sumner Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K. Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K.View Profile , Simon Buckingham Shum Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K. Knowledge Media Institute, The Open University, Milton Keynes, MK7 6AA, U.K.View Profile Authors Info & Claims CHI '98: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsJanuary 1998 Pages 95–102https://doi.org/10.1145/274644.274659Published:01 January 1998Publication History 22citation671DownloadsMetricsTotal Citations22Total Downloads671Last 12 Months19Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Many efforts have been made to exploit the properties of graphical notations to support argument construction and communication. In the context of design rationale capture, we are interested in graphical argumentation structures as cognitive tools to support individual and collaborative design in real time. This context of use requires a detailed understanding of how a new representational structure integrates into the cognitive and discursive flow of design, that is, whether it provides supportive or intrusive structure. This article presents a use-oriented analysis of a graphical argumentation notation known as QOC (Questions, Options, and Criteria). Through a series of empirical studies, we show that it provides most support when elaborating poorly understood design spaces, but is a distraction when evaluating well-constrained design spaces. This is explained in terms of the cognitive compatibility between argumentative reasoning and the demands of different modes of designing. We then provide an account based on the collaborative affordances of QOC in group design meetings, and extend this to discuss the evolution of QOC argumentation from short term working memory to long term group memory.

Article Multidisciplinary modelling in HCI design…in theory and in practice Share on Authors: Victoria Bellotti Apple Computer Inc., One Infinite Loop, MS: 301-4A, Cupertino, CA and Rank Xerox Research Centre (formerly EuroPARC), 61 Regent Street, Cambridge, CB2 1AB, UK Apple Computer Inc., One Infinite Loop, MS: 301-4A, Cupertino, CA and Rank Xerox Research Centre (formerly EuroPARC), 61 Regent Street, Cambridge, CB2 1AB, UKView Profile , Simon Buckingham Shum Human-Computer Interaction Group, Dept. Psychology, University of York, York, YO1 5DD, UK Human-Computer Interaction Group, Dept. Psychology, University of York, York, YO1 5DD, UKView Profile , Allan MacLean Rank Xerox Research Centre (formerly EuroPARC), 61 Regent Street, Cambridge, CB2 1AB, UK Rank Xerox Research Centre (formerly EuroPARC), 61 Regent Street, Cambridge, CB2 1AB, UKView Profile , Nick Hammond Human-Computer Interaction Group, Dept. Psychology, University of York, York, YO1 5DD, UK Human-Computer Interaction Group, Dept. Psychology, University of York, York, YO1 5DD, UKView Profile Authors Info & Claims CHI '95: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsMay 1995 Pages 146–153https://doi.org/10.1145/223904.223923Online:01 May 1995Publication History 23citation1,433DownloadsMetricsTotal Citations23Total Downloads1,433Last 12 Months23Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Simon Buckingham Shum, Nick Hammond; Delivering HCI modelling to designers: a framework and case study of cognitive modelling, Interacting with Computers,