Learning analytics dashboards (LADs) are widely used to present information about learners’ outcomes. However, few studies have examined how specific LAD elements, particularly those involving varying degrees of social comparison, contribute to students’ retrospective outcome emotions of pride, relief, disappointment, and shame. This study applies a transmodal analysis of eye-tracking data to identify viewing patterns of LAD elements critical to these emotions. Using data from 96 participants viewing dashboards that presented their assignment performance, we demonstrate how students’ gaze focus differed between those who experienced different retrospective outcome emotions and those who did not. Our findings suggest directions for future research on developing personalized, emotion-aware learning environments.

Learning Analytics Dashboards (LADs) have been studied to support student learning and motivation, yet how their effects evolve or endure over time remains in question. This study examines students’ immediate and retrospective emotional and motivational responses to two types of outcome-presenting dashboards: a class-performance comparison dashboard (CP-D) and an individual-comparison dashboard (IC-D). Using a mixed-methods approach with survey data, clustering of Achievement Goal Orientations (AGOs), and qualitative analysis of student reflections, the study explores how emotions persisted or diminished across the semester. Results showed that emotions more strongly associated with perceived control (e.g., pride, hope) tended to persist, whereas those less tied to control (e.g., assurance, hopelessness) were less stable. Reductions in anxiety and disappointment emerged most clearly among Mastery-Oriented students, with other clusters showing little differences between dashboard conditions. Interestingly, IC-D supported hope even among some lower-performing Non-Competitive students, while CP-D anchored students more strongly to the class average. These findings underscore that dashboard design interacts with performance and motivational profiles, suggesting the need for adaptive and hybrid designs that balance comparative feedback with individualized progress indicators.

The need to personalize learning analytics dashboards (LADs) is getting more recognized in learning analytics research community. In order to study the impact of these dashboards on learners, various types of prototypes have been designed and deployed in different settings. Applying Weiner’s attribution theory, our goal in this study was to understand the effect of dashboard information content on learners. We wanted to understand how elements of assignment grade, time spent on an assignment, assignment view, and proficiency in the dashboard affect students’ attribution of achievement and motivation for future work. We designed a qualitative study in which we analyzed participants’ responses and indicated behavioural changes after viewing the dashboard. Through in-depth interviews, we aimed to understand students’ interpretations of the designed dashboard, and to what extent social comparison impacts their judgments of learning. Students used multiple dimensions to attribute their success or failure to their ability and effort. Our results indicate that to maximize the benefits of dashboards as a vehicle for motivating change in students learning, the dashboard should promote effort in both personal and social comparison capacities.

This study explored two aspects of temporality in students’ SRL behaviours to understand the dynamics of SRL phase transitions. In the first aspect, which refers to the temporal order of Self-regulated learning (SRL) phases, we characterized four types of SRL processes based on phase transitions and the cyclical nature of SRL. The SRL types were mapped into the kinds of iterative behaviours over SRL phases which correspond to the theorized self-regulatory behaviours of students at different levels of SRL skills. We found a significant association between SRL types and the assignment grades that suggests the higher achieved learning outcomes, i.e., programming skills demonstrated in the assignments, being associated with more advanced SRL processes. This study also focused on the second aspect of temporality, which refers to the instance of time. We revealed the temporal dynamics between SRL phase transitions by analyzing time profiles for transitions in each SRL process type. Next, we showed that a two-day interval is a threshold by which most students iteratively transition from adapting to enactment phases, which provides a suitable time to intervene if the transition is not observed.

Research has emphasized that self-regulated learning (SRL) is critically important for learning. However, students have different capabilities of regulating their learning processes and individual needs. To help students improve their SRL capabilities, we need to identify students’ current behaviors. Specifically, we applied instructional design to create visible and meaningful markers of student learning at different points in time in LMS logs. We adopted knowledge engineering to develop a framework of proximal indicators representing SRL phases and evaluated them in a quasi-experiment in two different learning activities. A comparison of two sources of collected students’ SRL data, self-reported and trace data, revealed a relatively high agreement between our classifications (weighted kappa, κ = .74 and κ = .68). However, our indicators did not always discriminate adjacent SRL phases, particularly for enactment and adapting phases, compared with students’ real-time self-reported behaviors. Our behavioral indicators also were comparably successful at classifying SRL phases for different self-regulatory engagement levels. This study demonstrated how the triangulation of various sources of students’ self-regulatory data could help to unravel the complex nature of metacognitive processes.

The aim of student-facing dashboards is to support learning by providing students with actionable information and promoting self-regulated learning. We created a new dashboard design aligned with SRL theory, called MyLA, to better understand how students use a learning analytics tool. We conducted sequence analysis on students' interactions with three different visualizations in the dashboard, implemented in a LMS, for a large number of students (860) in ten courses representing different disciplines. To evaluate different students' experiences with the dashboard, we computed chi-squared tests of independence on dashboard users (52%) to find frequent patterns that discriminate students by their differences in academic achievement and self-regulated learning behaviors. The results revealed discriminating patterns in dashboard use among different levels of academic achievement and self-regulated learning, particularly for low achieving students and high self-regulated learners. Our findings highlight the importance of differences in students' experience with a student-facing dashboard, and emphasize that one size does not fit all in the design of learning analytics tools.

Performance and consistency play a large role in learning. This study analyzes the relation between consistency in students' online work habits and academic performance in a blended course. We utilize the data from logs recorded by a learning management system (LMS) in two information technology courses. The two courses required the completion of monthly asynchronous online discussion tasks and weekly assignments, respectively. We measure consistency by using Data Time Warping (DTW) distance for two successive tasks (assignments or discussions), as an appropriate measure to assess similarity of time series, over 11-day timeline starting 10 days before and up to the submission deadline. We found meaningful clusters of students exhibiting similar behavior and we use these to identify three distinct consistency patterns: highly consistent, incrementally consistent, and inconsistent users. We also found evidence of significant associations between these patterns and learner's academic performance.

With increasing abundance and ubiquity of mobile phones, desktop PCs, and tablets in the last decade, we are seeing students intermixing these modalities to learn and regulate their learning. However, the role of these modalities in educational settings is still largely under-researched. Similarly, little attention has been paid to the research on the extension of learning analytics to analyze the learning processes of students adopting various modalities during a learning activity. Traditionally, research on how modalities affect the way in which activities are completed has mainly relied upon self-reported data or mere counts of access from each modality. We explore the use of technological modalities in regulating learning via learning management systems (LMS) in the context of blended courses. We used data mining techniques to analyze patterns in sequences of actions performed by learners (n = 120) across different modalities in order to identify technological modality profiles of sequences. These profiles were used to detect the technological modality strategies adopted by students. We found a moderate effect size (∈2 = 0.12) of students' adopted strategies on the final course grade. Furthermore, when looking specifically at online discussion engagement and performance, students' adopted technological modality strategies explained a large amount of variance (η2 = 0.68) in their engagement and quality of contributions. The result implications and further research are discussed.

A mixed methods approach was undertaken in this exploratory study to better understand how learners perceive and utilize learning analytics visualizations during online discussions activities. Internal conditions such as goal orientation and numeracy were measured alongside the external conditions created by the discussion structure and learning analytics. Our results emphasize key factors that should be considered when designing learning analytics tools.

When designing learning analytics tools for use by learners we have an opportunity to provide tools that consider a particular learner's situation and the learner herself. To afford actual impact on learning, such tools have to be informed by theories of education. Particularly, educational research shows that individual differences play a significant role in explaining students' learning process. However, limited empirical research in learning analytics has investigated the role of theoretical constructs, such as motivational factors, that are underlying the observed differences between individuals. In this work, we conducted a field experiment to examine the effect of three designed learning analytics visualizations on students' participation in online discussions in authentic course settings. Using hierarchical linear mixed models, our results revealed that effects of visualizations on the quantity and quality of messages posted by students with differences in achievement goal orientations could either be positive or negative. Our findings highlight the methodological importance of considering individual differences and pose important implications for future design and research of learning analytics visualizations.

In this paper, we present the results of an exploratory study that examined the problem of automating content analysis of student online discussion transcripts. We looked at the problem of coding discussion transcripts for the levels of cognitive presence, one of the three main constructs in the Community of Inquiry (CoI) model of distance education. Using Coh-Metrix and LIWC features, together with a set of custom features developed to capture discussion context, we developed a random forest classification system that achieved 70.3% classification accuracy and 0.63 Cohen's kappa, which is significantly higher than values reported in the previous studies. Besides improvement in classification accuracy, the developed system is also less sensitive to overfitting as it uses only 205 classification features, which is around 100 times less features than in similar systems based on bag-of-words features. We also provide an overview of the classification features most indicative of the different phases of cognitive presence that gives an additional insights into the nature of cognitive presence learning cycle. Overall, our results show great potential of the proposed approach, with an added benefit of providing further characterization of the cognitive presence coding scheme.

Research on self-regulated learning has taken main two paths: self-regulated learning as aptitudes and more recently, self-regulated learning as events. This paper proposes the use of the Fuzzy miner process mining technique to examine the relationship between students' self-reported aptitudes (i.e., achievement goal orientation and approaches to learning) and strategies followed in self-regulated learning. A pilot study is conducted to probe the method and the preliminary results are reported.

Creating meaning from a wide variety of available information and being able to choose what to learn are highly relevant skills for learning in a connectivist setting. In this work, various approaches have been utilized to gain insights into learning processes occurring within a network of learners and understand the factors that shape learners' interests and the topics to which learners devote a significant attention. This study combines different methods to develop a scalable analytic approach for a comprehensive analysis of learners' discourse in a connectivist massive open online course (cMOOC). By linking techniques for semantic annotation and graph analysis with a qualitative analysis of learner-generated discourse, we examined how social media platforms (blogs, Twitter, and Facebook) and course recommendations influence content creation and topics discussed within a cMOOC. Our findings indicate that learners tend to focus on several prominent topics that emerge very quickly in the course. They maintain that focus, with some exceptions, throughout the course, regardless of readings suggested by the instructor. Moreover, the topics discussed across different social media differ, which can likely be attributed to the affordances of different media. Finally, our results indicate a relatively low level of cohesion in the topics discussed which might be an indicator of a diversity of the conceptual coverage discussed by the course participants.

All forms of learning take time. There is a large body of research suggesting that the amount of time spent on learning can improve the quality of learning, as represented by academic performance. The wide-spread adoption of learning technologies such as learning management systems (LMSs), has resulted in large amounts of data about student learning being readily accessible to educational researchers. One common use of this data is to measure time that students have spent on different learning tasks (i.e., time-on-task). Given that LMS systems typically only capture times when students executed various actions, time-on-task measures are estimated based on the recorded trace data. LMS trace data has been extensively used in many studies in the field of learning analytics, yet the problem of time-on-task estimation is rarely described in detail and the consequences that it entails are not fully examined.

In this design briefing, we introduce the Learn-B environment, our attempt in designing and implementing a research prototype to address some of the challenges inherent in workplace learning: the informal aspect of workplace learning requires knowledge workers to be supported in their self-regulatory learning (SRL) processes, whilst its social nature draws attention to the role of collective in those processes. Moreover, learning at workplace is contextual and on-demand, thus requiring organizations to recognize and motivate the learning and knowledge building activities of their employees, where individual learning goals are harmonized with those of the organization. In particular, we focus on the analytics-based features of Learn-B, illustrate their design and current implementation, and discuss how each of them is hypothesized to target the above challenges.

Feature modeling is one of the main techniques used in Software Product Line Engineering to manage the variability within the products of a family. Concrete products of the family can be generated through a configuration process. The configuration process selects and/or removes features from the feature model according to the stakeholders' requirements. Selecting the right set of features for one product from amongst all of the available features in the feature model is a complex task because: 1) the multiplicity of stakeholders' functional requirements; 2) the positive or negative impact of features on non-functional properties; and 3) the stakeholders' preferences w.r.t. the desirable non-functional properties of the final product. Many configurations techniques have already been proposed to facilitate automated product derivation. However, most of the current proposals are not designed to consider stakeholders' preferences and constraints especially with regard to non-functional properties. We address the software product line configuration problem and propose a framework, which employs an artificial intelligence planning technique to automatically select suitable features that satisfy both the stakeholders' functional and non-functional preferences and constraints. We also provide tooling support to facilitate the use of our framework. Our experiments show that despite the complexity involved with the simultaneous consideration of both functional and non-functional properties our configuration technique is scalable.

In Software Product Line Engineering, concrete products of a family can be generated through a configuration process over a feature model. The configuration process selects features from the feature model according to the stakeholders' requirements. Selecting the right set of features for one product from all the available features in the feature model is a cumbersome task because 1) the stakeholders may have diverse business concerns and limited resources that they can spend on a product and 2) features may have negative and positive contributions on different business concern. Many configurations techniques have been proposed to facilitate software developers' tasks through automated product derivation. However, most of the current proposals for automatic configuration are not devised to cope with business oriented requirements and stakeholders' resource limitations. We propose a framework, which employs an artificial intelligence planning technique to automatically select suitable features that satisfy the stakeholders' business concerns and resource limitations. We also provide tooling support to facilitate the use of our framework.

In this paper we describe the Reading Glove, a wearable RFID reader for interacting with a tangible narrative. Based on interviews with study participants, we present a set of observed themes for understanding how the wearable and tangible aspects of the Reading Glove influence the user experience. We connect our observational themes to theoretical notions from interactive narrative and tangible interaction to create a set of design considerations such as enacting a role, ownership and permission, multiplicity of interpretations and boundary objects.

Since the introduction in the early nineties, feature models receive a great deal of attention in industry and academia. Industrial success stories in applying feature models for modeling software product lines, and using them for configuring software-intensive systems motivate academia to discover ways to integrate different feature dependencies into the feature model, and automate verified feature configuration. In this paper we demonstrate how ontologies and Semantic Web technologies facilitate seamless integration of required external services and deployment platform capabilities into the feature model. Furthermore, we also contribute with an algorithm for automating staged configuration using Semantic Web reasoners to discover unfeasible features of the feature model.

Having the notion of services brought into the domain of component based software design has enriched the domain of modular self-contained components with support for loose coupling, dynamic runtime discovery, and late binding. However, loose coupling, runtime discovery, and composition for a service-oriented component model in its current scale mostly applies to homogeneous services that admit to the specifications of the original service component model. This design is in conflict with the initial outlook of SOA in which reusability of existing services is considered a primary goal. In this paper, we discuss ReCoIn, a remote service composition and integration framework, which aims to bring functionalities of traditional Web services into the local repository of a service oriented component model. We discuss how ReCoIn enables remote Web services to be properly discovered, encapsulated, and used in a service oriented component model framework, offering better service reuse and fault tolerance. In this paper, we layout the overall architectural design for ReCoIn and present its initial prototype.

We discuss three design strategies for improving the quality of social interaction and learning with interactive museum guides: 1) embodied interaction; 2) game-learning; 3) a hybrid system. We used these strategies in our prototype Kurio, which is aimed at supporting families visiting museums. The results of our evaluation show positive implications of implementing the design strategies: closing the social gap, naturalizing technology, and supporting exploration and discovery in learning.

The social semantic Web has recently emerged as a paradigm in which ontologies (aimed at defining, structuring and sharing information) and collaborative software (used for creating and sharing knowledge) have been merged together. Ontologies provide an effective means of capturing and integrating knowledge for feedback provisioning, while using collaborative activities can support pedagogical theories, such as social constructivism. Both technologies have developed separately in the e-learning domain; representing respectively a teacher-centered and a learner-centered approach for learning environments. In this paper we bridge the gap between these two approaches by leveraging the social semantic Web paradigm, and propose a collaborative semantic-rich learning environment in which folksonomies created from studentspsila collaborative tags contribute to ontology maintenance, and teacher-directed feedback.

This paper presents the results of a study on group interaction with a prototype known as socio-ec(h)o. socio-ec(h)o explores the design of sensing and display, user modeling, and interaction in an embedded interaction system utilizing a game structure. Our study involved the playing of our prototype system by thirty-six (36) participants grouped into teams of four (4). Our aim was to determine heuristics that we could use to further design the interaction and user model approaches for group and embodied interaction systems. We analyzed group interaction and performance based on factors of team cohesion and goal focus. We found that with our system, these factors alone could not explain performance. However, when transitions in the degrees of each factor, i.e. high, medium or low are considered, a clearer picture for performance emerges. The significance of the results is that they describe recognizable factors for positive group interaction. Author Keywords Groups, responsive environment, play, embodiment,

A business process usually includes multiple busi- ness partners that use systems with their business log- ics represented in different rule (or policy) languages. The integration of business processes is a major goal followed by business enterprises and involves the in- terchange of business rules and policies between part- ners. However, the variety of business rules employed by the partners presents a significant burden to the integration of collaborating business systems. In this paper, we propose a solution that enables translating rules and policies defined in different rule languages into a single general rule language (REWERSE I1 Rule Markup Language - R2ML) and processing them in a uniform manner. This provides a unified view of different partners' business rules in an integration process, while permitting the partners to continue to leverage their own business rules without changing their technologies. We show how the concepts of the KAoS policy language can be transformed to R2ML and then from R2ML to the other rule (or policy) lan- guages.

In our previous work, we developed the LOCO ontology framework which formalizes the notion of learning object context as a complex interplay of learning activities, learning objects, and learners. We now use that framework in conjunction with semantic annotation to generate different kinds of feedback (which we had identified by interviewing several Web educators) for educators to help them improve the learning process in Web-based settings. To test the feasibility of the proposed approach for feedback provision we developed a tool named LOCO-Analyst. Here we report on our experiences in developing LOCO-Analyst and using it to generate feedback out of the real data obtained from the iHelp Courses Learning Content Management System. Finally, we present evaluation results.

In this paper we discuss an adaptive museum guide prototype in which playfulness is a key design goal for the interaction experience. The interface for our prototype is a combined tangible user interface and audio display. We discuss how we determined the specific requirements for play through an ethnographic study and analysis based on ecological concepts of Bell and Nardi & O'Day. We found that we could consider play in two main forms in regard to the interface: content and physical play. We also found that play is highly contextual. Designers need to consider the situated nature of play for two reasons: 1) to best serve the overall design purpose; 2) in order to understand the nature and degree of play required. We augmented traditional user experience evaluation methods of questionnaires and interviews with observational analysis based on Djajadiningrat's descriptions of aesthetic interaction.

In this paper we describe our novel solution for Web services enabling users from the trusted organizations to access learning objects in the repository based on their attributes in their home organizations. The solution extends a Web-based Shibboleth system into the realm of Web services. It utilizes the Web services security SAML profile and combines it with the XACML access control policies. The technical solution is described in the context of the course management systems with complex access policies in operation at our campus.

This paper describes an ambient intelligent prototype known as socio-ec(h)o. socio-ec(h)o explores the design and implementation of a system for sensing and display, user modeling, and interaction models based on a game structure. The game structure includes, word puzzles, levels, body states, goals and game skills. Body states are body movements and positions that players must discover in order to complete a level and in turn represent a learned game skill. The paper provides an overview of background concepts and related research. We describe the prototype and game structure, provide a technical description of the prototype and discuss technical issues related to sensing, reasoning and display. The paper contributes by providing a method for constructing group parameters from individual parameters with real-time motion capture data; and a model for mapping the trajectory of participant's actions in order to determine an intensity level used to manage the experience flow of the game and its representation in audio and visual display. We conclude with a discussion of known and outstanding technical issues, and future research.

CSCW is one mechanism by which the TeleLearning Exchange Project promotes collaboration in the interdisciplinary research of the TeleLearning Network of Centres of Excellence (TL-NCE). Affiliated researchers are encouraged to share informal knowledge of their projects via three major strategies: site visits by an integration officer, a community Web repository (http://teleport.techbc.ca) and collaborative visits by students. An informal goal of the TeleLearning Exchange is to scaffold the next generation of collaborative researchers by actively involving students affiliated with the TL-NCE as agents of interdisciplinary cooperation and technical integration. The paper describes the design and development issues surrounding a Web repository tool to build a community of practice.

Integrating working and learning is seen as a desirable alternative to traditional training regimes. An integrated approach to working and learning is more gradual, contextual, problem oriented and cost effective. Knowledge technology aims to catalyze workplace learning, but requires the right organisational culture and brings additional costs regarding the articulation, representation and transmission of knowledge. Our approach reduces these costs by making articulation a natural part of collaborative working, designing for both informal and formal knowledge, and facilitating the transition of socially situated knowledge through enriched documents. To be successful, our approach has certain prequesites concerning organisational culture, and the nature of shared documents, organisational knowledge and work activities.

Engineering design is a complex activity, relying heavily on know-how gained from personal experience. Competitive pressures and new technology are making further demands on the skills and experience of designers, as effective knowledge reuse in design is seen as increasingly vital, and the work of design teams is often a collaborative and distributed activity. University students with a thorough knowledge of the engineering domain can be ill prepared for professional practice, with its increasing reliance on skills and know-how as well as knowledge of theory. Our approach aims to better prepare students for professional practice, through hands-on experience of design reuse, participation in distributed collaboration, and the development of presentation and documentation skills. Our case-study in the domain of modelling engineering systems, in which the course materials themselves are evolving and distributed, has ramifications for the publication model of educational materials, and the way students should be prepared for working life.