Contains fulltext : 316609.pdf (Publisher’s version ) (Open Access)

Personalised feedback in MOOCs is often prevented by the non-scalability of traditional approaches and the lack of pedagogical grounding of the current learning analytics (LA) solutions. One way to tackle such limitations is the adoption of a participatory approach through the active positioning of MOOC instructors in the design and development of LA solutions. Nonetheless, there is a scarcity of empirical proposals supporting these approaches in MOOCs. To that end, the current paper presents e-FeeD4Mi, a web-based tool, incorporating a set of catalogues, recommendations and a process, that aim to support instructors in the design of human-centred LA-informed feedback. We conducted an evaluative study with 6 MOOC instructors who employed the tool into their course designs to assess e-FeeD4Mi usefulness, usability and associated workload. The evidence gathered permitted us to understand how e-FeeD4Mi support the design of human-centred LA-based feedback in MOOCs. Altogether, the results showed a good usability and participants' satisfaction regarding the use of the tool for shaping personalised feedback. At the same time, participants offered ideas for further tool enrichment. This study expands upon the current body of empirical research on the human-centred approaches in the design of LA-driven interventions in MOOCs.

The recent advances in educational technology enabled the development of solutions that collect and analyse data from learning scenarios to inform the decision-making processes. Research fields like Learning Analytics (LA) and Artificial Intelligence (AI) aim at supporting teaching and learning by using such solutions. However, their adoption in authentic settings is still limited, among other reasons, derived from ignoring the stakeholders' needs, a lack of pedagogical contextualisation, and a low trust in new technologies. Thus, the research fields of Human-Centered LA (HCLA) and Human-Centered AI (HCAI) recently emerged, aiming to understand the active involvement of stakeholders in the creation of such proposals. This paper presents a systematic literature review of 47 empirical research studies on the topic. The results show that more than two-thirds of the papers involve stakeholders in the design of the solutions, while fewer papers involved them during the ideation and prototyping, and the majority do not report any evaluation. Interestingly, while multiple techniques were used to collect data (mainly interviews, focus groups and workshops), few papers explicitly mentioned the adoption of existing HC design guidelines. Further evidence is needed to show the real impact of HCLA/HCAI approaches (e.g., in terms of user satisfaction and adoption).

The provision of feedback in Massive Open Online Courses (MOOCs) is a non-trivial task for instructors. Current Learning Analytics (LA) solutions provide opportunities for scaling up such feedback strategies. Yet, most of these solutions lack course contextualization and guidance for MOOC instructors. This paper presents a conceptual framework and a web-based tool aimed at supporting instructors in the design and enactment of feedback strategies within MOOC learning designs. Both the framework and the tool are based on a structured process and include a set of catalogues facilitating the reflection and (re)design of these strategies. Additionally, this paper reports an evaluation of the framework catalogues and the usefulness of the framework process in an online workshop (N=11). As part of this ongoing work, we also explored the experience of participants with the tool. The results are positive, highlighting the usefulness of the framework catalogues, especially for non-experienced MOOC instructors. We envision that the initial evidence gathered can contribute to research on conceptual and technological tools supporting feedback design, and to promote the synergy among LA and human-centered approaches in MOOC settings.

Detecting learners who face problems in MOOCs usually poses difficulties due to the high instructor-learners ratio, the diversity of the population, and the asynchronous participation mode. Existing solutions mainly draw on self-reported problems in discussion forums and on dashboards displaying learners’ activity traces. However, these approaches cannot scale up easily or do not consider the course learning design. This paper presents a conceptual framework aimed at guiding MOOC instructors in the identification of potential learners' problems and indicators of such problems, considering the learning design of the course (e.g., types of activities, difficulty, etc.). An instrumental qualitative case study served for the evaluation and refinement of the framework. The results showed that the framework positively helped instructors to reflect on potential learners’ problems they had not considered beforehand, and to associate such problems with a set of indicators related to their learning designs.

Gamification strategies have been proposed to mitigate student disengagement and dropouts in massive online environments, due to the positive results shown by these strategies at lower scales. Among various gamification strategies, redeemable rewards have been identified as an effective element to intrinsically motivate students and increase their engagement in educational settings, including MOOCs. Yet, effective design, implementation and enactment of this gamification strategy in MOOC contexts might face new challenges, given the unique characteristics of these learning settings such as massiveness. As an attempt to help teachers use redeemable rewards in MOOCs, this paper analyzes the characteristics of MOOCs that influence its integration and presents a proposal of a system supporting the design, implementation and enactment of such rewards. The envisioned system is illustrated by a scenario that describes the main features of this system for teachers and students.