Multimodal Learning Analytics (MMLA) expands traditional learning analytics into the digital and physical learning environment, using diverse sensors and systems to collect information about education in more real-world environments. Challenges remain in making these technologies practical for capturing data in authentic learning situations. With the advent of readily accessible powerful artificial intelligence that includes multimodal large language models, new opportunities are available. However, few approaches allow access to these technologies, and most systems are developed for specific environments. Recent work has begun to make toolkits with access to collecting data from sensors, processing, and analyzing, yet these tools are challenging to integrate into a system. This paper introduces OpenMMLA, a toolkit approach that provides programming interfaces for harnessing these technologies into an MMLA platform with prebuilt pipelines, including the audio analyzer, indoor positioning, and video frame analyzer, offering multimodal data collection and visualizations and analytics. The paper provides an initial evaluation of the functionalities of the toolkit in data capturing and the implemented pipelines' performances.

This study explores the integration of Multimodal Learning Analytics (MMLA) within the dynamic learning ecosystem of the Glocal Classroom (GC). By employing the MMLA Model for Design and Analysis (MAMDA), our research proposes a conceptual model leveraging the GC's existing infrastructure into an MMLA system to enrich learning experiences and inform course design. Our methodology involves a case study approach guided by the six phases of MAMDA. Building on previous studies, including a systematic mapping of MMLA research and an investigation into MMLA system design. We seek to employ MMLA insights to comprehensively understand the learning experience, identify issues, and guide improvement strategies. Furthermore, we discuss potential challenges, mainly focusing on privacy and ethical considerations. The result of this work aims to facilitate a responsible and effective implementation of MMLA systems in educational settings.

In the dynamic landscape of digital education, the Glocal Classroom (GC) stands out as a multifaceted smart learning environment. The integration of Multimodal Learning Analytics (MMLA) comes as an intriguing proposition, promising insights into learning dynamics and enhancing educational outcomes. Encountering numerous interdependent considerations involved in the design and integration of MMLA systems, the MMLA design framework (MDF) addresses this challenge, providing a systematic approach. MDF consists of a phased and iterative method for designing MMLA systems. In this study, we delve into the details of the fifth phase, focusing on the development phase. Our primary objective is to assess and refine the applicability of MDF, by taking the integration of MMLA in GC as a use case. We analyze GC's technological infrastructure, evaluating existing hardware, network capabilities, and potential challenges. The central emphasis is on the technical architecture, specifically the hardware components supporting MMLA. By focusing on the technical complexities, the study provides insights into challenges and opportunities associated with MMLA implementation. The outcomes will deepen our understanding of technology in education and refine the MDF model, making it more effective for designing MMLA systems.

Multimodal Learning Analytics (MMLA) is an evolving sector within learning analytics that has become increasingly useful for examining complex learning and collaboration dynamics for group work across all educational levels. The availability of low-cost sensors and affordable computational power allows researchers to investigate different modes of group work. However, the field faces challenges stemming from the complexity and specialization of the systems required for capturing diverse interaction modalities, with commercial systems often being expensive or narrow in scope and researcher-developed systems needing to be more specialized and difficult to deploy. Therefore, more user-friendly, adaptable, affordable, open-source, and easy-to-deploy systems are needed to advance research and application in the MMLA field. The paper presents a field report on the design of mBox that aims to support group work across different contexts. We share the progress of mBox, a low-cost, easy-to-use platform grounded on learning theories to investigate collaborative learning settings. Our approach has been guided by iterative design processes that let us rapidly prototype different solutions for these settings.

Abstract Peer feedback has been widely used in computer-supported collaborative learning (CSCL) setting to improve students’ engagement with massive courses. Although the peer feedback process increases students’ self-regulatory practice, metacognition, and academic achievement, instructors need to go through large amounts of feedback text data which is much more time-consuming. To address this challenge, the present study proposes an automated content analysis approach to identify relevant categories in peer feedback based on traditional and sequence-based classifiers using TF-IDF and content-independent features. We use a data set from an extensive course (N = 231 students) in the setting of engineering higher education. In particular, a total of 2,444 peer feedback messages were analyzed. The CRF classification model based on the TF-IDF features achieved the best performance. The results illustrate that the ability to scale up the automatic analysis of peer feedback provides new opportunities for student-improved learning and improved teacher support in higher education at scale.

Designing MMLA systems is a complex task requiring a wide range of considerations. In this paper, we identify key considerations that are essential for designing MMLA systems. These considerations include data management, human factors, sensors and modalities, learning scenarios, privacy and ethics, interpretation and feedback, and data collection. The implications of these considerations are twofold: 1) The need for flexibility in MMLA systems to adapt to different learning contexts and scales, and 2) The need for a researcher-centered approach to designing MMLA systems. Unfortunately, the sheer number of considerations can lead to a state of "analysis paralysis," where deciding where to begin and how to proceed becomes overwhelming. This synthesis paper asks researchers to rethink the design of MMLA systems and aims to provide guidance for developers and practitioners in the field of MMLA.

Multimodal Learning Analytics (MMLA) provides opportunities for understanding and supporting collaborative problem-solving. However, the implementation of MMLA systems is challenging due to the lack of scalable technologies and limited solutions for collecting data from group work. This paper proposes the Multimodal Box (MBOX), an IoT-based system for MMLA, allowing the collection and processing of multimodal data from collaborative learning tasks. MBOX investigates the development and design for an IoT focusing on small group work in real-world settings. Moreover, MBOX promotes adaptation to different learning environments and enables a better scaling of computational resources used within the learning context.

In this study, we describe the results of our research to model collaborative problem-solving (CPS) competence based on analytics generated from video data. We have collected ~500 mins video data from 15 groups of 3 students working to solve design problems collaboratively. Initially, with the help of OpenPose, we automatically generated frequency metrics such as the number of the face-in-the-screen; and distance metrics such as the distance between bodies. Based on these metrics, we built decision trees to predict students' listening, watching, making, and speaking behaviours as well as predicting the students' CPS competence. Our results provide useful decision rules mined from analytics of video data which can be used to inform teacher dashboards. Although, the accuracy and recall values of the models built are inferior to previous machine learning work that utilizes multimodal data, the transparent nature of the decision trees provides opportunities for explainable analytics for teachers and learners. This can lead to more agency of teachers and learners, therefore can lead to easier adoption. We conclude the paper with a discussion on the value and limitations of our approach.

Multimodal learning analytics provides researchers new tools and techniques to capture different types of data from complex learning activities in dynamic learning environments. This paper investigates high-fidelity synchronised multimodal recordings of small groups of learners interacting from diverse sensors that include computer vision, user generated content, and data from the learning objects (like physical computing components or laboratory equipment). We processed and extracted different aspects of the students' interactions to answer the following question: which features of student group work are good predictors of team success in open-ended tasks with physical computing? The answer to the question provides ways to automatically identify the students' performance during the learning activities.

In this paper, we pose the question, can the tracking and analysis of the physical movements of students and teachers within a Practice-Based Learning (PBL) environment reveal information about the learning process that is relevant and informative to Learning Analytics (LA) implementations? Using the example of trials conducted in the design of a LA system, we aim to show how the analysis of physical movement from a macro level can help to enrich our understanding of what is happening in the classroom. The results suggest that Multimodal Learning Analytics (MMLA) could be used to generate valuable information about the human factors of the collaborative learning process and we propose how this information could assist in the provision of relevant supports for small group work. More research is needed to confirm the initial findings with larger sample sizes and refine the data capture and analysis methodology to allow automation.

Multimodal Learning Analytics (MMLA) captures, integrates and analyzes learning traces from different sources in order to obtain a more holistic understanding of the learning process, wherever it happens. MMLA leverages the increasingly widespread availability of diverse sensors, high-frequency data collection technologies and sophisticated machine learning and artificial intelligence techniques. The aim of this workshop is twofold: first, to expose participants to, and develop, different multimodal datasets that reflect how MMLA can bring new insights and opportunities to investigate complex learning processes and environments; second, to collaboratively identify a set of grand challenges for further MMLA research, built upon the foundations of previous workshops on the topic.

Introducing physical computing into regular school classes is challenged by constraints of schedules and curricula structures, which do not allow for time-consuming electronics prototyping. We present a novel approach to prototyping with physical computing components with the Arduino-based TALKOO kit: It comprises hardware modules, a visual IDE and prototyping material. Sensor and actuator modules are pluggable and do not require soldering and prior knowledge in electronics. The components have the ability to "talk" back to the visual IDE and to a learning analytics system. A new paradigm for visual programming maps physical modules onto virtual representations on screen making programming more intuitive. The TALKOO kit expands the field of application of physical computing for children in regular school contexts. Preliminary evaluation results show that children were able to build elaborative prototypes within an hour.

Systematic investigation of the collaborative problem solving process in open-ended, hands-on, physical computing design tasks requires a framework that highlights the main process features, stages and actions that then can be used to provide 'meaningful' learning analytics data. This paper presents an analysis framework that can be used to identify crucial aspects of the collaborative problem solving process in practice-based learning activities. We deployed a mixed-methods approach that allowed us to generate an analysis framework that is theoretically robust, and generalizable. Additionally, the framework is grounded in data and hence applicable to real-life learning contexts. This paper presents how our framework was developed and how it can be used to analyse data. We argue for the value of effective analysis frameworks in the generation and presentation of learning analytics for practice-based learning activities.

This poster explores how to develop a working framework for STEM education that uses both human annotated and machine data across a purpose-built learning environment. Our dual approach is to develop a robust framework for analysis and investigate how to design a learning analytics system to support hands-on engineering design tasks. Data from the first user tests are presented along with the framework for discussion.

Students’ difficulties in learning of mathematics have for a long time, been investigated by researchers in different fields. Within educational research there are claims that technological tools appropriately integrated in students’ mathematical work can support their understanding of a wide range of concepts in mathematics. This paper reports on the initial investigation for the design of Interactive Learning Environments (ILE) to support students’ learning of mathematics. The project is guided by the notion of Design Based Research (DBR) and aims to explore how to design ILE that support students’ understanding of integrals in particular. The initial study was conducted at a Swedish university with 10 students in 4 groups. The study confirmed difficulties in students’ understanding of integrals as reported in educational literature and provides a set of design aims for the next iteration of the ILE to support the learning.

This paper introduces the workshop "Exploring Design Methods for Mobile Learning" to be held at MobileHCI 2011, in Stockholm, Sweden.

Scientific practices increasingly incorporate sensors for data capture, information visualization for data analysis, and low-cost mobile devices for field-based inquiries incorporating open Web standards. While a broad range of design approaches for developing technology-enhanced learning has been used by researchers and practitioners for the last 15 years, significant challenges for educational use remain as new technologies and user experiences continually evolve outside the classroom. We focus on the specific design challenge of how to initiate the co-design process together with teachers, researchers, scientists, designers, and developers in order to devise and develop mobile science collaboratories that support open inquiry-based learning in ecology education. The outcomes presented in this paper point towards the need for additional methods to support co-design that take into consideration future user experiences needed for developing and implementing these types of learning activities.

New media technology students at university face many challenges from social to pedagogical when they start academic studies. One of the programs initial goals for these new students is to introduce them to some of the key social Web services that they will need to use for their schoolwork. The use of pervasive games to reach this specific purpose may help new students acclimate to these challenges. This paper presents a pervasive game that helps teams of students work together with new technologies to solve a mystery about a missing fictional professor. Our particular research interest relates to how pervasive games can be used to introduce social Web technologies and support team building for a university courses. The game presented in this paper show promise as a tool to get new university students in media technology actively involved in exploring social media web services.

In this paper we present our pedagogical and technological approach for supporting the design of novel situated learning activities that can be conducted both, outside the school and in the classroom. One main goal is to enhance the content of the curricula by bringing multimedia resources and mobile support to outdoor settings thus enriching the field experience. In order to illustrate these ideas we describe the outcomes of a trial we have conducted with thirty elementary school children. Moreover, we present the ubiquitous computing solutions we developed in order to support learning activities in the field of history. The results of our experiments indicate that children enjoyed learning in these kinds of environments where mobile devices are used in situ, thus supporting the learning activities in the context of which they are taking place.