Multimodal learning analytics (MMLA) research for building collaboration quality estimation models has shown significant progress. However, the generalizability of such models is seldom addressed. In this paper, we address this gap by systematically evaluating the across-context generalizability of collaboration quality models developed using a typical MMLA pipeline. This paper further presents a methodology to explore modelling pipelines with different configurations to improve the generalizability of the model. We collected 11 multimodal datasets (audio and log data) from face-to-face collaborative learning activities in six different classrooms with five different subject teachers. Our results showed that the models developed using the often-employed MMLA pipeline degraded in terms of Kappa from Fair (.20 < Kappa < .40) to Poor (Kappa < .20) when evaluated across contexts. This degradation in performance was significantly ameliorated with pipelines that emerged as high-performing from our exploration of 32 pipelines. Furthermore, our exploration of pipelines provided statistical evidence that often-overlooked contextual data features improve the generalizability of a collaboration quality model. With these findings, we make recommendations for the modelling pipeline which can potentially help other researchers in achieving better generalizability in their collaboration quality estimation models.

Multimodal Learning Analytics (MMLA) has been applied to collaborative learning, often to estimate collaboration quality with the use of multimodal data, which often have uneven time scales. The difference in time scales is usually handled by dividing and aggregating data using a fixed-size time window. So far, the current MMLA research lacks a systematic exploration of whether and how much window size affects the generalizability of collaboration quality estimation models. In this paper, we investigate the impact of different window sizes (e.g., 30 seconds, 60s, 90s, 120s, 180s, 240s) on the generalizability of classification models for collaboration quality and its underlying dimensions (e.g., argumentation). Our results from an MMLA study involving the use of audio and log data showed that a 60 seconds window size enabled the development of more generalizable models for collaboration quality (AUC 61%) and argumentation (AUC 64%). In contrast, for modeling dimensions focusing on coordination, interpersonal relationship, and joint information processing, a window size of 180 seconds led to better performance in terms of across-context generalizability (on average from 56% AUC to 63% AUC). These findings have implications for the eventual application of MMLA in authentic practice.

Cultural Heritage learners can highly benefit from ubiquitous learning approaches that connect in-classroom activities with active on-site learning opportunities. However, teachers face the problem that the current landscape of technological support for learning Cultural Heritage mostly consists of mobile applications for informal contexts, completely decoupled from curricular activities. This paper presents the results of an exploratory-sequential mixed-methods study that focuses on the feasibility, for non-technical expert teachers, of repurposing an existing mobile application in formal Cultural Heritage ubiquitous learning situations. Such feasibility is explored from the perspective of the orchestration metaphor. More specifically, we used the “5+3 aspects orchestration framework” to understand the orchestration challenges of four ubiquitous learning situations about Cultural Heritage carried out in two secondary schools. These situations involved five teachers and 139 students who used a mobile application called Casual Learn for several weeks. The results of the study suggest that, after a brief training, teachers can design, enact, and orchestrate ubiquitous learning situations that involve both in-classroom and out-classroom activities. The teachers were able to transfer to the students most of the orchestration load of out-classroom activities. The results also showed that the flow of learning artifacts and information between in- and out-classroom activities is a critical issue that needs to be addressed by developers of ubiquitous learning technological solutions.

This paper presents SLEek, an ontology for the context-aware recommendation of learning activities in Smart Learning Environments (SLEs). SLEek creates an actor-artifact network that is especially suitable for the context-aware recommendation of activities across spaces in formal and informal contexts. SLEek implementation reuses vocabularies from the Web of Data and is currently used in a dataset of 17K learning activities.

Learners have ubiquitous informal learning opportunities, but it is difficult to take advantage from them and relate them to their formal education. The connection of formal and informal learning is one of the aims of SLEs, but how to do it is still a question. This paper explores such connection by integrating the mobile application Casual Learn to the SLE SCARLETT and it discusses the challenges faced in such integration.

Multimodal Learning Analytics (MMLA) aims to support evidence-based educational practices by collecting, processing, analyzing and sense-making of multimodal evidence of learning. MMLA is not widespread yet and there are few tailored MMLA solutions to meet the requirements of a specific learning scenario. This PhD project investigates the main challenges behind the limited development of MMLA solutions and proposes an MMLA infrastructure as the main contribution. The proposed infrastructure includes three components - a data value chain, a data model and a software architecture. The overall project follows the design-based research methodology where multiple iterations are involved to refine the contributions.

There is an increasing interest in Multimodal Learning Analytics (MMLA), which involves complex technical issues in gathering, merging and analyzing different types of learning data from heterogeneous data sources. However, there is still no common reference architecture to face these technical challenges of MMLA. This paper summarizes the state of the art of MMLA software architectures through a systematic literature review. Our analysis of nine architecture proposals highlights the uneven support provided by existing architectures to the different activities of the analytics data value chain (DVC). We find out in those infrastructures that data organization and decision-making support have been under-explored so far. Based on the lessons learnt from the review, we also identify that design tensions like architecture distribution, flexibility and extensibility (and an increased focus on data organization and decision making) are some of the most promising issues to be addressed by the MMLA community in the near future.

This paper presents the potential of a social semantic infrastructure that implements an Actor Artifact Network (AAN) with the final goal of supporting learning analytics at the workplace. Two applications were built on top of such infrastructure and make use of the emerging relations of such a AAN. A preliminary evaluation shows that an AAN can be created out of the usage of both applications, thus opening the possibility to implement learning analytics at the workplace.

Linked Data is a methodology to publish data on the Web that facilitates the federation of data sources. Following this methodology, a huge quantity of data is being published on the Web of Data, but very little is currently consumed by educational applications. This paper proposes to overcome this situation by relating educational vocabularies to the ones already used on the Web of Data. This way, educational applications can take advantage of the data published on the Web even if such data was not published for educational purposes. As a case study, the vocabulary used by an educational tool search system, called Ontoolsearch, is related to ICT tool descriptions published by DBpedia and Factforge. Evaluation results show that 2905 tool descriptions can be retrieved from the Web of Data to be consumed by Ontoolsearch. In addition, most of these descriptions have a similar quality to the ones manually published by an educational tool expert.

The integration of existing educational computer-based tools is a current research trend aimed at increasing the range of learning situations that can be supported by most widely adopted Virtual Learning Environments (VLEs). Nevertheless, due to the technological and functional heterogeneity of both tools and VLEs, there is an integration cost that must be assumed and that varies depending on several design issues. This paper identifies the issues to be tackled when designing a VLE or a tool and the effects that the different alternatives may have in the integration cost. Besides, the paper discusses the advantages and drawbacks of previous research proposals when dealing with such issues. The final goal is to understand the impact of design decisions in the chances for acceptance of the adopted integration approaches.