Jelena Jovanovic 0001

dblp:11/16-1 · DBLP profile ↗
← Back
71ranked-venue papers
14as first author
11since 2021 · last 2025
0000-0002-1904-0446ORCID · conflict

Domains — the database's venue-derived domains; a paper can count in several

Applied, interdisciplinary, general and emerging computing · 46 · 9 first-author · 10 since 2021Human-computer interaction and ubiquitous computing · 44 · 9 first-author · 8 since 2021Artificial intelligence and machine learning · 15 · 3 first-author · 1 since 2021Databases, data management, data science and information retrieval · 13 · 2 first-author · 1 since 2021Software engineering, systems software and programming languages · 2 · 1 first-author · 1 since 2021
YearPublicationVenuePosition
2025 chatgptscrapeR: A Tool for Retrieving Student-AI Interactions
abstract
The rapid adoption of ChatGPT and other large language models (LLMs) in education has created new opportunities for human-AI collaboration research, e.g., studying interactions, automating support or implementing novel ways of assessment. However, existing methods for retrieving ChatGPT conversation data -either through OpenAI's API or manual transcription-are limited by technical, financial, and scalability constraints. This paper introduces chatGPTscrapeR, an open-source R package and Shiny web application that automates the extraction of ChatGPT conversation data from URLs. Thus, it enables researchers and educators to efficiently retrieve, organize, and subsequently analyze interaction logs, and their metadata. The retrieved data are ready to be assessed if they are part of an assignment or analyzed using different methods. In all such cases, automating the retrieval of human-AI interactions is instrumental for an efficient analysis of such interactions and for creating modern AI-enabled learning systems.
Sonsoles López-Pernas, Kamila Misiejuk, Jelena Jovanovic 0001, Miroslava Raspopovic Milic, Miguel Ángel Conde González, Mohammed Saqr
ICALT3
2025 Advancing privacy in learning analytics using differential privacy
abstract
This paper addresses the challenge of balancing learner data privacy with the use of data in learning analytics (LA) by proposing a novel framework by applying Differential Privacy (DP). The need for more robust privacy protection keeps increasing, driven by evolving legal regulations and heightened privacy concerns, as well as traditional anonymization methods being insufficient for the complexities of educational data. To address this, we introduce the first DP framework specifically designed for LA and provide practical guidance for its implementation. We demonstrate the use of this framework through a LA usage scenario and validate DP in safeguarding data privacy against potential attacks through an experiment on a well-known LA dataset. Additionally, we explore the trade-offs between data privacy and utility across various DP settings. Our work contributes to the field of LA by offering a practical DP framework that can support researchers and practitioners in adopting DP in their works.
Qinyi Liu, Ronas Shakya, Mohammad Khalil, Jelena Jovanovic 0001
LAK4
2024 The Interplay of Learning Analytics and Artificial Intelligence
abstract
The widespread use of digital systems and tools in education has opened up opportunities for collecting, measuring, and analysing data about user (learner, teacher) interactions with a variety of learning resources and activities, with the ultimate objective of better understanding learning and advancing both learning outcomes and the overall learning experience.This promise motivated the development of Learning Analytics (LA) as a research and practical field and the use of insights derived from learning trace data for evidencebased decision making in a variety of educational settings.While LA has made a significant contribution to better understanding of learning and the environments in which it takes place, many open questions and challenges remain.Furthermore, new opportunities and challenges continue to emerge with the everchanging modalities of teaching and learning, the latest of which are associated with the rapid development and accessibility of Artificial Intelligence (AI).Taking the cyclical model of LA as its exploration framework, this paper examines how key components of the LA model -namely data, methods, and actions -relate to and may benefit from the latest developments in AI, and especially Generative AI.Aiming for evidence-based analysis and discussion of the interplay between LA and AI, the paper relies on the latest empirical research in LA and the related research fields of AI in Education and Educational Data Mining.
Jelena Jovanovic 0001
FedCSIS1
2024 Towards Comprehensive Monitoring of Graduate Attribute Development: A Learning Analytics Approach in Higher Education
abstract
In response to the evolving demands of the contemporary workplace, higher education (HE) institutions are increasingly emphasising the development of transversal skills and graduate attributes (GAs). The development of GAs, such as effective communication, collaboration, and lifelong learning, are non-linear and follow distinct trajectories for individual learners. The ability to trace and measure the progression of GA remains a significant challenge. While previous studies have focused on empirical methods for measuring GAs in individual courses, a notable gap exists in understanding their longitudinal development within HE programs. To address this research gap, our study focuses on measuring and tracing the development of GAs in an Initial Teacher Education (ITE) undergraduate program at a large public university in Australia. By combining learning analytics (LA) with psychometric models, we analysed students’ assessment grades to measure learners’ GA development in each year of the ITE program. The resulting measurements enabled the identification of distinct profiles of GA attainment, as demonstrated by learners and their distinct pathways. The overall approach allows for a comprehensive representation of a learner's progress throughout the program of study. As such, the developed approach sets the grounds for more personalised learning support, program evaluation, and improvement of students’ GA attainment.
Abhinava Barthakur, Jelena Jovanovic 0001, Andrew Zamecnik, Vitomir Kovanovic, Gongjun Xu, Shane Dawson
LAK2
2024 Scaling While Privacy Preserving: A Comprehensive Synthetic Tabular Data Generation and Evaluation in Learning Analytics
abstract
Privacy poses a significant obstacle to the progress of learning analytics (LA), presenting challenges like inadequate anonymization and data misuse that current solutions struggle to address. Synthetic data emerges as a potential remedy, offering robust privacy protection. However, prior LA research on synthetic data lacks thorough evaluation, essential for assessing the delicate balance between privacy and data utility. Synthetic data must not only enhance privacy but also remain practical for data analytics. Moreover, diverse LA scenarios come with varying privacy and utility needs, making the selection of an appropriate synthetic data approach a pressing challenge. To address these gaps, we propose a comprehensive evaluation of synthetic data, which encompasses three dimensions of synthetic data quality, namely resemblance, utility, and privacy. We apply this evaluation to three distinct LA datasets, using three different synthetic data generation methods. Our results show that synthetic data can maintain similar utility (i.e., predictive performance) as real data, while preserving privacy. Furthermore, considering different privacy and data utility requirements in different LA scenarios, we make customized recommendations for synthetic data generation. This paper not only presents a comprehensive evaluation of synthetic data but also illustrates its potential in mitigating privacy concerns within the field of LA, thus contributing to a wider application of synthetic data in LA and promoting a better practice for open science.
Qinyi Liu, Mohammad Khalil, Jelena Jovanovic 0001, Ronas Shakya
LAK3
2024 Social Alignment Contagion in Online Social Networks
abstract
Researchers have already observed social contagion effects in both in-person and online interactions. However, such studies have primarily focused on users’ beliefs, mental states, and interests. In this article, we expand the state of the art by exploring the impact of social contagion on social alignment, i.e., whether the decision to socially align oneself with the general opinion of the users on the social network is contagious to one’s connections on the network or not. The novelty of our work in this article includes: 1) unlike earlier work, this article is among the first to explore the contagiousness of the concept of social alignment on social networks; 2) our work adopts an instrumental variable approach to determine reliable causal relations between observed social contagion effects on the social network; and 3) our work expands beyond the mere presence of contagion in social alignment and also explores the role of population heterogeneity on social alignment contagion. Based on the systematic collection and analysis of data from two large social network platforms, namely, Twitter and Foursquare, we find that a user’s decision to socially align or distance from social topics and sentiments influences the social alignment decisions of their connections on the social network. We further find that such social alignment decisions are significantly impacted by population heterogeneity.
Amin Mirlohi, Jalehsadat Mahdavimoghaddam, Jelena Jovanovic 0001, Feras N. Al-Obeidat, Mehdi Khani, Ali A. Ghorbani 0001, Ebrahim Bagheri
IEEE Trans. Comput. Soc. Syst.3
2023 Student Profiles of Change in a University Course: A Complex Dynamical Systems Perspective
abstract
Learning analytics approaches to profiling students based on their study behaviour remain limited in how they integrate temporality and change. To advance this area of work, the current study examines profiles of change in student study behaviour in a blended undergraduate engineering course. The study is conceptualised through complex dynamical systems theory and its applications in psychological and cognitive science research. Students were profiled based on the changes in their behaviour as observed in clickstream data. Measure of entropy in the recurrence of student behaviour was used to indicate the change of a student state, consistent with the evidence from cognitive sciences. Student trajectories of weekly entropy values were clustered to identify distinct profiles. Three patterns were identified: stable weekly study, steep changes in weekly study, and moderate changes in weekly study. The students with steep changes in their weekly study activity had lower exam grades and showed destabilisation of weekly behaviour earlier in the course. The study investigated the relationships between these profiles of change, student performance, and other approaches to learner profiling, such as self-reported measures of self-regulated learning, and profiles based on the sequences of learning actions.
Oleksandra Poquet, Jelena Jovanovic 0001, Abelardo Pardo
LAK2
2022 Effects of Internal and External Conditions on Strategies of Self-regulated Learning: A Learning Analytics Study
abstract
Self-regulated learning (SRL) skills are essential for successful learning in a technology-enhanced learning environment. Learning Analytics techniques have shown a great potential in identifying and exploring SRL strategies from trace data in various learning environments. However, these strategies have been mainly identified through analysis of sequences of learning actions, and thus interpretation of the strategies is heavily task and context dependent. Further, little research has been done on the association of SRL strategies with different influencing factors or conditions. To address these gaps, we propose an analytic method for detecting SRL strategies from theoretically supported SRL processes and applied the method to a dataset collected from a multi-source writing task. The detected SRL strategies were explored in terms of their association with the learning outcome, internal conditions (prior-knowledge, metacognitive knowledge and motivation) and external conditions (scaffolding). The study results showed our analytic method successfully identified three theoretically meaningful SRL strategies. The study results revealed small effect size in the association between the internal conditions and the identified SRL strategies, but revealed a moderate effect size in the association between external conditions and the SRL strategy use.
Namrata Srivastava, Yizhou Fan, Mladen Rakovic, Shaveen Singh, Jelena Jovanovic 0001, Joep van der Graaf, Lyn Lim, Surya Surendrannair, Jonathan Kilgour, Inge Molenaar, Maria Bannert, Johanna D. Moore, Dragan Gasevic
LAK5
2021 A learning analytic approach to unveiling self-regulatory processes in learning tactics
abstract
Investigation of learning tactics and strategies has received increasing attention by the Learning Analytics (LA) community. While previous research efforts have made notable contributions towards identifying and understanding learning tactics from trace data in various blended and online learning settings, there is still a need to deepen our understanding about learning processes that are activated during the enactment of distinct learning tactics. In order to fill this gap, we propose a learning analytic approach to unveiling and comparing self-regulatory processes in learning tactics detected from trace data. Following this approach, we detected four learning tactics (Reading with Quiz Tactic, Assessment and Interaction Tactic, Short Login and Interact Tactic and Focus on Quiz Tactic) as used by 728 learners in an undergrad course. We then theorised and detected five micro-level processes of self-regulated learning (SRL) through an analysis of trace data. We analysed how these micro-level SRL processes were activated during enactment of the four learning tactics in terms of their frequency of occurrence and temporal sequencing. We found significant differences across the four tactics regarding the five micro-level SRL processes based on multivariate analysis of variance and comparison of process models. In summary, the proposed LA approach allows for meaningful interpretation and distinction of learning tactics in terms of the underlying SRL processes. More importantly, this approach shows the potential to overcome the limitations in the interpretation of LA results which stem from the context-specific nature of learning. Specifically, the study has demonstrated how the interpretation of LA results and recommendation of pedagogical interventions can also be provided at the level of learning processes rather than only in terms of a specific course design.
Yizhou Fan, John Saint, Shaveen Singh, Jelena Jovanovic 0001, Dragan Gasevic
LAK4
2021 Student appreciation of data-driven feedback: A pilot study on OnTask
abstract
Feedback plays a crucial role in student learning. Learning analytics (LA) has demonstrated potential in addressing prominent challenges with feedback practice, such as enabling timely feedback based on insights obtained from large data sets. However, there is insufficient research looking into relations between student expectations of feedback and their experience with LA-based feedback. This paper presents a pilot study that examined students’ experience of LA-based feedback, offered with the OnTask system, taking into consideration the factors of students’self-efficacy and self-regulation skills. Two surveys were carried out at a Brazilian university, and the results highlighted important implications for LA-based feedback practice, including leveraging the ‘partnership’ between the human teacher and the computer, and developing feedback literacy among learners.
Yi-Shan Tsai, Rafael Ferreira Leite de Mello, Jelena Jovanovic 0001, Dragan Gasevic
LAK3
2021 On the causal relation between real world activities and emotional expressions of social media users
abstract
Abstract Social interactions through online social media have become a daily routine of many, and the number of those whose real world (offline) and online lives have become intertwined is continuously growing. As such, the interplay of individuals' online and offline activities has been the subject of numerous research studies, the majority of which explored the impact of people's online actions on their offline activities. The opposite direction of impact—the effect of real‐world activities on online actions—has also received attention but to a lesser degree. To contribute to the latter form of impact, this paper reports on a quasi‐experimental design study that examined the presence of causal relations between real‐world activities of online social media users and their online emotional expressions. To this end, we have collected a large dataset (over 17K users) from Twitter and Foursquare, and systematically aligned user content on the two social media platforms. Users' Foursquare check‐ins provided information about their offline activities, whereas the users' expressions of emotions and moods were derived from their Twitter posts. Since our study was based on a quasi‐experimental design, to minimize the impact of covariates, we applied an innovative model of computing propensity scores. Our main findings can be summarized as follows: (a) users' offline activities do impact their affective expressions, both of emotions and moods, as evidenced in their online shared textual content; (b) the impact depends on the type of offline activity and if the user embarks on or abandons the activity. Our findings can be used to devise a personalized recommendation mechanism to help people better manage their online emotional expressions.
Seyed Amin Mirlohi Falavarjani, Jelena Jovanovic 0001, Hossein Fani 0001, Ali A. Ghorbani 0001, Zeinab Noorian, Ebrahim Bagheri
J. Assoc. Inf. Sci. Technol.2
2020 Neural Embedding-Based Metrics for Pre-retrieval Query Performance Prediction
Negar Arabzadeh, Fattane Zarrinkalam, Jelena Jovanovic 0001, Ebrahim Bagheri
ECIR (2)3
2020 Supporting actionable intelligence: reframing the analysis of observed study strategies
abstract
Models and processes developed in learning analytics research are increasing in sophistication and predictive power. However, the ability to translate analytic findings to practice remains problematic. This study aims to address this issue by establishing a model of learner behaviour that is both predictive of student course performance, and easily interpreted by instructors. To achieve this aim, we analysed fine grained trace data (from 3 offerings of an undergraduate online course, N=1068) to establish a comprehensive set of behaviour indicators aligned with the course design. The identified behaviour patterns, which we refer to as observed study strategies, proved to be associated with the student course performance. By examining the observed strategies of high and low performers throughout the course, we identified prototypical pathways associated with course success and failure. The proposed model and approach offers valuable insights for the provision of process-oriented feedback early in the course, and thus can aid learners in developing their capacity to succeed online.
Jelena Jovanovic 0001, Shane Dawson, Srecko Joksimovic, George Siemens
LAK1
2020 Analytics of learning strategies: the association with the personality traits
abstract
Studying online requires well-developed self-regulated learning skills to properly manage one's learning strategies. Learning analytics research has proposed novel methods for extracting theoretically meaningful learning strategies from trace data originating from formal learning settings (online, blended, or flipped classroom). Thus identified strategies proved to be associated with academic achievement. However, automated extraction of theoretically meaningful learning strategies from trace data in the context of massive open online courses (MOOCs) is still under-explored. Moreover, there is a lacuna in research on the relations between automatically detected strategies and the established psychological constructs. The paper reports on a study that (a) applied a state-of-the-art analytic method that combines process and sequence mining techniques to detect learning strategies from the trace data collected in a MOOC (N=1,397), and (b) explored associations of the detected strategies with academic performance and personality traits (Big Five). Four learning strategies detected with the adopted analytics method were shown to be theoretically interpretable as the well-known approaches to learning. The results also revealed that the four detected learning strategies were predicted by conscientiousness, emotional instability, and agreeableness and were associated with academic performance. Implications for theoretical validity and practical application of analytics-detected learning strategies are also provided.
Wannisa Matcha, Dragan Gasevic, Jelena Jovanovic 0001, Nora'ayu Ahmad Uzir, Chris W. Oliver, Andrew Murray, Danijela Gasevic
LAK3
2020 Intergroup and interpersonal forum positioning in shared-thread and post-reply networks
abstract
Network analysis has become a major approach for analysing social learning, used to capture learner positioning in online forum networks. LA research investigated the association between positioning in forum networks with academic performance and discourse quality, the latter two serving as proxies for learning. However, the research findings have been inconsistent, in part due to the discrepancies in the adopted approaches to network construction. Yet, it is still unclear how online forum networks should be modelled to assure that the learners' network positioning is properly captured. To address this gap, the current study explored if some existing approaches to network construction may complement each other and thus offer richer insights. In particular, we hypothesised that the post-reply learner network could represent interpersonal positioning, whereas the network based on co-participation in discussion threads could encapsulate intergroup positioning. The study used learner social interaction data from a large edX MOOC forum to examine the relationship between these two kinds of network positioning. The results suggest that intergroup and interpersonal positioning may capture different aspects of social learning, potentially related to different learning outcomes. We find that although interpersonal and intergroup positioning indicators covary, these measures are not congruent for some 37% of forum posters. Network coevolution analysis also reveals an interdependent relationship between the intergroup and interpersonal centrality in a forum network. Co-occurrence of learners in a discussion thread prior to direct exchanges is predictive of a direct post-reply interaction at a later stage of the course, and vice-versa, suggesting that intergroup positioning is a precursor of direct communication. The study contributes to the discussion around the definition of learner forum positioning in learning analytics, and validated approaches towards measuring it.
Oleksandra Poquet, Jelena Jovanovic 0001
LAK2
2020 Analytics of time management and learning strategies for effective online learning in blended environments
abstract
This paper reports on the findings of a study that proposed a novel learning analytics methodology that combines three complimentary techniques - agglomerative hierarchical clustering, epistemic network analysis, and process mining. The methodology allows for identification and interpretation of self-regulated learning in terms of the use of learning strategies. The main advantage of the new technique over the existing ones is that it combines the time management and learning tactic dimensions of learning strategies, which are typically studied in isolation. The new technique allows for novel insights into learning strategies by studying the frequency of, strength of connections between, and ordering and time of execution of time management and learning tactics. The technique was validated in a study that was conducted on the trace data of first-year undergraduate students who were enrolled into two consecutive offerings (N2017 = 250 and N2018 = 232) of a course at an Australian university. The application of the proposed technique identified four strategy groups derived from three distinct time management tactics and five learning tactics. The tactics and strategies identified with the technique were correlated with academic performance and were interpreted according to the established theories and practices of self-regulated learning.
Nora'ayu Ahmad Uzir, Dragan Gasevic, Jelena Jovanovic 0001, Wannisa Matcha, Lisa-Angelique Lim, Anthea Fudge
LAK3
2020 Neural embedding-based specificity metrics for pre-retrieval query performance prediction
Negar Arabzadeh, Fattane Zarrinkalam, Jelena Jovanovic 0001, Feras N. Al-Obeidat, Ebrahim Bagheri
Inf. Process. Manag.3
2019 On the causal relation between users' real-world activities and their affective processes
abstract
Research in social network analytics has already extensively explored how engagement on online social networks can lead to observable effects on users' real-world behavior (e.g., changing exercising patterns or dietary habits), and their psychological states. The objective of our work in this paper is to investigate the flip-side and examine whether engaging in or disengaging from real-world activities would reflect itself in users' affective processes such as anger, anxiety, and sadness, as expressed in users' posts on online social media. We have collected data from Foursquare and Twitter and found that engaging in or disengaging from a real-world activity, such as frequenting at bars or stopping going to a gym, have direct impact on the users' affective processes. In particular, we report that engaging in a routine real-world activity leads to expressing less emotional content online, whereas the reverse is observed when users abandon a regular real-world activity.
Seyed Amin Mirlohi Falavarjani, Ebrahim Bagheri, Ssu Yu Zoe Chou, Jelena Jovanovic 0001, Ali A. Ghorbani 0001
ASONAM4
2019 Geometric Estimation of Specificity within Embedding Spaces
abstract
Specificity is the level of detail at which a given term is represented. Existing approaches to estimating term specificity are primarily dependent on corpus-level frequency statistics. In this work, we explore how neural embeddings can be used to define corpus-independent specificity metrics. Particularly, we propose to measure term specificity based on the distribution of terms in the neighborhood of the given term in the embedding space. The intuition is that a term that is surrounded by other terms in the embedding space is more likely to be specific while a term surrounded by less closely related terms is more likely to be generic. On this basis, we leverage geometric properties between embedded terms to define three groups of metrics: (1) neighborhood-based, (2) graph-based and (3) cluster-based metrics. Moreover, we employ learning-to-rank techniques to estimate term specificity in a supervised approach by employing the three proposed groups of metrics. We curate and publicly share a test collection of term specificity measurements defined based on Wikipedia's category hierarchy. We report on our experiments through metric performance comparison, ablation study and comparison against the state-of-the-art baselines.
Negar Arabzadeh, Fattane Zarrinkalam, Jelena Jovanovic 0001, Ebrahim Bagheri
CIKM3
2019 Detection of Learning Strategies: A Comparison of Process, Sequence and Network Analytic Approaches
Wannisa Matcha, Dragan Gasevic, Nora'ayu Ahmad Uzir, Jelena Jovanovic 0001, Abelardo Pardo, Jorge Javier Maldonado Mahauad, Mar Pérez-Sanagustín
EC-TEL4
2019 Discovering Time Management Strategies in Learning Processes Using Process Mining Techniques
Nora'ayu Ahmad Uzir, Dragan Gasevic, Wannisa Matcha, Jelena Jovanovic 0001, Abelardo Pardo, Lisa-Angelique Lim, Sheridan Gentili
EC-TEL4
2019 Introducing meaning to clicks: Towards traced-measures of self-efficacy and cognitive load
abstract
The use of learning trace data together with various analytical methods has proven successful in detecting patterns in learning behaviour, identifying student profiles, and clustering learning resources. However, interpretation of the findings is often difficult and uncertain due to a lack of contextual data (e.g., data on student motivation, emotion or curriculum design). In this study we explored the integration of student self-reports about cognitive load and self-efficacy into the learning process and collection of relevant students' perceptions as learning traces. Our objective was to examine the association of traced measures of relevant learning constructs (cognitive load and self-efficacy) with i) indicators of the students' learning behaviour derived from trace data, and ii) the students' academic performance. The results indicated the presence of association between some indicators of students' engagement with learning activities and traced measures of cognitive load and self-efficacy. Correlational analysis demonstrated significant positive correlation between the students' course performance and traced measures of cognitive load and self-efficacy.
Jelena Jovanovic 0001, Dragan Gasevic, Abelardo Pardo, Shane Dawson, Alexander Whitelock-Wainwright
LAK1
2019 Analytics of Learning Strategies: Associations with Academic Performance and Feedback
abstract
Learning analytics has the potential to detect and explain characteristics of learning strategies through analysis of trace data and communicate the findings via feedback. However, the role of learning analytics-based feedback in selection and regulation of learning strategies is still insufficiently explored and understood. This research aims to examine the sequential and temporal characteristics of learning strategies and investigate their association with feedback. Three years of trace data were collected from online pre-class activities of a flipped classroom, where different types of feedback were employed in each year. Clustering, sequence mining, and process mining were used to detect and interpret learning tactics and strategies. Inferential statistics were used to examine the association of feedback with the learning performance and the detected learning strategies. The results suggest a positive association between the personalised feedback and the effective strategies.
Wannisa Matcha, Dragan Gasevic, Nora'ayu Ahmad Uzir, Jelena Jovanovic 0001, Abelardo Pardo
LAK4
2019 The reflection of offline activities on users' online social behavior: An observational study
Seyed Amin Mirlohi Falavarjani, Fattane Zarrinkalam, Jelena Jovanovic 0001, Ebrahim Bagheri, Ali A. Ghorbani 0001
Inf. Process. Manag.3
2018 A Comparison of Features for the Automatic Labeling of Student answers to Open-ended Questions
Jesus Gerardo Alvarado Mantecon, Hadi Abdi Ghavidel, Amal Zouaq, Jelena Jovanovic 0001, Jenny McDonald
EDM4
2018 UMLS to DBPedia link discovery through circular resolution
abstract
Objective: The goal of this work is to map Unified Medical Language System (UMLS) concepts to DBpedia resources using widely accepted ontology relations from the Simple Knowledge Organization System (skos:exactMatch, skos:closeMatch) and from the Resource Description Framework Schema (rdfs:seeAlso), as a result of which a complete mapping from UMLS (UMLS 2016AA) to DBpedia (DBpedia 2015-10) is made publicly available that includes 221 690 skos:exactMatch, 26 276 skos:closeMatch, and 6 784 322 rdfs:seeAlso mappings. Methods: We propose a method called circular resolution that utilizes a combination of semantic annotators to map UMLS concepts to DBpedia resources. A set of annotators annotate definitions of UMLS concepts returning DBpedia resources while another set performs annotation on DBpedia resource abstracts returning UMLS concepts. Our pipeline aligns these 2 sets of annotations to determine appropriate mappings from UMLS to DBpedia. Results: We evaluate our proposed method using structured data from the Wikidata knowledge base as the ground truth, which consists of 4899 already existing UMLS to DBpedia mappings. Our results show an 83% recall with 77% precision-at-one (P@1) in mapping UMLS concepts to DBpedia resources on this testing set. Conclusions: The proposed circular resolution method is a simple yet effective technique for linking UMLS concepts to DBpedia resources. Experiments using Wikidata-based ground truth reveal a high mapping accuracy. In addition to the complete UMLS mapping downloadable in n-triple format, we provide an online browser and a RESTful service to explore the mappings.
John Cuzzola, Ebrahim Bagheri, Jelena Jovanovic 0001
J. Am. Medical Informatics Assoc.3
2017 From prediction to impact: evaluation of a learning analytics retention program
abstract
Learning analytics research has often been touted as a means to address concerns regarding student retention outcomes. However, few research studies to date, have examined the impact of the implemented intervention strategies designed to address such retention challenges. Moreover, the methodological rigor of some of the existing studies has been challenged. This study evaluates the impact of a pilot retention program. The study contrasts the findings obtained by the use of different methods for analysis of the effect of the intervention. The pilot study was undertaken between 2012 and 2014 resulting in a combined enrolment of 11,160 students. A model to predict attrition was developed, drawing on data from student information system, learning management system interactions, and assessment. The predictive model identified some 1868 students as academically at-risk. Early interventions were implemented involving learning and remediation support. Common statistical methods demonstrated a positive association between the intervention and student retention. However, the effect size was low. The use of more advanced statistical methods, specifically mixed-effect methods explained higher variability in the data (over 99%), yet found the intervention had no effect on the retention outcomes. The study demonstrates that more data about individual differences is required to not only explain retention but to also develop more effective intervention approaches.
Shane Dawson, Jelena Jovanovic 0001, Dragan Gasevic, Abelardo Pardo
LAK2
2017 RysannMD: A biomedical semantic annotator balancing speed and accuracy
John Cuzzola, Jelena Jovanovic 0001, Ebrahim Bagheri
J. Biomed. Informatics2
2016 Generating actionable predictive models of academic performance
abstract
The pervasive collection of data has opened the possibility for educational institutions to use analytics methods to improve the quality of the student experience. However, the adoption of these methods faces multiple challenges particularly at the course level where instructors and students would derive the most benefit from the use of analytics and predictive models. The challenge lies in the knowledge gap between how the data is captured, processed and used to derive models of student behavior, and the subsequent interpretation and the decision to deploy pedagogical actions and interventions by instructors. Simply put, the provision of learning analytics alone has not necessarily led to changing teaching practices. In order to support pedagogical change and aid interpretation, this paper proposes a model that can enable instructors to readily identify subpopulations of students to provide specific support actions. The approach was applied to a first year course with a large number of students. The resulting model classifies students according to their predicted exam scores, based on indicators directly derived from the learning design.
Abelardo Pardo, Negin Mirriahi, Roberto Martínez-Maldonado, Jelena Jovanovic 0001, Shane Dawson, Dragan Gasevic
LAK4
2016 Textual Affect Communication and Evocation Using Abstract Generative Visuals
abstract
In order to facilitate interaction in computer-mediated communication and enrich user experience in general, we introduce a novel textual emotion visualization approach, grounded in generative art and evocative visuals. The approach is centered on the idea that affective computer systems should be able to relate to, communicate, and evoke human emotions. It maps emotions identified in the text to evocative abstract animation. We examined two visualizations based on our approach and two common textual emotion visualization techniques, chat emoticons and avatars, along three dimensions: emotion communication, emotion evocation, and overall user enjoyment. Our study, organized as repeated measures within-subject experiment, demonstrated that in terms of emotion communication, our visualizations are comparable with emoticons and avatars. However, our main visualization based on abstract color, motion, and shape proved to be the best in evoking emotions. In addition, in terms of the overall user enjoyment, it gave results comparable with emoticons, but better than avatars.
Uros Krcadinac, Jelena Jovanovic 0001, Vladan Devedzic, Philippe Pasquier
IEEE Trans. Hum. Mach. Syst.2
2015 Filtering Inaccurate Entity Co-references on the Linked Open Data
John Cuzzola, Ebrahim Bagheri, Jelena Jovanovic 0001
DEXA (1)3
2015 Lexical Semantic Relatedness for Twitter Analytics
abstract
Existing work in the semantic relatedness literature has already considered various information sources such as WordNet, Wikipedia and Web search engines to identify the semantic relatedness between two words. We will show that existing semantic relatedness measures might not be directly applicable to microblogging content such as tweets due to i) the informality and short length of microblogging content, which can lead to shift in the meaning of words when used in microblog posts, ii) the presence of non-dictionary words that have their semantics defined/evolved by the Twitter community. Therefore, we propose the Twitter Space Semantic Relatedness (TSSR) technique that relies on the latent relation hypothesis to measure semantic relatedness of words on Twitter. We construct a graph representation of terms in tweets and apply a random walk procedure to produce a stationary distribution for each word, which is the basis for relatedness calculation. Our experiments examine TSSR from three different perspectives and show that TSSR is better suited for Twitter analytics compared to the standard semantic relatedness techniques.
Hossein Fani 0001, Ebrahim Bagheri, Jelena Jovanovic 0001
ICTAI4
2015 2nd int'l workshop on open badges in education (OBIE 2015): from learning evidence to learning analytics
abstract
Open digital badges are Web-enabled tokens of learning and accomplishment. Unlike traditional grades, certificates, and transcripts, badges include specific claims about learning accomplishments and detailed evidence in support of those claims. Considering the richness of data associated with Open Badges, it is reasonable to expect a very powerful predictive element at the intersection of Open Badges and Learning Analytics. This could have substantial implications for recommending and exposing students to a variety of curricular and co-curricular pathways utilizing data sources far more nuanced than grades and achievement tests. Therefore, this workshop was aimed at: i) examining the potentials of Open Badges (including the associated data and resources) to provide new and potentially unprecedented data for analysis; ii) examining the kinds of Learning Analytics methods and techniques that could be suitable for gaining valuable insights from and/or making predictions based on the evidence (data and resources) associated with badges, and iii) connecting Open Badges communities, aiming to allow for the exchange of experiences and learning from different cultures and communities.
Daniel T. Hickey, Jelena Jovanovic 0001, Steven Lonn, James E. Willis
LAK2
2015 What do cMOOC participants talk about in social media?: a topic analysis of discourse in a cMOOC
abstract
Creating meaning from a wide variety of available information and being able to choose what to learn are highly relevant skills for learning in a connectivist setting. In this work, various approaches have been utilized to gain insights into learning processes occurring within a network of learners and understand the factors that shape learners' interests and the topics to which learners devote a significant attention. This study combines different methods to develop a scalable analytic approach for a comprehensive analysis of learners' discourse in a connectivist massive open online course (cMOOC). By linking techniques for semantic annotation and graph analysis with a qualitative analysis of learner-generated discourse, we examined how social media platforms (blogs, Twitter, and Facebook) and course recommendations influence content creation and topics discussed within a cMOOC. Our findings indicate that learners tend to focus on several prominent topics that emerge very quickly in the course. They maintain that focus, with some exceptions, throughout the course, regardless of readings suggested by the instructor. Moreover, the topics discussed across different social media differ, which can likely be attributed to the affordances of different media. Finally, our results indicate a relatively low level of cohesion in the topics discussed which might be an indicator of a diversity of the conceptual coverage discussed by the course participants.
Srecko Joksimovic, Vitomir Kovanovic, Jelena Jovanovic 0001, Amal Zouaq, Dragan Gasevic, Marek Hatala
LAK3
2015 Evolutionary fine-tuning of automated semantic annotation systems
John Cuzzola, Jelena Jovanovic 0001, Ebrahim Bagheri, Dragan Gasevic
Expert Syst. Appl.2
2015 A comparative study of software tools for user story management
Sonja Dimitrijevic, Jelena Jovanovic 0001, Vladan Devedzic
Inf. Softw. Technol.2
2015 Comprehension and Learning of Social Goals Through Visualization
abstract
The concept of social goals refers to organizational goals that are defined in an open and transparent manner; they serve as social objects that incite both formal and informal collaboration around shared interests/objectives. Our objective is to facilitate the comprehension of social goals and examine the role of social goals as scaffolds of social learning in an organization. To this end, we followed an approach based on the visualization of social goals and explored how different presentations of goals, specifically, faceted goal browsing, graph-based visualization, and timeline-based visualization, contribute to the realization of the stated objective. To assess this approach, we conducted a between subjects study where each participant performed a set of goal comprehension tasks with one of the examined presentations of goals. The study demonstrated that our visualizations of goals increase the accuracy of the overall comprehension of an organization's goals; this positive effect is also present when the comprehension of relationships-either explicit or implicit ties-among social goals is needed. The results also confirmed that our graph-based visualization of social goals could serve as a facilitator of social learning in an organization.
Jelena Jovanovic 0001, Ebrahim Bagheri, Dragan Gasevic
IEEE Trans. Hum. Mach. Syst.1
2013 An empirical evaluation of ontology-based semantic annotators
abstract
One of the most important prerequisites for achieving the Semantic Web vision is semantic annotation of data/resources. Semantic annotation enriches unstructured and/or semistructured content with a context that is further linked to the structured domain-specific knowledge. In particular, ontologybased semantic annotators enable the selection of a specific ontology to annotate content. This paper presents results of an empirical study of recent ontology-based annotators, namely Stanbol, KIM, and SDArch. Specifically, we evaluated the robustness of these annotators with respect to specific features of ontology concepts such as the length of concepts? labels and their linguistic categories (e.g., prepositions and conjunctions). Our results show that although significantly correlated according to most of the conducted evaluations, tools still exhibit their unique features that could be a topic of new research.
Srecko Joksimovic, Jelena Jovanovic 0001, Dragan Gasevic, Amal Zouaq, Zoran Jeremic
K-CAP2
2013 Synesketch: An Open Source Library for Sentence-Based Emotion Recognition
abstract
Online human textual interaction often carries important emotional meanings inaccessible to computers. We propose an approach to textual emotion recognition in the context of computer-mediated communication. The proposed recognition approach works at the sentence level and uses the standard Ekman emotion classification. It is grounded in a refined keyword-spotting method that employs: a WordNet-based word lexicon, a lexicon of emoticons, common abbreviations and colloquialisms, and a set of heuristic rules. The approach is implemented through the Synesketch software system. Synesketch is published as a free, open source software library. Several Synesketch-based applications presented in the paper, such as the the emotional visual chat, stress the practical value of the approach. Finally, the evaluation of the proposed emotion recognition algorithm shows high accuracy and promising results for future research and applications.
Uros Krcadinac, Philippe Pasquier, Jelena Jovanovic 0001, Vladan Devedzic
IEEE Trans. Affect. Comput.3
2012 Students Online Interaction in a Blended Learning Environment - A Case Study of the First Experience in using an LMS
Ivana Mijatovic, Jelena Jovanovic 0001, Sandra Jednak
CSEDU (2)2
2012 1st International Workshop on Learning Analytics and Linked Data
abstract
The main objective of the 1st International Workshop on Learning Analytics and Linked Data (#LALD2012) is to connect the research efforts on Linked Data and Learning Analytics in order to create visionary ideas and foster synergies between the two young research fields. Therefore, the workshop will collect, explore, and present datasets, technologies and applications for Technology Enhanced Learning (TEL) to discuss Learning Analytics approaches that make use of educational data or Linked Data sources. During the workshop, an overview of available educational datasets and related initiatives will be given. The participants will have the opportunity to present their own research with respect to educational datasets, technologies and applications and discuss major challenges to collect, reuse, and share these datasets.
Hendrik Drachsler, Stefan Dietze, Wolfgang Greller, Mathieu d'Aquin, Jelena Jovanovic 0001, Abelardo Pardo, Wolfgang Reinhardt 0001, Katrien Verbert
LAK5
2012 Learn-B: a social analytics-enabled tool for self-regulated workplace learning
abstract
In this design briefing, we introduce the Learn-B environment, our attempt in designing and implementing a research prototype to address some of the challenges inherent in workplace learning: the informal aspect of workplace learning requires knowledge workers to be supported in their self-regulatory learning (SRL) processes, whilst its social nature draws attention to the role of collective in those processes. Moreover, learning at workplace is contextual and on-demand, thus requiring organizations to recognize and motivate the learning and knowledge building activities of their employees, where individual learning goals are harmonized with those of the organization. In particular, we focus on the analytics-based features of Learn-B, illustrate their design and current implementation, and discuss how each of them is hypothesized to target the above challenges.
Melody Siadaty, Dragan Gasevic, Jelena Jovanovic 0001, Nikola Milikic, Zoran Jeremic, Aleksandar Giljanovic, Marek Hatala
LAK3
2012 Student modeling and assessment in intelligent tutoring of software patterns
Zoran Jeremic, Jelena Jovanovic 0001, Dragan Gasevic
Expert Syst. Appl.2
2012 Adaptive neuro-fuzzy pedagogical recommender
Zoran Sevarac, Vladan Devedzic, Jelena Jovanovic 0001
Expert Syst. Appl.3
2011 Self-regulated Learners and Collaboration: How Innovative Tools Can Address the Motivation to Learn at the Workplace?
Teresa Schäfer, Claudia Magdalena Fabian, Melody Siadaty, Jelena Jovanovic 0001, Kai Pata, Dragan Gasevic
EC-TEL4
2011 Linked Data Metrics for Flexible Expert Search on the Open Web
Milan Stankovic, Jelena Jovanovic 0001, Philippe Laublet
ESWC (1)2
2011 A Semantic Web-enabled Tool for Self-Regulated Learning in the Workplace
abstract
Self-regulated learning processes have a potential to enhance the motivation of knowledge workers to take part in learning and knowledge building activities, and thus contribute to the resolution of an important research challenge in workplace learning. An equally important research challenge for successful completion of each step of a self-regulatory process is to enable learners to be aware of characteristics of their organizationally embedded learning context. In this paper, we describe how a combination of pedagogy and Semantic Web-based technologies can be utilized to address the above two challenges. Specifically, we demonstrate the proposed solution through the Learning Pal tool which leverages ontologies to support self-regulation in organizational learning.
Melody Siadaty, Jelena Jovanovic 0001, Kai Pata, Teresa Schäfer, Dragan Gasevic, Nikola Milikic
ICALT2
2010 Leveraging Semantic Technologies for Harmonization of Individual and Organizational Learning
Melody Siadaty, Jelena Jovanovic 0001, Dragan Gasevic, Zoran Jeremic, Teresa Schäfer
EC-TEL2
2009 Can Educators Develop Ontologies Using Ontology Extraction Tools: An End-User Study
Marek Hatala, Dragan Gasevic, Melody Siadaty, Jelena Jovanovic 0001, Carlo Torniai
EC-TEL4
2009 Project-Based Collaborative Learning Environment with Context-Aware Educational Services
Zoran Jeremic, Jelena Jovanovic 0001, Dragan Gasevic, Marek Hatala
EC-TEL2
2009 Semantically-Enabled Project-Based Collaborative Learning of Software Patterns
abstract
Teaching and learning software design patterns (DPs) is not an easy task. Apart from learning individual DPs and the principle behind them, students should learn how to apply them in real-life situations. Therefore, to make the learning process of DPs effective, it is necessary to include a project component in which students, usually in small teams, develop a medium-sized software application. Following this paradigm, and using active learning techniques, project-based learning (PBL) and collaborative learning (CL), we have developed a learning environment for software DPs which leverages semantic technologies to integrate several existing learning systems and tools.
Zoran Jeremic, Jelena Jovanovic 0001, Dragan Gasevic
ICALT2
2009 Semantic Web Technologies for the Integration of Learning Tools and Context-Aware Educational Services
Zoran Jeremic, Jelena Jovanovic 0001, Dragan Gasevic
ISWC2
2008 Towards a Semantic-Rich Collaborative Environment for Learning Software Patterns
Zoran Jeremic, Jelena Jovanovic 0001, Dragan Gasevic
EC-TEL2
2008 Semantic Technologies for Socially-Enhanced Context-Aware Mobile Learning
Melody Siadaty, Ty Mey Eap, Jelena Jovanovic 0001, Dragan Gasevic, Carlo Torniai, Marek Hatala
EC-TEL3
2008 A Semantic-Rich Framework for Learning Software Patterns
abstract
Current approaches to learning software patterns are based on individual use of different learning systems and tools. With this ‘fragmented’ approach it is very hard to provide support for context-aware learn-ing and offer personalized learning experience to students. In this paper, we propose a new approach to learning software patterns that integrates existing Learning Management Systems, domain specific tools for software modeling and relevant online repositories of software patterns into a complex learning framework that supports collaborative learning. This framework is based on the semantic web technologies.
Zoran Jeremic, Jelena Jovanovic 0001, Dragan Gasevic
ICALT2
2008 E-Learning meets the Social Semantic Web
abstract
The social semantic Web has recently emerged as a paradigm in which ontologies (aimed at defining, structuring and sharing information) and collaborative software (used for creating and sharing knowledge) have been merged together. Ontologies provide an effective means of capturing and integrating knowledge for feedback provisioning, while using collaborative activities can support pedagogical theories, such as social constructivism. Both technologies have developed separately in the e-learning domain; representing respectively a teacher-centered and a learner-centered approach for learning environments. In this paper we bridge the gap between these two approaches by leveraging the social semantic Web paradigm, and propose a collaborative semantic-rich learning environment in which folksonomies created from studentspsila collaborative tags contribute to ontology maintenance, and teacher-directed feedback.
Carlo Torniai, Jelena Jovanovic 0001, Dragan Gasevic, Scott Bateman, Marek Hatala
ICALT2
2008 Leveraging the Social Semantic Web in Intelligent Tutoring Systems
Jelena Jovanovic 0001, Carlo Torniai, Dragan Gasevic, Scott Bateman, Marek Hatala
Intelligent Tutoring Systems1
2007 LOCO-Analyst: A Tool for Raising Teachers' Awareness in Online Learning Environments
Jelena Jovanovic 0001, Dragan Gasevic, Christopher Brooks 0001, Vladan Devedzic, Marek Hatala
EC-TEL1
2007 Leveraging the Semantic Web for Providing Educational Feedback
abstract
In our previous work, we developed the LOCO ontology framework which formalizes the notion of learning object context as a complex interplay of learning activities, learning objects, and learners. We now use that framework in conjunction with semantic annotation to generate different kinds of feedback (which we had identified by interviewing several Web educators) for educators to help them improve the learning process in Web-based settings. To test the feasibility of the proposed approach for feedback provision we developed a tool named LOCO-Analyst. Here we report on our experiences in developing LOCO-Analyst and using it to generate feedback out of the real data obtained from the iHelp Courses Learning Content Management System. Finally, we present evaluation results.
Jelena Jovanovic 0001, Dragan Gasevic, Christopher Brooks 0001, Ty Mey Eap, Vladan Devedzic, Marek Hatala, Griff Richards
ICALT1
2007 Ontology-based content model for scalable content reuse
abstract
The paper presents Abstract Compound Content Model (ACCM), a generic content model which we have devel-oped aiming to facilitate interoperability, repurposing and integration of diverse platform specific content models. Based on this model we have developed the ACCM ontology in order to turn the ACCM's elements (i.e., content units and content aggregations) into resources that can be directly accessed and thus reused. The paper also presents our current work on the implementation of an ACCM-based content management system that enables efficient storage, indexing, search and retrieval of content units as they are defined in the ACCM ontology.
Sasa Nesic, Jelena Jovanovic 0001, Dragan Gasevic, Mehdi Jazayeri
K-CAP2
2006 Ontologies to Support Learning Design Context
Jelena Jovanovic 0001, Dragan Gasevic, Christopher Brooks 0001, Colin Knight, Griff Richards, Gordon I. McCalla
EC-TEL1
2006 Dynamic Assembly of Personalized Learning Content on the Semantic Web
Jelena Jovanovic 0001, Dragan Gasevic, Vladan Devedzic
ESWC1
2006 Learning Object Context on the Semantic Web
abstract
The paper presents an ontology-based framework for capturing learning context related information important for personalization of both learning objects (LOs) and learning designs (LDs). The central part of the framework is the LO Context ontology, that bridges a learning content ontology and a LD ontology. The LO context ontology is aimed at capturing information about the actual usage of a LO inside a LD, such as the learning activity the LO was used in, the pedagogical role assumed by the LO (e.g. exercise), the learner’s features (represented in the form of the learner model) and the like. Furthermore we suggest the architecture of an adaptive educational system that leverages the proposed approach to enable personalization and reuse of LOs and LDs.
Jelena Jovanovic 0001, Colin Knight, Dragan Gasevic, Griff Richards
ICALT1
2006 JavaDON: an open-source expert system shell
Bojan Tomic, Jelena Jovanovic 0001, Vladan Devedzic
Expert Syst. Appl.2
2006 Ontology-Based Automatic Annotation of Learning Content
abstract
This paper presents an ontology-based approach to automatic annotation of learning objects’ (LOs) content units that we tested in TANGRAM, an integrated learning environment for the domain of Intelligent Information Systems. The approach does not primarily focus on automatic annotation of entire LOs, as other relevant solutions do. Instead, it provides a solution for automatic metadata generation for LOs’ components (i.e., smaller, potentially reusable, content units). Here we mainly report on the content-mining algorithms and heuristics applied for determining values of certain metadata elements used to annotate content units. Specifically, the focus is on the following elements: title, description, unique identifier, subject (based on a domain ontology), and pedagogical role (based on an ontology of pedagogical roles). Additionally, as TANGRAM is grounded on an LO content structure ontology that drives the process of an LO decomposition into its constituent content units, each thus generated content unit is implicitly semantically annotated with its role/position in the LO’s structure. Employing such semantic annotations, TANGRAM allows assembling content units into new LOs personalized to the users’ goals, preferences, and learning styles. In order to provide the evaluation of the proposed solution, we describe our experiences with automatic annotation of slide presentations, one of the most common LO types.
Jelena Jovanovic 0001, Dragan Gasevic, Vladan Devedzic
Int. J. Semantic Web Inf. Syst.1
2005 Ontology of Learning Object Content Structure
Jelena Jovanovic 0001, Dragan Gasevic, Katrien Verbert, Erik Duval
AIED1
2005 Ontologies for Reusing Learning Object Content
abstract
The paper proposes a framework for building learning object (LO) content using ontologies. In the previous work on using ontologies to describe LOs, researchers employed ontologies exclusively for describing LOs' metadata. Although such an approach is useful for searching for LOs in LO Repositories, it does not provide us with features to reuse components of LOs, nor to incorporate an explicit specification of domain semantics into LO content. We propose the use of two kinds of ontologies as a solution to this problem: content structure ontologies and domain ontologies.
Dragan Gasevic, Jelena Jovanovic 0001, Vladan Devedzic, Marko Boskovic
ICALT2
2005 Achieving knowledge interoperability: An XML/XSLT approach
Jelena Jovanovic 0001, Dragan Gasevic
Expert Syst. Appl.1
2004 Enhancing Learning Object Content on the Semantic Web
abstract
This paper gives a proposal to enhance learning object (LO) content using ontologies and semantic Web languages. In the previous work on using ontologies to describe LOs, researchers have built ontologies for description of metadata. However, these ontologies do not improve an LO's content. We suggest creating LOs that have content marked up in accordance with domain ontologies. Accordingly, LOs can be used not only as learning materials, but they can also be used in real-world applications.
Dragan Gasevic, Jelena Jovanovic 0001, Vladan Devedzic
ICALT2
2004 Ontologies for Creating Learning Object Content
Dragan Gasevic, Jelena Jovanovic 0001, Vladan Devedzic
KES2
2004 A GUI for Jess
Jelena Jovanovic 0001, Dragan Gasevic, Vladan Devedzic
Expert Syst. Appl.1