Jay Holland

dblp:76/5866 · DBLP profile ↗
← Back
21ranked-venue papers
1as first author
12since 2021 · last 2025
0000-0002-2314-8601ORCID · corroborated

Domains — the database's venue-derived domains; a paper can count in several

Applied, interdisciplinary, general and emerging computing · 17 · 1 first-author · 9 since 2021Human-computer interaction and ubiquitous computing · 8 · 1 first-author · 2 since 2021Software engineering, systems software and programming languages · 2 · 2 since 2021Artificial intelligence and machine learning · 1
YearPublicationVenuePosition
2025 Fostering Interactive Engagement in Active Video Watching via Adaptive Comment Recommendations
Ehsan Bojnordi, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland
AIED (6)5
2025 Video-Based Empathy Training for Software Engineers
abstract
Empathy, i.e., the ability to understand and feel what others are going through, is essential for value-based and user-centered software development. Empathy helps software engineers fully understand client needs, but also impacts how software engineers work with each other (e.g., within their team). However, junior and less experienced software engineers may not always understand what empathy means and why it matters in a technical domain like software development (and therefore do not pursue opportunities to develop it). We present a video-based training technique for empathy of software engineers. We also show preliminary findings of using the technique in a software engineering project course for second-year software engineering students. We report on student learning, engagement, as well as the perceptions of students on the training technique.
Antonija Mitrovic, Matthias Galster, Sanna Malinen, Sreedevi Sankara Iyer, Raul Vincent W. Lumapas, Negar Mohammadhassan, Jay Holland
CSEE&T7
2024 Personalized Comment Reviewing in Active Video Watching: Investigation of Learners' Cognitive Load
abstract
In the context of video-based learning, particularly active video watching, social learning is facilitated by allowing students to review comments written by their peers. In previous studies with AVW-Space, all students had access to the same comments, despite differences in their knowledge. This paper presents a study in which each student received a personalized list of comments to review, based on their student model. The results show that our intervention encouraged learners to think deeper about the comments during reviewing comments.
Ehsan Bojnordi, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland
ICCE5
2024 Enhancing Social Learning in Active Video Watching
abstract
To learn effectively by watching videos, learners need to engage actively with video content. Writing comments on videos (also known as video annotation) is a common way of engagement in Active Video Watching (AVW). Reviewing comments on videos written by peers, as a form of social learning, has also been shown to increase learning. In this paper, we extended social learning in AVW by enabling learners to respond to comments written by their peers (rather than just reviewing peer comments) and replacing a categorical comment rating design with a numerable binary one. We investigated the impact of using such a form of comment reviewing on learning, engagement, and students' perceptions. The findings show that our intervention results in increased learning, engagement and student satisfaction compared to the situation when students could not respond to comments.
Ehsan Bojnordi, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland, Negar Mohammadhassan
ICCE5
2024 Graduate School of Informatics, Kyoto University
abstract
Recent studies on Explainable Artificial Intelligence (XAI) in education show benefits for student learning. However, integrating XAI in AI-based education (AIED) systems requires understanding students' explanation needs. Some approaches to adding XAI to AIED systems include participatory design and co-design involving learners. This study presents a participatory approach to implement explanations in Active Video Watching (AVW). We designed explanations based on the requirements on timing and presentation of explanations and additional feedback from learners during the participatory activity. The implemented explanations support students who made low to medium-quality comments on video content by explaining how comment quality was determined. Furthermore, explanations included recommendations to improve future comments. We present the results of a pilot study on explanations in an AVW platform.
Raul Vincent W. Lumapas, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Pasan Peiris, Jay Holland
ICCE6
2023 Adding Interactive Mode to Active Video Watching
abstract
Presentation skills are crucial for tertiary students and graduates but are difficult to teach. We augmented active video watching (AVW) approach with the possibility of interactions among students, and conducted an experiment with AVW- Space, an online platform which supports video-based learning. The participants watched and commented on videos first. In the second phase, the participants reviewed, rated, and responded to their peers’ comments. We found that students who interacted with other students and responded to their comments increased their conceptual understanding of presentation skills.
Ehsan Bojnordi, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland
ICCE5
2023 Evaluating the Assessment of Comment Quality in Learning Communication Skills using Active Video Watching
abstract
Supporting student engagement remains one of the key challenges in video-based learning. This challenge is addressed by active video watching (AVW), a learning approach that supports engagement through different interventions, such as note-taking in the form of comments that learners submit while watching videos. One platform to support AVW is AVW-Space. Previous studies on AVW-Space detail improvements in the system, such as the integration of Artificial Intelligence and Machine Learning (ML) models in the comment feature of the system. This study investigates two machine learning models used to automatically assess the quality of comments when learning communication skills via AVW. One model is generated based on a large set of comments created by students when engaging with videos about presentation skills. For this study, a new model is developed from comments that students submitted when engaging with videos about communication skills. Results show that the new model, which was created from data on communication skills, performed better when assessing comments for communication skills compared to the model generated from comments for another skill. This has been demonstrated by the higher value of inter-rater agreement with the comment quality assessment made by human coders.
Raul Vincent W. Lumapas, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland, Negar Mohammadhassan
ICCE5
2023 Question-Driven Design Process for XAI in Active Video Watching
abstract
Designing explanations for Artificial Intelligence (AI) systems continues to be a challenge due to AI's black-box nature. Among the solutions developed to help in designing explanations in AI technologies is the Question-Driven Design Process for Explainable Artificial Intelligence (XAI) User Experience. In this paper, we report on our experiences using the question-driven design process for XAI in active video watching. We used Active Video Watching (AVW)-Space, an AVW platform developed at the University of Canterbury, as the context for AVW. In the question analysis process, we elicited questions from users on the AI features of the system. We conducted a survey to elicit questions from users on the AI features of the system. We conducted a survey to elicit three human raters categorized the user questions into the different XAI bank categories. Results show that most users tend to ask "how" and "why" questions about the AI-enabled features in the platform. The results of the question analysis will be used in mapping the determined question categories to potential XAI techniques. This can help in deciding the types of explanations to provide to users of AVW in future works on XAI in active video watching.
Raul Vincent W. Lumapas, Antonija Mitrovic, Matthias Galster, Sanna Malinen, Pasan Peiris, Jay Holland
ICCE6
2023 Soft skills required from software professionals in New Zealand
abstract
Soft skills (e.g., communication) significantly contribute to software project success. We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to characteristics of hiring organizations, and (c) how reliably we can automatically identify soft skills in job adverts to support their continuous analysis. We focus on soft skills required by organizations in New Zealand, a country with a small but growing software sector characterized by a skills shortage, reliance on offshoring, and embedded in a bi-cultural context. We manually analyzed 530 job adverts from New Zealand’s largest portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without pre-defined soft skills. We complemented the manual analysis with an automated analysis using Flexiterm (an approach for term recognition). We found explicit references to soft skills in 82% of adverts. Adverts from recruitment agencies (compared to hiring companies) included fewer soft skills. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related skills are most in demand. Soft skills related to broader human or societal values (e.g., empathy, cultural awareness) or distributed development are not common. Soft skills do not depend on company size or core business and domain of companies, or whether a company operates globally. Automatically identifying soft skills in adverts is error-prone. Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. On the other hand, identified soft skills only partially overlap with those reported in other skills classifications. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.
Matthias Galster, Antonija Mitrovic, Sanna Malinen, Jay Holland, Pasan Peiris
Inf. Softw. Technol.4
2023 Effectiveness of Video-based Training for Face-to-face Communication Skills of Software Engineers: Evidence from a Three-year Study
abstract
Objectives. Communication skills are crucial for effective software development teams, but those skills are difficult to teach. The goal of our project is to evaluate the effectiveness of teaching face-to-face communication skills using AVW-Space, a platform for video-based learning that provides personalized nudges to support student's engagement during video watching. Participants. The participants in our study are second-year software engineering students. The study was conducted over three years, with students enrolled in a semester-long project course. Study Method. We performed a quasi-experimental study over three years to teach face-to-face communication using AVW-Space, a video-based learning platform. We present the instance of AVW-Space we developed to teach face-to-face communication. Participants watched and commented on 10 videos and later commented on the recording of their own team meeting. In 2020, the participants ( n = 50) did not receive nudges, and we use the data collected that year as control. In 2021 ( n = 49) and 2022 ( n = 48), nudges were provided adaptively to encourage students to write more and higher-quality comments. Findings. The findings from the study show the effectiveness of nudges. We found significant differences in engagement when nudges were provided. Furthermore, there is a causal effect of nudges on the interaction time, the total number of comments written, and the number of high-quality comments, as well as on learning. Finally, participants exposed to nudges reported higher perceived learning. Conclusions. Our research shows the effect of nudges on student engagement and learning while using the instance of AVW-Space for teaching face-to-face communication skills. Future work will explore other soft skills, as well as providing explanations for the decisions made by AVW-Space.
Antonija Mitrovic, Matthias Galster, Sanna Malinen, Jay Holland, Ja'afaru Musa, Negar Mohammadhassan, Raul Vincent W. Lumapas
ACM Trans. Comput. Educ.4
2022 What Soft Skills Does the Software Industry *Really* Want? An Exploratory Study of Software Positions in New Zealand
abstract
Background: Soft skills of software professionals (e.g., communication, interpersonal skills) significantly contribute to project and product success. Aims: We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to types of software engineering positions, and (c) how soft skills relate to characteristics of hiring organizations. We focus on organizations in New Zealand, a country with a relatively small but growing software sector characterized by a skills shortage and embedded in a bi-cultural context. Method: We used a qualitative research method and manually analyzed 530 job adverts from New Zealand’s largest job portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without a pre-defined set of soft skills. Results: We found explicit references to soft skills in 82% of adverts. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related soft skills are most in demand, regardless of the type of position. Soft skills related to broader human or societal values (e.g., empathy or cultural awareness) or distributed development are not frequently requested. Soft skills do not depend on company size or core business. Conclusions: Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.
Matthias Galster, Antonija Mitrovic, Sanna Malinen, Jay Holland
ESEM4
2022 How Much Support is Necessary for Self-Regulated Learning?
Antonija Mitrovic, Jay Holland
ICCE2
2020 Effect of Non-mandatory Use of an Intelligent Tutoring System on Students' Learning
Antonija Mitrovic, Jay Holland
AIED (1)2
2016 A Virtual Reality Environment for Rehabilitation of Prospective Memory in Stroke Patients
abstract
Prospective Memory (PM), or remembering to perform actions in the future, is of crucial importance for everyday life. This kind of memory is often impaired in stroke survivors and can interfere with independent living. We have developed a computer-based treatment which uses visual imagery to teach participants how to remember time- and event-based prospective memory tasks better. After the treatment, participants practiced their PM skills using videos first, and later in a Virtual Reality (VR) environment. The VR environment uses Constraint-Based Modeling (CBM) to track the user actions and provide individual feedback. We report on a study with 15 stroke survivors, which shows that our treatment is highly effective.
Moffat Mathews, Antonija Mitrovic, Stellan Ohlsson, Jay Holland, Audrey McKinley
KES4
2014 From Tutoring to Cognitive Rehabilitation: Exploiting CBM to Support Memory Training
abstract
Constraint-Based Modeling (CBM) is an effective student modeling approach which has been used successfully in a wide range of instructional domains. Within the Intelligent Computer Tutoring Group (ICTG), we have developed numerous constraint-based tutors and demonstrated their effectiveness in real courses. In this paper, however, we discuss how we use CBM in the area of cognitive rehabilitation after stroke. Our computer-based treatment is aimed at improving prospective memory. Participants are first trained on how to use visual imagery and then practice in a Virtual Reality (VR) environment. We present how we use constraints to track the participant’s progress when performing tasks in the VR environment.
Antonija Mitrovic, Moffat Mathews, Stellan Ohlsson, Jay Holland, Audrey McKinlay, Scott Ogden, Anthony Bracegirdle, Sam Dopping-Hepenstal
ICCE4
2013 The Effect of Interaction Granularity on Learning with a Data Normalization Tutor
Amali Weerasinghe, Antonija Mitrovic, Amir Shareghi Najar, Jay Holland
AIED4
2012 Supporting Self-Directed Learning Skills in Learning Management Systems
Amali Weerasinghe, Antonija Mitrovic, Moffat Mathews, Jay Holland, Myse Elmadani
ICCE4
2012 Do Your Eyes Give It Away? Using Eye Tracking Data to Understand Students' Attitudes towards Open Student Model Representations
Moffat Mathews, Antonija Mitrovic, Bin Lin 0006, Jay Holland, Neville Churcher
ITS4
2012 Exploring Two Strategies for Teaching Procedures
Antonija Mitrovic, Moffat Mathews, Jay Holland
ITS3
2011 The Effects of Domain and Collaboration Feedback on Learning in a Collaborative Intelligent Tutoring System
Jay Holland, Nilufar Baghaei, Moffat Mathews, Antonija Mitrovic
AIED1
2006 Authoring Constraint-Based Tutors in ASPIRE
Antonija Mitrovic, Pramuditha Suraweera, Brent Martin, Konstantin Zakharov, Nancy Milik, Jay Holland
Intelligent Tutoring Systems6