Students working in teams in software engineering group project often communicate ineffectively, which reduces the quality of deliverables, and is therefore detrimental for project success. An important step towards addressing areas of improvement is identifying which changes to communication will improve team performance the most. We applied two different communication analysis techniques, triad census and socio-technical congruence, to data gathered from a twosemester software engineering group project. Triad census uses the presence of edges between groups of three nodes as a measure of network structure, while socio-technical congruence compares the fit of a team's communication to their technical dependencies. Our findings suggest that each team's triad census for a given sprint is promising as a predictor of the percentage of story points they pass, which is closely linked to project success. Meanwhile, socio-technical congruence is inadequate as the sole metric for predicting project success in this context. We discuss these findings, and their potential applications improve communication in a software engineering group project.

Empathy, i.e., the ability to understand and feel what others are going through, is essential for value-based and user-centered software development. Empathy helps software engineers fully understand client needs, but also impacts how software engineers work with each other (e.g., within their team). However, junior and less experienced software engineers may not always understand what empathy means and why it matters in a technical domain like software development (and therefore do not pursue opportunities to develop it). We present a video-based training technique for empathy of software engineers. We also show preliminary findings of using the technique in a software engineering project course for second-year software engineering students. We report on student learning, engagement, as well as the perceptions of students on the training technique.

In the context of video-based learning, particularly active video watching, social learning is facilitated by allowing students to review comments written by their peers. In previous studies with AVW-Space, all students had access to the same comments, despite differences in their knowledge. This paper presents a study in which each student received a personalized list of comments to review, based on their student model. The results show that our intervention encouraged learners to think deeper about the comments during reviewing comments.

To learn effectively by watching videos, learners need to engage actively with video content. Writing comments on videos (also known as video annotation) is a common way of engagement in Active Video Watching (AVW). Reviewing comments on videos written by peers, as a form of social learning, has also been shown to increase learning. In this paper, we extended social learning in AVW by enabling learners to respond to comments written by their peers (rather than just reviewing peer comments) and replacing a categorical comment rating design with a numerable binary one. We investigated the impact of using such a form of comment reviewing on learning, engagement, and students' perceptions. The findings show that our intervention results in increased learning, engagement and student satisfaction compared to the situation when students could not respond to comments.

Active Video Watching supports engagement through scalable interventions, such as notetaking in the form of comments. Machine Learning is used to categorize comments based on their quality to provide personalized feedback to students. In previous work on AVW-Space, an online portal for active video watching, a machine learning model was trained using data from several studies on presentation skills. In this paper, we explore the effectiveness in assessing the comment quality of this model in Face-to-Face Meeting Communication skills in comparison to a model trained specifically for this soft skill. We used Explainable Artificial Intelligence to identify and compare the important features of the models. Results show the need for comment quality assessment models to be specific to the soft skill in question and show major differences between their important features, highlighting the necessity to create a model specific to a particular soft skill.

Recent studies on Explainable Artificial Intelligence (XAI) in education show benefits for student learning. However, integrating XAI in AI-based education (AIED) systems requires understanding students' explanation needs. Some approaches to adding XAI to AIED systems include participatory design and co-design involving learners. This study presents a participatory approach to implement explanations in Active Video Watching (AVW). We designed explanations based on the requirements on timing and presentation of explanations and additional feedback from learners during the participatory activity. The implemented explanations support students who made low to medium-quality comments on video content by explaining how comment quality was determined. Furthermore, explanations included recommendations to improve future comments. We present the results of a pilot study on explanations in an AVW platform.

Presentation skills are crucial for tertiary students and graduates but are difficult to teach. We augmented active video watching (AVW) approach with the possibility of interactions among students, and conducted an experiment with AVW- Space, an online platform which supports video-based learning. The participants watched and commented on videos first. In the second phase, the participants reviewed, rated, and responded to their peers’ comments. We found that students who interacted with other students and responded to their comments increased their conceptual understanding of presentation skills.

Supporting student engagement remains one of the key challenges in video-based learning. This challenge is addressed by active video watching (AVW), a learning approach that supports engagement through different interventions, such as note-taking in the form of comments that learners submit while watching videos. One platform to support AVW is AVW-Space. Previous studies on AVW-Space detail improvements in the system, such as the integration of Artificial Intelligence and Machine Learning (ML) models in the comment feature of the system. This study investigates two machine learning models used to automatically assess the quality of comments when learning communication skills via AVW. One model is generated based on a large set of comments created by students when engaging with videos about presentation skills. For this study, a new model is developed from comments that students submitted when engaging with videos about communication skills. Results show that the new model, which was created from data on communication skills, performed better when assessing comments for communication skills compared to the model generated from comments for another skill. This has been demonstrated by the higher value of inter-rater agreement with the comment quality assessment made by human coders.

Designing explanations for Artificial Intelligence (AI) systems continues to be a challenge due to AI's black-box nature. Among the solutions developed to help in designing explanations in AI technologies is the Question-Driven Design Process for Explainable Artificial Intelligence (XAI) User Experience. In this paper, we report on our experiences using the question-driven design process for XAI in active video watching. We used Active Video Watching (AVW)-Space, an AVW platform developed at the University of Canterbury, as the context for AVW. In the question analysis process, we elicited questions from users on the AI features of the system. We conducted a survey to elicit questions from users on the AI features of the system. We conducted a survey to elicit three human raters categorized the user questions into the different XAI bank categories. Results show that most users tend to ask "how" and "why" questions about the AI-enabled features in the platform. The results of the question analysis will be used in mapping the determined question categories to potential XAI techniques. This can help in deciding the types of explanations to provide to users of AVW in future works on XAI in active video watching.

Video-based learning (VBL) provides self-paced and flexible learning. However, VBL is often a passive learning method. Active video watching (AVW) has been proposed as an approach to increase learner engagement. We investigate the motivation and perception of learners towards gamification to further increase engagement in AVW. Results from a survey in New Zealand and the Philippines show a positive perception towards integrating gamification into AVW, with learners preferring a combination of game elements rather than individual elements. Our findings provide foundations for a gamification intervention in AVW.

Soft skills (e.g., communication) significantly contribute to software project success. We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to characteristics of hiring organizations, and (c) how reliably we can automatically identify soft skills in job adverts to support their continuous analysis. We focus on soft skills required by organizations in New Zealand, a country with a small but growing software sector characterized by a skills shortage, reliance on offshoring, and embedded in a bi-cultural context. We manually analyzed 530 job adverts from New Zealand’s largest portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without pre-defined soft skills. We complemented the manual analysis with an automated analysis using Flexiterm (an approach for term recognition). We found explicit references to soft skills in 82% of adverts. Adverts from recruitment agencies (compared to hiring companies) included fewer soft skills. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related skills are most in demand. Soft skills related to broader human or societal values (e.g., empathy, cultural awareness) or distributed development are not common. Soft skills do not depend on company size or core business and domain of companies, or whether a company operates globally. Automatically identifying soft skills in adverts is error-prone. Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. On the other hand, identified soft skills only partially overlap with those reported in other skills classifications. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.

Objectives. Communication skills are crucial for effective software development teams, but those skills are difficult to teach. The goal of our project is to evaluate the effectiveness of teaching face-to-face communication skills using AVW-Space, a platform for video-based learning that provides personalized nudges to support student's engagement during video watching. Participants. The participants in our study are second-year software engineering students. The study was conducted over three years, with students enrolled in a semester-long project course. Study Method. We performed a quasi-experimental study over three years to teach face-to-face communication using AVW-Space, a video-based learning platform. We present the instance of AVW-Space we developed to teach face-to-face communication. Participants watched and commented on 10 videos and later commented on the recording of their own team meeting. In 2020, the participants ( n = 50) did not receive nudges, and we use the data collected that year as control. In 2021 ( n = 49) and 2022 ( n = 48), nudges were provided adaptively to encourage students to write more and higher-quality comments. Findings. The findings from the study show the effectiveness of nudges. We found significant differences in engagement when nudges were provided. Furthermore, there is a causal effect of nudges on the interaction time, the total number of comments written, and the number of high-quality comments, as well as on learning. Finally, participants exposed to nudges reported higher perceived learning. Conclusions. Our research shows the effect of nudges on student engagement and learning while using the instance of AVW-Space for teaching face-to-face communication skills. Future work will explore other soft skills, as well as providing explanations for the decisions made by AVW-Space.

Background: Soft skills of software professionals (e.g., communication, interpersonal skills) significantly contribute to project and product success. Aims: We aim to understand (a) what are relevant soft skills in software engineering, (b) how soft skills relate to types of software engineering positions, and (c) how soft skills relate to characteristics of hiring organizations. We focus on organizations in New Zealand, a country with a relatively small but growing software sector characterized by a skills shortage and embedded in a bi-cultural context. Method: We used a qualitative research method and manually analyzed 530 job adverts from New Zealand’s largest job portal for technology-related positions. We identified soft skills following an inductive approach, i.e., without a pre-defined set of soft skills. Results: We found explicit references to soft skills in 82% of adverts. We identified 17 soft skills and proposed a contextualized software engineering description. Communication-related soft skills are most in demand, regardless of the type of position. Soft skills related to broader human or societal values (e.g., empathy or cultural awareness) or distributed development are not frequently requested. Soft skills do not depend on company size or core business. Conclusions: Employers explicitly ask for soft skills. Our findings support previous studies that highlight the importance of communication. Characteristics specific to New Zealand do not impact the demand for soft skills. Our findings benefit researchers in human aspects of software engineering and to those responsible for staff, curricula and professional development.

Gamification has gained much popularity, due to its positive effects on learner engagement and motivation in online learning environments. However, there is still insufficient understanding of factors, including personal traits, which affect learning, as well as studies focusing on learning behaviors which can be targeted by gamification. This paper investigates the causal effects of gamification on student learning outcomes, and the role of the students' background knowledge and prior gamification experience in the relationship. The context of our study is SQL-Tutor, an intelligent tutoring system. Although we found no evidence of improvement in learning outcomes of the gamified group, the low prior knowledge students who received badges had higher time-on-task, made more attempts on problems and received more hints during interaction with the system. We also found that students who had previous gamification experience spent more time on problem solving as compared to those who had no prior gamification experience.

Intelligent Tutoring Systems (ITSs) are effective in supporting learning, as shown in numerous studies. The goal of our project is to develop an adaptive strategy that would be capable of identifying situations during problem solving in which the student would benefit from worked examples. As a first step towards developing such a strategy, we conducted a pilot study in the context of SQL-Tutor, a mature ITS that teaches database querying. The participant could ask for a worked example whenever he/she wanted during problems solving. After each example, the participant specified whether the example was useful, and whether additional examples were needed. Participants’ facial expressions and eye gaze were recorded. The findings show that the participants generally found examples useful, although in some cases they stated additional examples would be beneficial. The analysis of the eye gaze shows that students compared provided examples to their own solutions. Affect analysis shows that negative emotions reduced while engagement increased when participants viewed examples, and immediately after examples.

Engagement and motivation is always a challenge in online learning environments. The benefits of learning environments have proven its history for many years, but effectively engaging users with these environments and motivating them is an active and important research problem. In this work, I will investigate the potential of gamification on motivation and user engagement in an intelligent tutoring system SQL-Tutor. This work is inspired by the growing trend of gamification and its positive effects in various domains.

Research shows that Worked Examples (WE) and Erroneous Examples (ErrEx) provide learning benefits, particularly when presented alternatively with problems to solve. We previously proposed an adaptive strategy for selecting WE, ErrEx, and Problem Solving (PS) adaptively based on the student's problem-solving score and found that the adaptive strategy was beneficial for students in comparison to learning from a fixed sequence of alternating WE/PS pairs and ErrEx/PS pairs [1]. Students who received learning activities adaptively achieved the same learning outcomes as their peers in a fixed condition, but with fewer learning activities [2]. In this paper, we investigate a different adaptive strategy, which provides WEs and ErrExs to novices, and ErrEx and PS to advanced students. We found that the original adaptive strategy [2] is more effective than the new adaptive strategy. Furthermore, both novices and advanced students who learned with the original adaptive strategy demonstrated better performance on the post-test.

In this paper we leverage the explicit user profile (relating to experience, knowledge, and self-regulation) to predict user engagement in active video watching. Data from two user studies for informal learning of presentation skills in a Higher Education context is used to develop and validate the prediction models. Our results show that these user characteristics can reasonably predict the overall engagement (inactive, passive and constructive learners). Our approach can be used to inform adaptive interventions that prevent disengagement and enhance the learning experience.

Learning from examples, either alone or combined with problem solving has been proven to be beneficial for learning in Intelligent Tutoring System. However, it is generally unknown how much example-based assistance should be provided. We previously found that erroneous examples prepared students better for problem solving in comparison to worked examples when the order of learning activities is fixed [2]. However, students do not necessarily need all learning activities. We introduced a novel strategy which adaptively decides which learning activity (a worked example, an incorrect example, a problem, or none at all) is appropriate for a student based on his/her performance in SQL-Tutor. In this paper, we investigate the effect of the adaptive strategy on students with different levels of prior knowledge. We found both novices and advanced students who received learning activities adaptively achieved the same learning outcomes as their peers in a fixed condition, but with fewer learning activities. Surprisingly, there was no significant difference on the number of learning activities between novices and advanced students.

Videos can be a powerful medium for acquiring soft skills, where learning requires contextualisation in personal experience and ability to see different perspectives. However, to learn effectively while watching videos, students need to actively engage with video content. We implemented interactive notetaking during video watching in an active video watching system (AVW) as a means to encourage engagement. This paper proposes a systematic approach to utilise learning analytics for the introduction of adaptive intervention - a choice architecture for personalised nudges in the AVW to extend learning. A user study was conducted and used as an illustration. By characterising clusters derived from user profiles, we identify different styles of engagement, such as parochial learning, habitual video watching, and self-regulated learning (which is the target ideal behaviour). To find opportunities for interventions, interaction traces in the AVW were used to identify video intervals with high user interest and relevant behaviour patterns that indicate when nudges may be triggered. A prediction model was developed to identify comments that are likely to have high social value, and can be used as examples in nudges. A framework for interactive personalised nudges was then conceptualised for the case study.

Learning from problem solving, worked examples, and Erroneous Examples (ErrEx) have all proven to be effective learning strategies. However, what kind of learning material should be provided to students with different level of prior knowledge within Intelligent Tutoring Systems (ITSs) is still an open question. Recently, alternating worked examples and problem solving (AEP) has been shown to benefit students compared to problems only or worked examples only in SQL-Tutor (Najar & Mitrovic, 2013). However, how students with different prior knowledge learn from ErrEx in SQL-Tutor is unknown. In this paper, we compared AEP to a new instructional strategy (WPEP) which provides ErrEx in addition to worked examples and problem solving to students. The results show that that both novices and advanced students improved their post-test scores significantly in either condition. Our findings also show that novices acquired significantly more debugging knowledge when erroneous examples were presented (WPEP) in comparison to the AEP condition. Moreover, both novices and advanced students benefitted from ErrEx. In particular, advanced students who studied with erroneous examples showed better performance on problem solving as measured by the number of attempts per problem.

We present Chreos Tutor, a constraint-based tutor embedded into Chreos, an existing business software system. The goal of Chreos Tutor is to teach users new to Chreos how to complete realistic tasks. In Chreos Tutor, the student interface is the combination of a new tutoring screen and existing Chreos data input screens. We conducted a study investigating the effect of feedback Chreos Tutor provides, which shows that the feedback resulted in a significantly higher learning gain. An experienced Chreos user found the tutor to be a preferred training option for new users.

Learning by watching videos has become the dominant way of learning for millennials. However, watching videos is a passive form of learning which usually results in a low level of engagement. As the result, video-based learning often results in poor learning outcomes. One of the proven strategies to increase engagement is to integrate interactive activities such as quizzes and assessment problems into videos. Although this strategy increases engagement, it requires changing existing videos and therefore substantial effort from the teacher. We have developed the Active Video Watching (AVW) system that enables the teacher to use existing videos from YouTube without modifications. The teacher is required to define a set of aspects for videos, which serve as reflective scaffolds in order to increase engagement and focus learners’ thinking. AVW provides a Personal Space for individual learners to link their personal experiences while watching videos. The comments collected can be used by the individuals to reflect on their own thoughts or to be shared with other learners in the Social Space. We conducted a study with postgraduate students on presentation skills. The results show that the level of engagement with AVW was high, and that the aspects were effective as reflection prompts. We plan to conduct further studies related to other types of soft skills, and also to further extend AVW to provide individualized feedback to students.

Prospective Memory (PM), or remembering to perform actions in the future, is of crucial importance for everyday life. This kind of memory is often impaired in stroke survivors and can interfere with independent living. We have developed a computer-based treatment which uses visual imagery to teach participants how to remember time- and event-based prospective memory tasks better. After the treatment, participants practiced their PM skills using videos first, and later in a Virtual Reality (VR) environment. The VR environment uses Constraint-Based Modeling (CBM) to track the user actions and provide individual feedback. We report on a study with 15 stroke survivors, which shows that our treatment is highly effective.

Worked Examples (WEs) and Erroneous Examples (ErrExs) have proven to be effective in supporting learning. It has been found that WEs are beneficial for novices, while ErrExs are more suitable for advanced students. However, how such learning materials should be presented in order to improve learning of different categories of students within Intelligent Tutoring Systems (ITSs) is still an open question. We focus on approaches that can be used to motivate students with different prior knowledge to gain benefits from example-based learning. As the first step, we conducted an experiment to find students’ preferences between the original interface and the refined interface of SQL-Tutor. The results indicate that most of the students prefer the refined interface, since its layout is clearer and the organization is more efficient during learning. We plan to conduct a study that will investigate ways to improve interaction between students and ErrExs during learning.

Although Intelligent Tutoring Systems (ITSs) have proven to be very effective in supporting learning, keeping students who interact with them engaged in their activity remains a challenge. In this study, we use machine learning techniques to predict whether the student is going to abandon the current problem. The study has been done in the context of SQL-Tutor, a constraint-based ITS that teaches students how to query relational databases. We extracted a number of features from past data and used the J48 algorithm to train a decision tree. The model was used in a lab session to make predictions and provide limited intervention in order to prevent potential abandonments. Overall, the classifier demonstrated a promising performance. The results also provided insights as to what areas can be improved in future.

We describe Chreos Tutor, a constraint-based tutor embedded into Chreos, an existing business software system. The goal of the tutor is to teach users new to Chreos how to complete realistic tasks. The student interface is the combination of a new tutoring screen and existing Chreos data input screens. We have conducted a pilot study investigating the usability of Chreos Tutor, the results of which show that the tutor is promising.

Learning from worked examples (WE) has been shown to be beneficial for novices. We have previously conducted two studies, comparing learning from examples to tutored problem solving in SQL-Tutor, and Intelligent Tutoring System (ITS). The first study showed that interleaving examples with supported problem solving is an optimal choice compared to using either of those two types of learning in isolation. In the second study, we added an adaptive strategy for selecting WE or problems to be given to the learner, which proved to be superior to the fixed sequence of WE and problems. In this paper, we focus on how students with different levels of knowledge process WEs. Our goal is to identify meaningful differences in example processing that can be used to provide adaptive hints to the learner. In order to comprehend SQL examples, the learner needs to understand the database which is used as the context. We analysed eye movements collected from a quasi-experiment, and found a significant difference in the amount of attention students paid to database schemas.

Understanding students’ interactions with Intelligent Tutoring Systems (ITSs) allows us to improve the system as well as our pedagogical practices. Engaging students in tutorial dialogues is one of the strategies used by ITSs, which has been proven to improve learning significantly. This paper presents preliminary findings of a project that investigates how students interact with the tutorial dialogues in EER-Tutor using interaction videos in addition to eye-gaze data. We discuss some frequent misconceptions and behaviors student exhibited. Students usually focus on correcting one error at a time and then immediately submit their solutions to get feedback, thus not taking advantage of opportunities to reflect on what they have learnt. Based on the results, we identify several future directions of work on using eye-tracking for on-line adaptation.

Constraint-Based Modeling (CBM) is an effective student modeling approach which has been used successfully in a wide range of instructional domains. Within the Intelligent Computer Tutoring Group (ICTG), we have developed numerous constraint-based tutors and demonstrated their effectiveness in real courses. In this paper, however, we discuss how we use CBM in the area of cognitive rehabilitation after stroke. Our computer-based treatment is aimed at improving prospective memory. Participants are first trained on how to use visual imagery and then practice in a Virtual Reality (VR) environment. We present how we use constraints to track the participant’s progress when performing tasks in the VR environment.

Eye-movement tracking is a potential source of real-time adaptation in a learning environment. In order to have a more comprehensive a nd accurate picture of a user's interactions with a learning environment, we need to know which interface features he/she visually inspected, what strategies they used and what cognitive efforts they made to complete tasks. Such knowledge allows intelligen t systems to be proactive, rather than reactive, to users' actions. Tutorial dialogues is one of the strategies used by Intelligent Tutoring Systems (ITSs) and has been empirically shown to significantly improve learning. EER-Tutor is a constraint-based IT S used to teach conceptual database design. This paper presents the preliminary results of a project that investigates how students interact with the tutorial dialogues in EER-Tutor using both eye-gaze data and student-system interaction logs. Our findings indicate that advanced students are selective of the interface areas they visually focus on whereas novices waste time by paying attention to interface areas that are inappropriate for the task at hand. Novices are also unaware that they require help with the tutorial dialogues.

Prior research shows that novices learn more from examples than unsupported problem solving. Intelligent Tutoring Systems (ITS) support problem solving in many ways, adaptive feedback being one of them. However, when students repeatedly request hints from ITSs, problem solving is eventually replaced with worked examples when students request solutions to the current step or the whole problem. We conducted a study to observe the difference in learning outcomes when novices and advanced students learn from examples or with an ITS. The study had three conditions: Examples Only (EO), Problems Only (PO) and Alternating Examples and Problems (AEP). After each example/problem, students received Self-Explanation (SE) prompts. The result shows that no vices learnt significantly more conceptual knowledge in the AEP compared to the PO condition. Moreover, novices in the AEP and PO conditions performed significantly better on SE prompts than students in the EO condition. Advanced students who learnt from examples only did not significantly improve in the study. Overall, the study suggests using AEP for novices and either AEP or PO for advanced students. The results clearly reveal that using examples alone is not an effective approach for novices and advanced students in comparison with ITSs.

Students often have misconceptions in the domain they are studying. Misconception identification is a difficult task but allows teachers to create strategies to appropriately address misconceptions held by students. This project investigates a data-driven technique to discover students' misconceptions in interactions with constraint-based Intelligent Tutoring Systems (ITSs). This analysis has not previously been done. EER-Tutor is one such constraint-based ITS, which teaches conceptual database design using Enhanced Entity-Relationship (EER) data modelling. As with any ITS, a lot of data about each student's interaction within EER-Tutor are available: as individual student models, containing constraint histories, and logs, containing detailed information about each student action. This work can be extended to other ITSs and their relevant domains.

Although examples are frequently used by human tutors, they are not common in Intelligent Tutoring Systems (ITS). Previous research studies over the last three decades compared learning from examples to unsupported problem solving. Only recently there have been studies comparing learning from examples to problem solving in ITSs. This paper reviews those studies. We discuss unsolved issues such as when and how examples should be provided in intelligent tutoring systems, and some options to improve learning from examples.

Although Intelligent Tutoring Systems (ITSs) have well proven their effectiveness in many learning domains, building them have always required extensive effort and time. ASPIRE authoring system has been used in developing constraint based tutors (CBTs) before but this will be the first attempt to develop a CBT and embed it within an existing system. We present the research and development of DM-Tutor, the first CBT to be embedded within the Management Information System (MIS) for palm oil plantation management. We discuss the research and development of DM Tutor with the help of ASPIRE. We also include future work planned for DM-Tutor.

Open Student Models (OSM) are beneficial for improving students' domain knowledge and meta-cognitive skills. The way in which the student model is displayed may be an important factor which has not been investigated adequately in the context of Intelligent Tutoring Systems (ITS). In our study, the control group had skill meters, while the experimental group additionally could access the OSM represented as a concept list, concept hierarchy or a concept map. The results show that OSM do have a positive effect on students' learning. However, the students showed clear preferences towards simpler representations than the more complex ones.

Researchers have long been interested in tutorial dialogues as they are considered to be one of the critical factors contributing to the effectiveness of human one-on-one tutoring. We discuss an evaluation study that investigates the effectiveness of adaptive tutorial dialogues in database design. EER-Tutor, a database design tutor was enhanced to facilitate adaptive tutorial dialogues. The control group participants received non-adaptive dialogues regardless of their knowledge level and explanation skills. The experimental group participants received adaptive dialogues that were customised based on their student models. The performance on pre- and post-tests indicated that the experimental group participants learned significantly more than their peers. The subjective responses indicated no difference in their impression towards the quality of the dialogues and the understandability of the questions. However there was clear evidence that the control group did not like having to go through the entire dialogue before resuming problem-solving.

Workshop jointly chaired by Cristina Conati, and Tania Mitrovic.

ITSs for ill-defined domains have attracted a lot of attention recently, which is well-deserved, as such ITSs are hard to develop. The first step towards such ITSs is reaching a wide agreement about the terminology used in the area. In this paper, we discuss the two important dimensions of ill-definedness: the domain and the instructional task. By the domain we assume declarative domain knowledge, or the domain theory, while the instructional task is the task the student is learning, in terms of problem-solving skills. It is possible to have a well-defined domain and still have ill-defined instructional tasks in the same domain. We look deeper at the features of ill-defined tasks, which all contribute to their ill/well defined nature. The paper discusses model-tracing and constraint-based modeling, in terms of their suitability for ill-defined tasks and domains. We show that constraint-based modeling can be used in both well- and illdefined domains, and illustrate our conclusion using several instructional tasks.

Constraint-based modelling (CBM) is a student modelling technique for intelligent tutoring systems (ITS) that is especially suited to complex, open-ended domains. It is easier to build tutors in such domains using CBM than other common approaches. The authors present WETAS (Web-Enabled Tutor Authoring System), a tutoring engine that facilitates the rapid implementation of ITS in new domains using CBM. They describe the architecture of WETAS and give examples of two domains they have implemented. They also present the results of an evaluation of a tutoring system built using WETAS in a New Zealand school.

The paper describes the design and development of NORMIT, an intelligent tutoring system (ITS) that teaches database normalization to university students. NORMIT is a Web-enabled system, and we discuss its architecture and techniques used to deal with multiple students. We also discuss constraint-based modeling (CBM), the underlying student and domain modelling approach. NORMIT is the first in the series of constraint-based tutors developed at ICTG that teaches a procedural task, and we comment on the suitability of CBM for such tasks. We also discuss the plans for the evaluation of the system and future work.

The paper proposes the M-OBLIGE model for building multitutor ontology-based learning environments. We show bow the model can be applied to tutors in the database domain. The proposed model can be used as a framework for integrating multiple tutors on the Web.

This paper presents a first step towards an intelligent problem selection agent for the SQL-Tutor Intelligent Tutoring system. Currently SQL-Tutor uses an overly simple problem selection strategy, which selects a problem based on a single construct the student has most problems with. This strategy very often results in problems that are too easy/difficult for the student. Here we propose an intelligent problem-selection agent, which identifies the appropriate problem for a student in two stages. It firstly predicts the number of errors the student will make on a set of problems, and then in the second stage decides on a suitable problem for the student. In order to develop such an agent, we trained a feed-forward, backpropagation neural network to predict the number of errors a student will make. The achieved prediction accuracy is high, showing that a neural network is capable of making such predictions. However, the developed network cannot be used on-line, as it requires values that are not readily available. We present the plan for developing a modified network and for completing the problem selection agent.

Self-explanation is an effective teaching/learning strategy that has been used in several intelligent tutoring systems in the domains of Mathematics and Physics to facilitate deep learning. Since all these domains are well structured, the instructional material to self-explain can be clearly defined. We are interested in investigating whether self-explanation can be used in an open-ended domain. For this purpose, we enhanced KERMIT, an intelligent tutoring system that teaches conceptual database design. The resulting system, KERMIT-SE, supports self-explanation by engaging students in tutorial dialogues when their solutions are erroneous. We plan to conduct an evaluation in July 2002, to test the hypothesis that students will learn better with KERMIT-SE than without self-explanation.

SQL, the dominant database language, is a simple and highly structured language; yet, students have many difficulties learning it. This paper presents SQL-Tutor, an Intelligent Teaching System designed as a guided discovery learning environment, which helps students in overcoming these difficulties. We present design issues and the current state in the implementation of the system, with special focus on individualization of instruction towards a particular student.

Reconstructive bug modeling is a well‐known approach to student modeling in intelligent tutoring systems, suitable for modeling procedural tasks. Domain knowledge is decomposed into the set of primitive operators and the set of conditions of their applicability. Reconstructive modeling is capable of describing errors that come from irregular application of correct operators. The main obstacle to successfulness of this approach is such decomposition of domain knowledge to primitive operators with a very low level of abstraction so that bugs could never occur within them. The other drawback of this modeling scheme is its efficiency because it is usually done offline, due to vast search spaces involved. This article reports a novel approach to reconstructive modeling based on machine‐learning techniques for inducing procedures from traces. The approach overcomes the problems of reconstructive modeling by its interactive nature. It allows online model generation by using domain knowledge and knowledge about the student to focus the search on the portion of the problem space the student is likely to traverse while solving the problem. Furthermore, the approach is not only incremental, but also truly interactive because it involves the student in explicit dialogs about his or her goals. In such a way, it is possible to determine whether the student knows the operator he or she is trying to apply. Pedagogical actions and the student model are generated interchangeably, thus allowing for dynamic adaptation of instruction, problem generation, and immediate feedback on student's errors. The approach presented is examined in the context of the symbolic integration tutoring system (SINT), an intelligent tutoring system (ITS) for the domain of symbolic integration.

Programming by example is a powerful way of bestowing on nonprogrammers the ability to communicate tasks to a computer. When creating procedures from examples it is necessary to be able to infer the existence of variables, conditional branches, and loops. This article explores the role of empirical or “similarity-based” learning in this process. For a concrete example of a procedure induction system, we use an existing scheme called METAMOUSE which allows graphical procedures to be specified from examples of their execution. A procedure is induced from the first example, and can be generalized in accordance with examples encountered later on. We describe how the system can be enhanced with Mitchell's candidate elimination algorithm, one of the simplest empirical learning techniques, to improve its ability to recognize constraints in a comprehensive and flexible manner. Procedure induction is, no doubt, a very complex task. This work revealed usefulness and effectiveness of empirical learning in procedure induction, although it cannot be a complete substitute for specific preprogrammed, domain knowledge in situations where this is readily available. However, in domains such as graphical editing, where knowledge is incomplete and/or incorrect, the best way to pursue may prove to be a combination of similarity- and explanation-based learning. © 1994 John Wiley & Sons, Inc.