Claudia Szabo

dblp:11/1325 · DBLP profile ↗
← Back
61ranked-venue papers
14as first author
16since 2021 · last 2026
0000-0003-2501-1155ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 20 · 9 first-author · 6 since 2021Software engineering, systems software and programming languages · 16 · 2 first-author · 5 since 2021Databases, data management, data science and information retrieval · 9Artificial intelligence and machine learning · 5 · 1 first-author · 1 since 2021Systems, architecture and hardware · 3 · 1 first-authorSecurity and privacy · 3Applied, interdisciplinary, general and emerging computing · 2Computer networks · 1
YearPublicationVenuePosition
2026 LLM-led Socratic Dialogues on Ethics in Computing
abstract
In a Socratic dialogue, a teacher engages a student in a series of questions to prompt critical thinking, guiding the student to clarify and elaborate on their own view and to consider alternative perspectives. In this experience report, we analyse the suitability of large language models (LLMs) as automated Socratic tutors. Postgraduate computer science students from two universities and multiple courses held one-to-one interactive dialogues with LLMs on professional ethics and technology ethics scenarios. We found that the LLMs' questions were diverse and broadly scaffolded critical thinking. Students were able to demonstrate most of the cognitive levels of Bloom's Taxonomy in their answers. However, we also observed some significant limitations of LLMs as Socratic tutors, including unfounded praise of poor responses and reformulating answers for students rather than guiding students to do this themselves. Further, although many students interacted effectively with the LLM, others did not engage authentically with the process. These limitations can potentially be mitigated via prompt engineering and activity framing. Our work contributes an LLM-supported learning activity and our lessons learned on using current state-of-the-art LLMs as Socratic tutors for teaching ethics in computing. In addition, our codebooks provide a set of criteria to drive the evaluation of LLMs used for similar activities.
Rachel Cardell-Oliver, Claudia Szabo, Kaie Maennel, Hamish Russell
ITiCSE (1)2
2026 Towards a Shared Framework for Selection, Design, and Evaluation of Mastery Learning Models in Computing Education
Claudia Szabo, Miranda C. Parker, Judithe Sheard, Giulia Alberini, Andrew Luxton-Reilly, Stephanos Matsumoto, Fiona McNeill, Charlotte Pierce, Naaz Sibia, Jan Vahrenhold, Craig B. Zilles
ITiCSE (2)1
2026 On the Modelling of Aggregated Behaviour for Simulation: An Event-Based Architecture
Adam Banham, Claudia Szabo, Ryan Beruldsen
SIGSIM-PADS2
2025 Resilient Auto-Scaling of Microservice Architectures with Efficient Resource Management
abstract
Horizontal Pod Auto-scalers (HPAs) are crucial for managing resource allocation in microservice architectures to handle fluctuating workloads. However, traditional HPAs fail to address resource disruptions caused by faults, cyberattacks, maintenance, and other operational challenges. These disruptions result in resource wastage, service unavailability, and HPA performance degradation. To address these challenges, we propose SecureSmart HPA, a resilient and resource-efficient HPA for microservice architectures. SecureSmart HPA monitors microservice resource demands, detects disruptions, evaluates resource wastage, and dynamically adjusts scaling decisions to enhance the resilience of auto-scaling operations. Furthermore, SecureSmart HPA enables resource sharing among microservices, optimizing scaling efficiency in resource-constrained environments. Experimental evaluation demonstrates that under disruption severities of 25%, 50%, and 75% resource wastage, SecureSmart HPA delivers robust performance, underscoring its resilience and efficiency in volatile, resource-constrained environments.
Hussain Ahmad, Christoph Treude, Markus Wagner 0007, Claudia Szabo
APSEC4
2025 SCALAR: Self-Calibrating Adaptive Latent Attention Representation Learning
abstract
High-dimensional, heterogeneous data with complex feature interactions pose significant challenges for traditional predictive modeling approaches. While Projection to Latent Structures (PLS) remains a popular technique, it struggles to model complex non-linear relationships, especially in multivariate systems with high-dimensional correlation structures. This challenge is further compounded by simultaneous interactions across multiple scales, where local processing fails to capture crossgroup dependencies. Additionally, static feature weighting limits adaptability to contextual variations, as it ignores sample-specific relevance. To address these limitations, we propose a novel method that enhances predictive performance through novel architectural innovations. Our architecture introduces an adaptive kernel-based attention mechanism that processes distinct feature groups separately before integration, enabling capture of local patterns while preserving global relationships. Experimental results show substantial improvements in performance metrics, compared to the state-of-the-art methods across diverse datasets.
Farwa Abbas, Hussain Ahmad, Claudia Szabo
ICTAI3
2025 Simulation Framework Architecture for Assessing Dynamic Distributed Network Task Coordination Implementations
Marika Colby, Claudia Szabo
SIGSIM-PADS2
2025 Models of Mastery Learning for Computing Education
abstract
The application of mastery learning, where students progress through their learning in a self-paced manner until they have mastered specific concepts, is considered appealing for teaching introductory programming courses. Despite its growing popularity in computing and its extensive use in other disciplines, there is no overview of the design of courses that use mastery learning. In this position paper, we present an overview of five mastery learning models and discuss examples of how these can be applied in practice, both in foundational programming as well as more advanced courses. Our analysis focuses on the student progression through the course, the assessment structure, and the support for self-paced learning, including for struggling students. This work provides a greater understanding of mastery learning and its application in a computing education context.
Claudia Szabo, Miranda C. Parker, Michelle Friend, Johan Jeuring, Tobias Kohn, Lauri Malmi, Judithe Sheard
SIGCSE (1)1
2025 Towards resource-efficient reactive and proactive auto-scaling for microservice architectures
abstract
Microservice architectures have become increasingly popular in both academia and industry, providing enhanced agility, elasticity, and maintainability in software development and deployment. To simplify scaling operations in microservice architectures, container orchestration platforms such as Kubernetes feature Horizontal Pod Auto-scalers (HPAs) designed to adjust the resources of microservices to accommodate fluctuating workloads. However, existing HPAs are not suitable for resource-constrained environments, as they make scaling decisions based on the individual resource capacities of microservices, leading to service unavailability, resource mismanagement, and financial losses. Furthermore, the inherent delay in initializing and terminating microservice pods hinders HPAs from timely responding to workload fluctuations, further exacerbating these issues. To address these concerns, we propose Smart HPA and ProSmart HPA, reactive and proactive resource-efficient horizontal pod auto-scalers respectively. Smart HPA employs a reactive scaling policy that facilitates resource exchange among microservices, optimizing auto-scaling in resource-constrained environments. For ProSmart HPA, we develop a machine-learning-driven resource-efficient scaling policy that proactively manages resource demands to address delays caused by microservice pod startup and termination, while enabling preemptive resource sharing in resource-constrained environments. Our experimental results show that Smart HPA outperforms the Kubernetes baseline HPA, while ProSmart HPA exceeds both Smart HPA and Kubernetes HPA by reducing resource overutilization, overprovisioning, and underprovisioning, and increasing resource allocation to microservice applications. • Hierarchical architecture based horizontal pod auto-scaler. • Reactive and proactive resource-efficient auto-scaling policies. • Reactive auto-scaler outperforms Kubernetes baseline auto-scaler. • Proactive auto-scaler outperforms both reactive and Kubernetes auto-scalers. • Reduction in resource overutilization, underprovisioning, and overprovisioning.
Hussain Ahmad, Christoph Treude, Markus Wagner 0007, Claudia Szabo
J. Syst. Softw.4
2025 Towards Multi-Class Socio-Technical Congruence: Assessing Coordination in Collaborative Software Development Settings
abstract
ABSTRACT Effective coordination between contributors with different functional roles is fundamental for the success of collaboration‐centric software development paradigms such as DevSecOps. However, quantitatively assessing coordination in such settings has received limited attention. We introduce multi‐class socio‐technical congruence (), an extension of the widely studied socio‐technical congruence () framework to address this gap. Our metric enables the assessment of coordination in a setting where contributors with different functional roles or alignments collaborate. Using a large‐scale exploratory case study, we evaluated for two classes (i.e., ). Specifically, we calculated for 100 systematically selected projects from the TravisTorrent dataset, considering developers (dev) and security‐focused developers (sf‐devs) as the two types of contributors with different functional alignments (i.e., two classes). We hypothesized that the dev and sf‐dev interaction would have a quantifiable impact on the vulnerability score () of each project. Our results show a moderate negative association between and , with the Spearman correlation reaching 0.427 (), indicating that higher levels of coordination between dev and sf‐dev led to projects with a lower incidence of high‐severity vulnerabilities. In addition, shows a stronger negative relationship with than , suggesting that it is the more sensitive indicator of this relationship. Therefore, the specific instantiation of our proposed metric, , performs comparatively better than for measuring cross‐functional coordination in our selected projects. However, further research is needed to explore its broader applicability.
Roshan Namal Rajapakse, Claudia Szabo
J. Softw. Evol. Process.2
2024 Smart HPA: A Resource-Efficient Horizontal Pod Auto-Scaler for Microservice Architectures
abstract
Microservice architectures have gained prominence in both academia and industry, offering enhanced agility, reusability, and scalability. To simplify scaling operations in microservice architectures, container orchestration platforms such as Kubernetes feature Horizontal Pod Auto-scalers (HPAs) designed to adjust the resources of microservices to accommodate fluctuating workloads. However, existing HPAs are not suitable for resource-constrained environments, as they make scaling decisions based on the individual resource capacities of microservices, leading to service unavailability and performance degradation. Furthermore, HPA architectures exhibit several issues, including inefficient data processing and a lack of coordinated scaling operations. To address these concerns, we propose Smart HPA, a flexible resource-efficient horizontal pod auto-scaler. It features a hierarchical architecture that integrates both centralized and decentralized architectural styles to leverage their respective strengths while addressing their limitations. We introduce resource-efficient heuristics that empower Smart HPA to exchange resources among microservices, facilitating effective auto-scaling of microservices in resource-constrained environments. Our experimental results show that Smart HPA outperforms the Kubernetes baseline HPA by reducing resource overutilization, overprovisioning, and underprovisioning while increasing resource allocation to microservice applications.
Hussain Ahmad, Christoph Treude, Markus Wagner 0007, Claudia Szabo
ICSA4
2024 Code Refactoring Strategies of Third Year Software Engineering Students
abstract
Code refactoring is a critical graduate skill as software developers embark on careers that often require the maintenance of large code bases. Despite its importance, code refactoring is often either not taught explicitly or only taught to novice programmers, while the skills gained from refactoring smaller programs may not necessarily transfer to managing large code bases. In addition, the code refactoring skills of more advanced programmers remain not fully understood. Therefore, in this paper, we report on our experience of introducing a code refactoring assignment in a third year course of a Software Engineering degree. We analyse the students' approaches to refactoring and challenges encountered, identifying areas for curriculum development.
Roshan Rajapakse, Claudia Szabo
ITiCSE (1)2
2024 Detecting Emergent Behavior in Complex Systems: A Machine Learning Approach
abstract
The live identification of emergent behavior in complex systems with little a-priori information is a challenging task and existing approaches are either applicable to a small subset of models or do not scale well. In contrast, post-mortem approaches that have a more in-depth understanding of the characteristics of emergent properties often struggle with analyzing a large amount of data to extract relationships between the variables, events, and entities whose interaction eventually leads to emergent behavior. Machine learning approaches have been promoted as potential replacements of existing approaches, due to their ability to analyze large amounts of data without a-priori knowledge of existing relationships. In this paper, we present a first step towards the use of supervised learning approaches to identify and predict emergent behavior. Our hybrid approach unifies live and post-mortem perspectives by relying on a visual inspection of the simulation run and the simulation data set to identify a set of features that are more likely to generate emergent behavior (post-mortem) which are then used by a machine learning module to predict emergent behavior (live). Our analysis shows the potential of such approaches but also highlights challenges and future avenues of research.
Simranjeet Singh Dahia, Claudia Szabo
SIGSIM-PADS2
2024 Contested Communication in C2 Multi-agent Simulations
abstract
Mobile Ad-hoc Networks (MANETs) are frequently deployed in dynamic and contested military environments and it is critical that their behaviour is well understood. Existing MANET research is application agnostic, providing an in-depth view of network behaviour at the packet and route level. However, in order to determine decision impact at various levels, there is a need to understand the effects of network behaviour on the applications that use them. In addition, it is critical that the impact of node failures is well understood, in order for the system resilience to be evaluated. We evaluate the failure tolerance of different network topologies in contested scenarios by integrating communications, networking models and their limitations in a command and control (C2) simulation. Results indicate that network topology directly affects network performance, highlighting the importance for considering network topology in contested environments.
Huey Pretila, Benjamin Campbell, Claudia Szabo
SIGSIM-PADS3
2024 Unlocking Excellence in Educational Research: Guidelines for High-Quality Research that Promotes Learning for All
abstract
While there are multiple standards bodies that define characteristics of high-quality, there are limited guidelines on conducting equity-enabling research, particularly in the context of high quality and in computing education. As part of an ACM ITiCSE Working Group in 2023, we engaged in a concept analysis and structured literature review to identify high-impact practices for conducting both high-quality and equity-enabling education research. As a result of this work, we produced a set of guidelines across each major phase of research that integrates characteristics of high-quality education research with those that are necessary for producing research that is designed to honor and meet the needs of various subgroups of learners. Special emphasis is given to the role that the researcher plays in shaping the research based upon how the researcher's lived experiences, perspectives, and training influences their work. During this special session, we will review each set of guidelines and engage attendees in reflection and discussion of them and how they can use the guidelines to enhance their education research.
Monica McGill, Sarah Smith Heckman, Michael Liut, Ismaila Temitayo Sanusi, Claudia Szabo
SIGCSE (2)5
2023 Building Recommendations for Conducting Equity-Focused, High Quality K-12 Computer Science Education Research
abstract
To investigate and identify promising practices in equitable K-12 computer science (CS) education, the capacity for education researchers to conduct this research must be rapidly built globally. Simultaneously, concerns have arisen over the last few years about the quality of research that is being conducted and the lack of equity-focused research.
Monica McGill, Sarah Smith Heckman, Christos Chytas, Lien Diaz, Michael Liut, Vera A. Kazakova, Ismaila Temitayo Sanusi, Selina Marianna Shah, Claudia Szabo
ITiCSE (2)9
2022 Self-adaptation in Microservice Architectures: A Case Study
abstract
Most software companies deploy microservices be-hind API Gateways or load balancers to separate their business logic while at the same time serving their customers according to their SLAs. Today, internet companies serve an average of 150–200 million users efficiently in rapidly changing conditions, where autonomic self-adaptation solutions are critical. At such a large scale, self-adaptation has to address challenges related to high availability and reliability, in a variety of scenarios. In this industry experience report, we present the implementation of a self-adaptation approach for microservice architectures that can operate at a large scale and address availability and reliability concerns. Our prototype builds on current industry standards of observability tools used to track the system's internal state. We implement a lightweight MAPE-K loop that reduces the time taken to add self-adaptability and the total cost of ownership. Our case study focuses on dynamic rate limiting, where the implementation of our architecture was able to trigger and execute self-adaptation in under 1 second. We present our architecture, an overview of our prototype implementation and suite of tools used, and discuss our empirical observations.
Sree Ram Boyapati, Claudia Szabo
ICECCS2
2020 Optimizing Communication Strategies in Contested and Dynamic Environments
abstract
Contested and dynamic environments such as those of military operations and crisis situations have poor and unreliable network conditions and participants usually only have an incomplete, local, and quickly changing view of the system. In such systems, optimizing how nodes communicate such that important messages arrive in a timely manner without degrading network performance is critical. SMARTNet is a middleware that prioritizes and controls the messages sent by each node, with the aim of preserving network bandwidth, while at the same time achieving timely delivery of messages within a contested and dynamic environment. In this industry experience report, we propose the integration of evolutionary algorithms with the SMARTNet middleware allowing it to learn the best bandwidth ratio for each different message type. We propose a centralized integration, where a single node performs the evolutionary algorithm (EA) and determines a communication strategy that is subsequently followed by all SMARTNet nodes. Our results show an improvement of nearly 50% over the baseline, at a cost of significant pre-run preparation for the EA to converge on a potential solution. Our analysis also shows the benefits of using an application-specific metric as an objective of the EA, and our discussion identifies new research avenues.
Claudia Szabo, Vanja Radenovic, Gregory Judd, Dustin Craggs, Kin Leong Lee, Xiaoshan Chen, Kevin S. Chan
ICECCS1
2020 Adaptive Performance Anomaly Detection in Distributed Systems Using Online SVMs
abstract
Performance anomaly detection is crucial for long running, large scale distributed systems. However, existing works focus on the detection of specific types of anomalies, rely on historical failure data, and cannot adapt to changes in system behavior at run time. In this work, we propose an adaptive framework for the detection and identification of complex anomalous behaviors, such as deadlocks and livelocks, in distributed systems without historical failure data. Our framework employs a two-step process involving two online SVM classifiers on periodically collected system metrics to identify at run time normal and anomalous behaviors such as deadlock, livelock, unwanted synchronization, and memory leaks. Our approach achieves over 0.70 F-score in detecting previously unseen anomalies and 0.78 F-score in identifying the type of known anomalies with a short delay after the anomalies appear, and with minimal expert intervention. Our experimental analysis uses system execution traces from our in-house distributed system with varied behaviors and a dataset by Yahoo!, and shows the benefits of our approach as well as future research challenges.
Javier Álvarez Cid-Fuentes, Claudia Szabo, Katrina Falkner
IEEE Trans. Dependable Secur. Comput.2
2019 A Periodic Table of Computing Education Learning Theories
abstract
Computing education research is built on the use of suitable methods within appropriate theoretical frameworks to provide guidance and solutions for our discipline, in a way that is rigorous and repeatable. However, the scale of theory covered extends well beyond the CS discipline and includes educational theory, behavioural psychology, statistics, economics, and game theory, among others. A computing education researcher's journey towards appropriate and discipline relevant theory can be challenging and, when a researcher has learned one area of theory, it can be easy to return to familiar theory, as it may not be clear what the next step could be. The periodic table is a visual arrangement of the elements to group like with like, providing insight into how families of elements will react. Could we do the same with learning theories located in the domain of computer science education, and would it be useful? The working group will identify and survey existing literature on relationships between key areas of theory in computing education, identify ways of organising these research areas to show how knowledge of one could assist another, and produce initial graphical representations of theory and their relationship groupings to assist researchers in understanding how computing theory is currently used in the discipline and what theories might become of interest.
Claudia Szabo, Nick Falkner, Andrew Petersen 0001, Heather Bort, Cornelia Connolly, Kathryn I. Cunningham, Peter Donaldson, Arto Hellas, Judithe Sheard
ITiCSE1
2019 Final Year Students' Approaches to Implementing Complex Distributed Systems
abstract
Understanding how final year students build complex software systems is critical for determining whether desired graduate outcomes have been met, for identifying curriculum gaps, and for designing scaffolding and support structures. A large body of work focuses on the programming strategies employed by novice programmers, with few existing research in understanding programming strategies and development focus of final year students, in particular with respect to non-functional requirements. In this paper, we analyse consecutive revisions of 77 students across two cohorts that implemented a large and complex Distributed Systems assignment with several non-functional requirements. To obtain a qualitative overview of the students' approach to software development, we manually read and tagged all sourcefiles in all assignment revisions with specific development focus categories. Our analysis identifies how the students' development focus evolves throughout the assignment timeline. We visualise the software development process and identify several areas that require further support.
Claudia Szabo, Michael Scott Pointon
ITiCSE1
2019 Adaptive Swarm Control for Mobile Resource Placement in Wireless Ad-Hoc Networks
abstract
The use of additional radio resources in disconnected wireless ad-hoc networks can take the form of range extension or data ferrying between graph components. The former approach places resources between disconnected components to act as a radio relay, while the latter approach has resources move between disconnected components to deliver waiting traffic; algorithms exist for both functions in the literature. However, in some operational environments such as tactical networks, connectivity will likely vary in different regions of the network due to localised geographical and radio-frequency issues. To address this problem, this article presents a swarming-inspired algorithm that is able to achieve contextually appropriate behaviour using a single set of rules. The approach reduces latency by as much as 45% with 8 resource nodes for low graph connectivity as compared to a Travelling Salesman Problem solution. An almost arbitrarily large improvement is seen when the approach adopts a relaying strategy, reconnecting the network completely in high graph connectivity cases. Leveraging connectivity results from Random Geometric Graph theory, an analysis of the algorithm's ability to adapt to various network densities is presented.
Bradley Fraser, Andrew Coyle, Claudia Szabo, Robert A. Hunjet
WOWMOM3
2019 Transactional Behavior Verification in Business Process as a Service Configuration
abstract
Business Process as a Service (BPaaS) is an emerging type of cloud service that offers configurable and executable business processes to clients over the Internet. As BPaaS is still in early years of research, many open issues remain. Managing the configuration of BPaaS builds on areas such as software product lines and configurable business processes. The problem has concerns to consider from several perspectives, such as the different types of variable features, constraints between configuration options, and satisfying the requirements provided by the client. In our approach, we use temporal logic templates to elicit transactional requirements from clients that the configured service must adhere to. For formalizing constraints over configuration, feature models are used. To manage all these concerns during BPaaS configuration, we develop a structured process that applies formal methods while directing clients through specifying transactional requirements and selecting configurable features. The Binary Decision Diagram (BDD) analysis is then used to verify that the selected configurable features do not violate any constraints. Finally, model checking is applied to verify the configured service against the transactional requirement set. We demonstrate the feasibility of our approach with several validation scenarios and performance evaluations.
Scott Bourne, Claudia Szabo, Quan Z. Sheng
IEEE Trans. Serv. Comput.2
2018 A review of introductory programming research 2003-2017
abstract
A broad review of research on the teaching and learning of programming was conducted by Robins et al. in 2003. Since this work there have been several reviews of research concerned with the teaching and learning of programming, in particular introductory programming. However, these reviews have focused on highly specific aspects, such as student misconceptions, teaching approaches, program comprehension, potentially seminal papers, research methods applied, automated feedback for exercises, competency-enhancing games, and program visualisation. While these aspects encompass a wide range of issues, they do not cover the full scope of research into novice programming. Some notable areas that have not been reviewed are assessment, academic integrity, and novice student attitudes to programming. There does not appear to have been a comprehensive review of research into introductory programming since that of Robins et al. It is therefore timely to conduct and present such a review in order to gain an understanding of the research focuses, to highlight advances in knowledge since 2003, and to indicate possible future directions for research. The working group will conduct a systematic literature review based on the guidelines proposed by Kitchenham et al. This research project is well suited to an ITiCSE working group as the synthesis and discussion of the literature will benefit from input from a variety of researchers drawn from different backgrounds and countries.
Andrew Luxton-Reilly, Simon, Ibrahim Albluwi, Brett A. Becker, Michail N. Giannakos, Amruth N. Kumar, Linda M. Ott, James H. Paterson, Michael 'Adrir' Scott, Judithe Sheard, Claudia Szabo
ITiCSE11
2018 An adaptive framework for the detection of novel botnets
Javier Álvarez Cid-Fuentes, Claudia Szabo, Katrina Falkner
Comput. Secur.2
2018 Model-driven performance prediction of systems of systems
Katrina Falkner, Claudia Szabo, Vanea Chiprianov, Gavin Puddy, Marianne Rieckmann, Daniel Fraser, Cathlyn Aston
Softw. Syst. Model.2
2018 SNAF: Observation filtering and location inference for event monitoring on twitter
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng, Xiu Susie Fang
World Wide Web2
2017 Identifying Domains and Concepts in Short Texts via Partial Taxonomy and Unlabeled Data
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng, Wei Zhang 0098, Yongrui Qin
CAiSE2
2017 Identifying Domain-Specific Cognitive Strategies for Software Engineering
abstract
Due to the rapidly changing nature of today's work environment, software engineering (SE) students are required to have self-regulated learning (SRL) and problem solving skills. Previous research suggests that training students in the use of domain-specific cognitive strategies and using scaffolded instruction for strategy training improves students' SRL and problem solving task performance. In order to identify SE-specific cognitive strategies, we conducted a survey of advanced-level SE students. We then conducted a pre-test and post-test experiment with one control and two treatment groups, to analyze the effectiveness of identified strategies in improving students' task performance. The control group was not exposed to any strategies, while one treatment group was instructed verbally in the use of strategies and the other was trained using a newly developed scaffolded strategy training module. The results of the experiment demonstrate significant improvement in post-test task performance for both treatment groups, with a further increase in performance for those undertaking the training module.
Shifa-e-Zehra Haidry, Katrina Falkner, Claudia Szabo
ITiCSE3
2017 Silence, Words, or Grades: The Effects of Lecturer Feedback in Multi-Revision Assignments
abstract
Detailed in-depth feedback on programming assignments is beneficial because it identifies specific software design and development aspects that students can improve on. For the feedback to be effective, it is important that students are given the opportunity to address the feedback in a timely manner. However, detailed in-depth feedback often needs to be manually written by the lecturer or marker, especially for large and complex final year assignments where automated test suites are difficult to implement, introducing potential delay in providing the feedback. Following existing work, we propose a two-stage assignment design where students receive feedback on their final submission and are then given the opportunity to address the feedback. We analyse 147 assignment submissions and show that this assignment design improves assignment marks when compared to a single-stage submission, with failure rates dropping by up to 30%. To determine the impact of in-depth detailed feedback, we compare the learning outcomes across two years where students taking the two-stage assignment were given either detailed feedback or simple feedback consisting of component marks at the initial stage. We show the benefits of both approaches and analyse the potential advantages of providing more costly, manual feedback.
Claudia Szabo, Nick Falkner
ITiCSE1
2017 Understanding the Effects of Lecturer Intervention on Computer Science Student Behaviour
abstract
A key challenge for computer science educators worldwide is providing effective feedback and support to students, to ensure they are engaged with the course. This includes online feedback on discussion forums as well as feedback on programming assignments. Due to the significant problems of scale that need to be addressed, effective lecturer intervention is difficult, and at the same time the effect of intervention in online discussion forums is challenging to measure accurately. The same problem occurs when marking programming assignments, where detailed, in-depth feedback is often replaced with output from failed testcases, which the students sometimes proceed to address without giving thought to the quality of their overall solutions.
Claudia Szabo, Nick Falkner, Mohsen Dorodchi, Antti Knutas, Francesco Maiorana
ITiCSE1
2017 Understanding the Effects of Intervention on Computer Science Student Behaviour in On-line Forums
abstract
A key challenge for educators using online discussion forums is how to provide effective feedback and support to students, to ensure they are engaged with discussions on the forums, and do not disengage from the course. In addition, there is a significant problem of scale to address. Even relatively low population forums of less than a hundred students can generate thousands of posts, so it is infeasible for the lecturer to monitor every discussion to identify disengaging students. There is a need to understand the act of intervention, in order to provide automated tools to better assist teaching staff with this task. Measuring the impact of intervention can be challenging and requires us to, first, understand what ``standard'' behaviour looks like across different student groups and identify topics where intervention would be most effective. In this paper, we identify the impact of intervention on different groups of students, characterising their behaviour in terms of response time and activity, compare the different responses to programming-related questions and other questions, and identify the useful aspects of this study for computer science educators. We conduct an initial examination of the impact of the nature of the question on intervention effectiveness and propose an analysis method that can be applied to any computer science forum. we showcase the application of the method to three courses. Our results indicate that associating student activity with the number of forum posts is misleading, as students who are only reading the forums respond also to intervention.
Daniel La Vista, Nick Falkner, Claudia Szabo
ITiCSE3
2017 Formal Forum Triage: Towards the Strategic Selection of Responses to Student Discussion Forums
abstract
It can be difficult for educators with limited resources to decide which queries need immediate attention when a high volume of questions arises on the discussion forum. This becomes increasingly complex as the educator aims to obtain the best outcome across all threads in a timely fashion. Existing approaches to automated forum analysis provide a useful grouping of messages and identify common discussions, but require additional attention towards effective intervention. Research has shown that the timing of messages relative to associated deadlines is a key indicator of priority. In this paper, we propose the formal representation of events within a discussion forum to facilitate the definition of potential and existing intervention strategies. We enhance forum events with information about teaching activities, such as assignment deadlines, and discuss intervention strategies.
Nick Falkner, Claudia Szabo, Katrina Falkner
L@S2
2017 Broadening Participation in Computer Science: Key Strategies from International Findings
abstract
This special session is based around key findings of a Barbara Cail STEM Fellowship that aims to inform key stakeholders about international best practices for broadening participation and diversity in computer science. This special session provides opportunities for the audience to contribute to this research discussion and to analyse and develop strategies for their own unique contexts, in a facilitated approach using the benchmark framework and key findings.
Rebecca Vivian, Katrina Falkner, Claudia Szabo
SIGCSE3
2016 Extreme User and Political Rumor Detection on Twitter
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng
ADMA3
2016 Applying Validated Pedagogy to MOOCs: An Introductory Programming Course with Media Computation
abstract
Significant advances have been made in the learning and teaching of Introductory Programming, including the integration of active and contextualised learning pedagogy. However, Massively Open Online Courses (MOOCs), where Computer Science and, more specifically, introductory programming courses dominate, do not typically adopt such pedagogies or lessons learned from more traditional learning environments. Moreover, the improvement of learning within the MOOC context in terms of discipline-specific pedagogy, and the improvement of student learning outcomes and processes have not been studied in depth.
Katrina Falkner, Nick Falkner, Claudia Szabo, Rebecca Vivian
ITiCSE3
2016 Model-driven performance prediction of systems of systems
Katrina Falkner, Claudia Szabo, Vanea Chiprianov
MoDELS2
2016 Improving Object and Event Monitoring on Twitter Through Lexical Analysis and User Profiling
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng
WISE (2)2
2016 Reduce or remove: Individual sensor reliability profiling and data cleaning
abstract
Environmental sensing using multitudes of wirelessly connected sensors is becoming critical for resolving environmental problems, given recent technology advances in the Internet of Things (IoT). Current environmental sensing projects typically deploy commodity sensors, which are known to be unreli able and prone to produce noisy and erroneous data. Moreover, the majority of current sensor data cleaning techniques have not moved beyond using the mean or the median of spatially correlated readings, thus providing unsatisfying accuracies. In this paper, we propose a sensor reliability-based cleaning method, called Influence Mean (IM), which uses weighted aggregation based on individual sensor reliabilities. We investigate whether reducing or removing unreliable sensors can be more effective to provide accurate cleaning results, by designing and testing respective algorithms on synthetic and real datasets. The experimental results show that our method generally improves the data cleaning accuracy, particularly when the behaviors of unreliable sensors vary drastically from reliable sensors.
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng
Intell. Data Anal.2
2015 An Estimation Maximization Based Approach for Finding Reliable Sensors in Environmental Sensing
abstract
Emerging Internet of Things (IoT)-based environmental sensing projects provide large-scale sensing data from individual sensors with high reading frequencies. These readings are usually produced by commodity sensors with varied reliabilities, and inevitably contain noises and errors. Most existing data cleaning techniques focus on issues such as communication overhead reduction and energy preservation, and do not take advantage of the unaggregated data from individual sensors that IoT environmental sensing projects offer. In this paper, we propose an Expectation Maximization algorithm for finding reliable sensors in environmental sensing data that assumes the preservation of individual sensor readings and high reading frequencies. Our approach simultaneously finds the environmental feature model and the faulty state of the sensors. Our extensive experiments show that the proposed approach is significantly more effective than existing approaches. Particularly, in a case where reliable sensors and faulty sensors differ significantly in their readings, the maximum squared error for other approaches exceeds 200, but for our approach is only 1.23.
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng
ICPADS2
2015 Evolution of Software Development Strategies
abstract
The development of discipline-specific cognitive and meta-cognitive skills is fundamental to the successful mastery of software development skills and processes. This development happens over time and is influenced by many factors, however its understanding by teachers is crucial in order to develop activities and materials to transform students from novice to expert software engineers. In this paper, we analyse the evolution of learning strategies of novice, first year students, to expert, final year students. We analyse reflections on software development processes from students in an introductory software development course, and compare them to those of final year students, in a distributed systems development course. Our study shows that computer science - specific strategies evolve as expected, with the majority of final year students including design before coding in their software development process, but that several areas still require scaffolding activities to assist in learning development.
Katrina Falkner, Claudia Szabo, Rebecca Vivian, Nick Falkner
ICSE (2)2
2015 Novice Code Understanding Strategies during a Software Maintenance Assignment
abstract
Existing efforts on teaching software maintenance have focussed on constructing adequate codebases that students with limited knowledge could maintain, with little focus on the learning outcomes of such exercises and of the approaches that students employ while performing maintenance. An analysis of the code understanding strategies employed by novice students as they perform software maintenance exercises is fundamental for the effective teaching of software maintenance. In this paper, we analyze the strategies employed by second year students in a maintenance exercise over a large codebase. We analyze student reflections on their code understanding, maintenance process and the use of tools. We show that students are generally capable of working with large codebases. Our study also finds that the majority of students follow a systematic approach to code understanding, but that their approach can be significantly improved through the use of tools and a better understanding of reverse engineering approaches.
Claudia Szabo
ICSE (2)1
2015 The Development of a Dashboard Tool for Visualising Online Teamwork Discussions
abstract
Many software development organisations today adopt global software engineering (GSE) and agile models, requiring software engineers to collaborate and develop software in flexible, distributed, online teams. However, many employers have expressed concern that graduates lack teamwork skills and one of the most commonly occurring problems with GSE models are issues with project management. Team managers and educators often oversee a number of teams and the large corpus of data, in combination with agile models, make it difficult to efficiently assess factors such as team role distribution and emotional climate. Current methods and tools for monitoring software engineering (SE) teamwork in both industry and education settings typically focus on member contributions, reflection, or product outcomes, which are limited in terms of real-time feedback and accurate behavioural analysis. We have created a dashboard that extracts and communicates team role distribution and team emotion information in real-time. Our proof of concept provides a real-time analysis of teamwork discussions and visualises team member emotions, the roles they have adopted and overall team sentiment during the course of a collaborative problem-solving project. We demonstrate and discuss how such a tool could be useful for SE team management and training and the development of teamwork skills in SE university courses.
Rebecca Vivian, Hamid Tarmazdi, Katrina Falkner, Nick Falkner, Claudia Szabo
ICSE (2)5
2015 Gender Gap in Academia: Perceptions of Female Computer Science Academics
abstract
Despite increased attention from Universities and Industry, the low representation of female students in Computer Science undergraduate degrees remains a major issue. Recognising this issue, leading tech companies have established strong and committed diversity initiatives but have only reached up to 17\% female representation in their tech departments. The causes of the reduced attraction and retention of female students are varied and have been widely studied, advancing the understanding of why female students do not take up or leave Computer Science. However, few analyses look at the perceptions of the females that have stayed in the field. In this paper, we explore the viewpoints of female academics and postgraduate students in Computer Science with various undergraduate backgrounds and pathways into academia. Our analysis of their interviews shows the influence of family, exposure, culture, sexism and gendered thought on their perceptions of the field, and of themselves and their peers. We identify that perceptions of identity conflict and a lack of belonging to the discipline persist even for these high-performing professionals.
Katrina Falkner, Claudia Szabo, Dee Michell, Anna Szorenyi, Shantel Thyer
ITiCSE2
2015 Using Learning Analytics to Visualise Computer Science Teamwork
abstract
Industry has called upon academia to better prepare Computer Science graduates for teamwork, especially in developing the soft skills necessary for collaborative work. However, the teaching and assessment of teamwork is not easy, with instructors being pressed for time and a lack of tools available to efficiently analyse student teamwork, where large cohorts are involved. We have developed a teamwork dashboard, founded on learning analytics, learning theory and teamwork models that analyses students' online teamwork discussion data and visualises the team mood, role distribution and emotional climate. This tool allows educators to easily monitor teams in real-time. Educators may use the tool to provide students with feedback about team interactions as well as to identify problematic teams. We present a case study, trialing the dashboard on one university Computer Science course and include reflections from the course lecturer to determine its utility in monitoring online student teamwork.
Hamid Tarmazdi, Rebecca Vivian, Claudia Szabo, Katrina Falkner, Nick Falkner
ITiCSE3
2015 Online Behavior Identification in Distributed Systems
abstract
The diagnosis, prediction, and understanding of unexpected behavior is crucial for long running, large scale distributed systems. However, existing works focus on the identification of faults in specific time moments preceded by significantly abnormal metric readings, or require a previous analysis of historical failure data. In this work, we propose an online behavior classification system to identify a wide range of undesired behaviors, which may appear even in healthy systems, and their evolution over time. We employ a two-step process involving two online classifiers on periodically collected system metrics to identify at runtime normal and anomalous behaviors such as deadlock, starvation and livelock, without any previous analysis of historical failure data. Our approach achieves over 80% accuracy in detecting unexpected behaviors and over 90% accuracy in identifying their type with a short delay after the anomalies appear, and with minimal expert intervention. Our experimental analysis uses system execution traces obtained from a Google cluster and from our in-house distributed system with varied behaviors, and shows the benefits of our approach as well as future research challenges.
Javier Álvarez Cid-Fuentes, Claudia Szabo, Katrina Falkner
SRDS2
2015 Sense and Focus: Towards Effective Location Inference and Event Detection on Twitter
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng
WISE (1)2
2015 Classifying Perspectives on Twitter: Immediate Observation, Affection, and Speculation
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng, Xiu Susie Fang
WISE (1)2
2014 Architectural Support for Model-Driven Performance Prediction of Distributed Real-Time Embedded Systems of Systems
Vanea Chiprianov, Katrina Falkner, Claudia Szabo, Gavin Puddy
ECSA3
2014 Student projects are not throwaways: teaching practical software maintenance in a software engineering course
abstract
Teaching software engineering through group-based project work supported by theory lectures is effective, as recognized by both academia and industry. However, exposing students to practical software maintenance is often overlooked in favor of building software from scratch under the guidance of a lecturer or client. The developed software is usually delivered to the lecturer/client and no maintenance efforts are further required. In contrast, industry projects require fresh graduates to perform maintenance exercises and very rarely to build software from scratch. To address this issue, existing software maintenance assignments usually focus on small codebases of very good quality, in which artificial issues are introduced. In this paper, we propose to enhance a group-based project course with a software maintenance assignment that uses a medium-sized, student-produced codebase with real software bugs. Our analysis shows the effectiveness of our approach and highlights future avenues for improvement.
Claudia Szabo
SIGCSE1
2014 Evaluating GameDevTycoon for teaching software engineering
abstract
Academia and industry recognize the effectiveness of teaching Software Engineering through group-based project work supported by lectures discussing software engineering theory. However, while undertaking such project work, only a very small number of students in the team are exposed to team leadership and project management. This is because teams usually struggle with organization and timely task completion, and there is usually no time left to rotate leadership roles. To alleviate this problem, several gaming approaches have been proposed. In this paper, we analyze GameDevTycoon, the most recent addition to such games. We include a gameplay and reflection component in our group-based project course and perform a quantitative analysis of a team management and leadership aspects that the students encountered during their gameplay. We further compare and evaluate GameDevTycoon against five other software engineering-focused games. Our analysis shows the advantages and disadvantages of using GameDevTycoon for teaching project management and highlight further directions towards better inclusion in the curriculum.
Claudia Szabo
SIGCSE1
2014 Neo-piagetian theory as a guide to curriculum analysis
abstract
The development of a coherent curriculum, encapsulating appropriate topics, learning materials and assessment, is crucial for a successful educational experience. However, designing such a curriculum is a complicated task, with challenges in tracing the development of concepts across multiple courses and ensuring that assessment is at an appropriate level at specific points in the curricula.
Claudia Szabo, Katrina Falkner
SIGCSE1
2014 Cleaning Environmental Sensing Data Streams Based on Individual Sensor Reliability
Yihong Zhang 0001, Claudia Szabo, Quan Z. Sheng
WISE (2)2
2014 Science in the Cloud: Allocation and Execution of Data-Intensive Scientific Workflows
Claudia Szabo, Quan Z. Sheng, Trent Kroeger, Yihong Zhang 0001, Jian Yu 0002
J. Grid Comput.1
2014 Behavior modeling and automated verification of Web services
Quan Z. Sheng, Zakaria Maamar, Lina Yao 0001, Claudia Szabo, Scott Bourne
Inf. Sci.4
2014 Web services composition: A decade's overview
Quan Z. Sheng, Xiaoqiang Qiao, Athanasios V. Vasilakos, Claudia Szabo, Scott Bourne, Xiaofei Xu 0001
Inf. Sci.4
2013 Model-Driven Performance Prediction of Distributed Real-Time Embedded Defense Systems
abstract
Autonomous defence systems are typically characterised by hard constraints on space, weight and power. These constraints have a strong impact on the non-functional properties, and performance, of the final system. System execution modelling tools permit early prediction of the performance of model driven systems, however the focus to date has been on understanding the performance of a model rather than determining if it meets performance requirements, and subsequently carrying out analysis to reveal the causes of any requirement violations. In this paper, we propose an integrated approach to performance prediction of model-driven distributed real time embedded defence systems. Our architectural prototyping system supports a scenario-driven experimental platform for evaluating model suitability within a set of deployment and real-time performance constraints. We present an overview of our performance prediction system, demonstrating the integration of modelling, execution and visualisation, and discuss a case study to illustrate our approach.
Katrina Falkner, Vanea Chiprianov, Nick Falkner, Claudia Szabo, James Hill, Gavin Puddy, Daniel Fraser, Adrian Johnston, Marianne Rieckmann, Andrew Wallis
ICECCS4
2013 Post-mortem analysis of emergent behavior in complex simulation models
abstract
Analyzing and validating emergent behavior in component-based models is increasingly challenging as models grow in size and complexity. Despite increasing research interest, there is a lack of automated, formalized approaches to identify emergent behavior and its causes. As part of our integrated framework for understanding emergent behavior, we propose a post-mortem emergence analysis approach that identifies the causes of emergent behavior in terms of properties of the composed model and properties of the individual model components, and their interactions. In this paper, we detail the use of reconstructability analysis for post-mortem analysis of known emergent behavior. The two-step process first identifies model components that are most likely to have caused emergent behavior, and then analyzes their interaction. Our case study using small and large examples demonstrates the applicability of our approach.
Claudia Szabo, Yong Meng Teo
SIGSIM-PADS1
2013 Formalization of emergence in multi-agent systems
abstract
Emergence is a distinguishing feature in systems, especially when complexity grows with the number of components, interactions, and connectivity. There is immense interest in emergence, and a plethora of definitions from philosophy to sciences. Despite this, there is a lack of consensus on the definition of emergence and this hinders the development of a formal approach to understand and predict emergent behavior in multi-agent systems. This paper proposes a grammar-based set-theoretic approach to formalize and verify the existence and extent of emergence without prior knowledge or definition of emergent properties. Our approach is based on weak (basic) emergence that is both generated and autonomous from the underlying agents. In contrast with current work, our approach has two main advantages. By focusing only on system interactions of interest and feasible combinations of individual agent behavior, state-space explosion is reduced. In formalizing emergence, our extended grammar is designed to model agents of diverse types, mobile agents, and open systems. Theoretical and experimental studies using the boids model demonstrate the complexity of our formal approach.
Yong Meng Teo, Ba Linh Luong, Claudia Szabo
SIGSIM-PADS3
2013 Verifying Transactional Requirements of Web Service Compositions Using Temporal Logic Templates
Scott Bourne, Claudia Szabo, Quan Z. Sheng
WISE (1)2
2012 Evolving multi-objective strategies for task allocation of scientific workflows on public clouds
abstract
With the increase in deployment of scientific application on public and private clouds, the allocation of workflow tasks to specific cloud instances to reduce runtime and cost has emerged as an important challenge. The allocation of scientific workflows on public clouds can be described through a variety of perspectives and parameters and has been proved to be NP-complete. This paper presents an optimization framework for task allocation on public clouds. We present a solution that considers important parameters such as workflow runtime, communication overhead, and overall execution cost. Our multi-objective optimization framework builds on a simple and extensible cost model and uses a heuristic to determine the optimal number of cloud instances to be used. Using the Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3) as an example, we show how our optimization heuristics lead to significantly better strategies than other state-of-the-art approaches. Specifically, our single-objective optimization is slightly better than a simple heuristic and a particle swarm optimization approach for small workflows, and achieves significant improvements for larger workflows. In a similar manner, our multi-objective optimization obtains similar results to our single-objective optimization for small-size workflows, and achieves up to 80% improvement for large-size workflows.
Claudia Szabo, Trent Kroeger
IEEE Congress on Evolutionary Computation1
2012 Ensuring Well-Formed Conversations between Control and Operational Behaviors of Web Services
Scott Bourne, Claudia Szabo, Quan Z. Sheng
ICSOC2