Chengran Yang, Ting Zhang, Jinfeng Jiang, Xin Zhou, Haoye Tian, Mingzhe Du, Jieke Shi, Junkai Chen, Yikun Li, Eng Lieh Ouh, Lwin Khin Shar, David Lo. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Reflective learning enhances understanding, especially when instructors promptly address difficulties raised in student reflections. Automated doubt detection can reduce time for instructors, yet existing classification approaches take substantial time for manual annotation and model training. This paper investigates whether large and small language models (LLMs, SLMs) can automate doubt detection without time-consuming training. Using a dataset of anonymized student reflections, we evaluate zero-shot, few-shot prompting, and multi-step reasoning against prior supervised classification baselines. We show that LLMs (GPT-4o, Claude-4, Gemini-2.5) surpass earlier F1 scores without prompting, while prompting further improves their performance. However, using proprietary LLMs can raise cost and privacy concerns. We also show that selected SLMs (Mistral, Qwen) outperform baselines while addressing these concerns. We extend the analysis with a category-level error study, showing that explicit doubts are detected more reliably, while tentative doubts (softened by cautious language), learning challenge (arising from difficulties in applying concepts), and masked doubts (concealed by positive or polite phrasing) are missed more often. These findings highlight both the promise and limitations of language models for doubt detection and the need to ensure that cautious or polite learners who may not express their doubts explicitly are recognized and supported in learning analytics systems.

This study investigates ChatGPT-4o's ability to answer multi-modal assessment exercises in computer science (CS) courses. While the use of large language models (LLMs) to answer text-based exercises are extensively researched, their ability to answer exercises involving artifacts of other modalities remains underexplored. To close this gap, we evaluate ChatGPT-4o's answers to 120 multi-modal CS exercises in programming, software design, human-computer interaction, statistical analysis, process analysis, and simulation. The multi-modal artifacts in these exercises include class diagrams, sequence diagrams, user interface images, analytical charts, workflow diagrams and object-flow diagrams. Our comparisons to the expected answers of these exercises show that ChatGPT-4o performs well for exercises with class and sequence diagrams possibly due to the availability of more data for training. The potential for misuse by students highlights these exercises are better suited for closed-book exams or as scaffolding activities. ChatGPT-4o answers better for those multi-modal exercises designed to assess students at the lower levels of Bloom's taxonomy than the higher levels. This discrepancy is possibly due to ChatGPT-4o's lack of understanding underlying design concepts and limited ability to generate new multi-modal artifacts, making exercises requiring higher order of cognitive thinking suitable for take-home assignment. We hope the insights from this study provide a foundation to develop effective multi-modal assessments.

This study explores the integration of a Large Language Model (LLM) based chatbot, PromptTutor, into flipped classrooms (FC) for undergraduate Computer Science (CS) education. PromptTutor is designed to provide personalized, immediate feedback to support student learning in FC by incorporating reflective learning and scaffolding strategies. The traditional FC typically lacks this immediate feedback during the pre-class learning phase, risking decreased student motivation according to existing literature. This study examines if students improve in learning outcomes and motivation after using PromptTutor. Through a controlled crossover experiment with 50 students, the study demonstrates statistically significant improvements in students' quiz performance and motivation compared to traditional FC. Our work underscores the potential of LLM-based tools in addressing FC challenges, offering actionable insights for educators and institutional leaders in technology-enhanced learning environments.

A previous study on the use of the Empath library in the prediction of Service Level Agreements (SLA) reveals the quality levels required for meaningful interaction between government customer service officers and citizens. On the other hand, past implementation of the Citizen Question-Answer system (CQAS), a type of Question-Answer model, suggests that such models if put in place can empower government customer service officers to reply faster and better with recommended answers. This study builds upon the research outcomes from both arenas of studies and introduces an innovative system design that allows the officers to incorporate the outputs from Empath X SLA predictor and CQAS (a type of Question Answer model) as critical inputs to ChatGPT engine, known as AI Based Citizen Question-Answer Recommender (ACQAR).Empath X SLA predictor anticipates the expected service response time based on citizens’ emotional state. These valuable inputs coupled with the recommended answer provided by the CQAS will serve as prompt inputs to ChatGPT to craft contextually aware responses. This ensures that the final response considers the citizen’s emotional needs, expected service timeline, and recommended answers from official government documents.While the full-scale deployment of this pilot system, ACQAR, is pending, this paper presents a comprehensive blueprint for governments seeking to modernize citizen service delivery. By fusing sentiment analysis, SLA prediction, question-answer models, and ChatGPT, this system design aims to revolutionize government-citizen interactions, delivering more empathetic, efficient, and tailored responses, while not violating SLA.This paper serves as a foundational step towards the practical development and implementation of an intelligent system (ACQAR) that holds the potential to significantly enhance citizen satisfaction, foster trust in government services, and strengthen overall government-citizen relationships.

This paper explores ChatGPT’s potential in aiding government agencies, drawing from a case study based on a government agency in Singapore. While ChatGPT’s text generation abilities offer promise, it brings inherent challenges, including data opacity, potential misinformation, and occasional errors. These issues are especially critical in government decision-making.Public administration’s core values of transparency and accountability magnify these concerns. Ensuring AI alignment with these principles is imperative, given the potential repercussions on policy outcomes and citizen trust.AI explainability plays a central role in ChatGPT’s adoption within government agencies. To address these concerns, we propose strategies like prompt engineering, data governance, and the adoption of interpretability tools such as SHapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME). These tools aid in understanding and enhancing ChatGPT’s decision-making processes.This paper underscores the urgency for government agencies to adopt a proactive stance by proposing a 4-Steps framework completed with potential measures to enhance ChatGPT’s explainability within the specific context of public administration. Collaborative efforts between AI practitioners and public administrators are essential for striking an equilibrium between the capabilities of ChatGPT and the unique demands of government operations, ultimately ensuring a responsible integration of ChatGPT into public administration processes.

In this study, we assess the efficacy of employing the ChatGPT language model to generate solutions for coding exercises within an undergraduate Java programming course. ChatGPT, a large-scale, deep learning-driven natural language processing model, is capable of producing programming code based on textual input. Our evaluation involves analyzing ChatGPT-generated solutions for 80 diverse programming exercises and comparing them to the correct solutions. Our findings indicate that ChatGPT accurately generates Java programming solutions, which are characterized by high readability and well-structured organization. Additionally, the model can produce alternative, memory-efficient solutions. However, as a natural language processing model, ChatGPT struggles with coding exercises containing non-textual descriptions or class files, leading to invalid solutions. In conclusion, ChatGPT holds potential as a valuable tool for students seeking to overcome programming challenges and explore alternative approaches to solving coding problems. By understanding its limitations, educators can design coding exercises that minimize the potential for misuse as a cheating aid while maintaining their validity as assessment tools.

Service Level Agreement (SLA) plays a significant role in the relationship between citizens and the government. It stipulates the quality levels required for the meaningful interaction between the two parties. Most SLA predictive models consider end-to-end duration and frequency of failed service requests as model inputs with little research on the analysis of textual details of the service request. This is an issue for government bodies as the latter do not just want to meet SLA, but also be proactive by knowing the citizens before assisting them. Inclusion of textual data potentially answer to this requirement of knowing the citizen before the officer tries to meet SLA. In this paper, we attempt to enrich SLA predictive process by analysing the textual data contained in the service requests. Based on a dataset of 800k case records from a customer service centre based in Singapore, we use text analytics to derive features from the dataset, which will be included with other commonly used variables in the prediction of SLA. We further explore the use of the Empath library to provide a categorical outcome that is more beneficial for the customer service officers to understand the citizen, than a numerical outcome. Based on our experiments, we observe that a predictive model built via logistic regression performs the best with an accuracy of 75%. This result remains valid when Empath categories are included as an input variable. This paper adds to the body of research work done in citizen service by proposing an SLA predictive model that incorporates lexical features from textual data to facilitate proactive citizen service delivery.

This innovative practice full paper describes our experiences of integrating information technology certifications into an undergraduate computing curriculum. As the technology landscape evolves, a common challenge for educators in computing programs is designing an industry-relevant curriculum. Over the years, industry practitioners have taken technology certifications to validate themselves against a base level of technical knowledge currently in demand in industry. Information technology (IT) certifications can also offer paths for academic computing programs to stay relevant to industry needs. However, identifying relevant IT certifications and integrating it into an academic curriculum requires a careful design approach as substantial efforts are needed by educators to design and deliver courses. This paper describes our proposed approach and experiences of identifying and mapping IT certifications for an undergraduate Information Systems (IS) curriculum. We first review our IS curriculum with an academic standard and an industry skills framework to identify relevant job roles for students. Next, we perform a web search in the Google Job Search engine for job postings relevant to these job roles and apply text analytics techniques to identify the IT certifications referenced in these job postings. We also share our implementation experiences of integrating two highly referenced IT certifications - Amazon Web Services (AWS) Cloud Practitioner and Solutions Architect - Associate certification exams in two undergraduate computing courses. We learnt that there is a need to allocate efforts for the faculty to manage the logistics matters. Students appreciate the teaching efforts and are motivated to train themselves for the certifications. In the first year of the implementation, we achieved 90% and 96% exam pass rates in AWS Cloud Practitioner certification and Solutions Architect - Associate certification, respectively. We hope that our approach and the lessons learnt can help other educators consider integrating IT certifications into their computing curricula.

This innovative practice full paper describes our experiences conducting cybersecurity capture the flag (CTF) competition for cybersecurity enthusiast participants (inclusive of both tertiary students and working professionals) local and abroad during the COVID-19 pandemic. Learning and appreciation of cybersecurity concepts for our participants with little to no technical background can be challenging. Gamification methods such as capture the flag competition style is a popular form of cybersecurity education to help participants overcome this challenge and identify talents. Participants get to apply theoretical concepts in a controlled environment, solve hands-on tasks in an informal, game-like setting and gain hands-on active learning experience. CTF competitions can be held at physical locations or virtually. However, the COVID-19 pandemic catalyses all major events that are traditionally held physically to go virtual (likewise for physical CTF events). The pandemic limits our physical interactions, changes the dynamics of our engagements with the participants and how participants learn. We have to adapt our CTF competition design and conduct it in a virtual format during the COVID-19 pandemic that is compliant with local pandemic regulations as well. This paper describes these adaptations for a semi-international CTF competition conducted for our participants. We conduct the competition entirely virtual and adapt the cybersecurity exercises to be attempted without the participant's physical presence. While we devise ways to validate participants' involvement, it is still more challenging to limit cheating than in a physical environment. However, with appropriate mitigating controls in place (reducing risks to acceptable levels), we were able to achieve similar outcomes compared to a physical event despite the lack of physical interactions. Over 1400 participants registered for our competition, and with the help of over 40 staff, we successfully conducted this 48 hours virtual CTF competition. We further analyse the participants' online activity during the competition, their survey responses after the competition and derive our lessons learnt. We hope that these experiences, analysis and findings are useful for educators or organisers who wish to adopt online CTF to improve the learning outcomes of teaching cybersecurity education.

This full paper presents an experience report describing lessons learnt from innovative practice use of technologies in an undergraduate computing capstone course. At our school, around fifty-five teams comprising of around 300 students take this course every year. With fifty-five teams, we needed a system to schedule presentations; improve communications; collaborate between stakeholders; share knowledge; monitor progress; team up students; match students to projects; improve grading process; showcase posters; and track improvements using analytics. The Learning Management Systems (LMS) is great to manage course content and grade submission. On the other hand, students are required to conduct agile sprint reviews with university internal and external stakeholders. The LMS forum requires external stakeholders to be registered and adhere to the university terms of use. Instead of using the LMS forum, we used a wiki. Wiki is a flexible platform used by various stakeholders to collaborate and contribute, we used it as a delivery platform for assessment as well. However, wiki lacks finer user access control and complex workflow features. In our capstone course we need workflows to schedule milestone presentations based on stakeholder's changing availabilities; to match students to teams using a content-based recommendation system; and to showcase project posters. We build custom systems to meet these workflow needs by proposing them as capstone course projects for our students. We share the lessons learnt from building custom systems. By sharing our experience, we hope to support more teachers willing to innovate in their use of technology in teaching.

This full paper evaluates the effectiveness of doubts identification and clarification methods applied in undergraduate computing design courses. Many undergraduate courses in computing require students to understand abstract design concepts. Exposed to the design concepts for the first time, students need to be able to identify and clarify their doubts about the abstract concepts in order to make the right design decisions. In this study, we seek to evaluate the effectiveness of six methods that help students to identify and clarify their doubts. These methods vary in their timing (immediate or delayed), communication style (online or face-to-face) and participation style (individual or group-based). We conduct surveys on the effectiveness of these methods in two undergraduate computing design courses, a course on design thinking and another on software architecture design. We evaluate the quantitative and qualitative responses for each method used. Students generally prefer methods with immediate feedback on face-to-face communication with specific questions that allow them to have another perspective of the content covered. Methods that ask self-reflective questions on what they learnt on their own are less preferred based on this survey. This study outcome illustrates that the students require more guidance to be self-directed and wants more immediate face-to-face guidance in clearing doubts on the concepts taught. We hope this study can help educators to compare and choose the suitable methods to draw out and clarify doubts for their students.

This Innovative Practice Full Paper discusses our course design on software architecture to meet the learning expectations of two groups of software engineers. Software engineers with working experiences frequently find themselves the need to upskill in their lifelong learning journey. Their learning expectations are shaped not just by their need to know but also other learning characteristics such as their working experiences. In many cases, we design courses based on the required learning outcomes and assessment criteria. In this paper, we wish to find out whether our course design on software architecture has met the learning expectations of our students over eight years. Our study data involves two groups of software engineers to upskill in two courses (1) software engineering practitioners taking a public software architecture design course (2) postgraduate students taking a Master's level software engineering programme. We explain how we evolve the course design based on their students' feedback after each run of the courses. Although the feedback of each run show encouraging results, we discover gaps in our design when we cumulatively analyze the trends of their learning expectations with their qualitative comments over the years. Our design is able to consistently meet some but not all the learning expectations that recur over the years and we discuss the reasons for this outcome and potential further interventions. We hope this discussion and trend analysis process can help course designers to improve their course design.

The 2019 edition of Stack Overflow developer survey highlights that, for the first time, Python outperformed Java in terms of popularity. The gap between Python and Java further widened in the 2020 edition of the survey. Unfortunately, despite the rapid increase in Python's popularity, there are not many testing and debugging tools that are designed for Python. This is in stark contrast with the abundance of testing and debugging tools for Java. Thus, there is a need to push research on tools that can help Python developers.

Traditionally, teaching is usually one directional where the instructor imparts knowledge and there is minimal interaction between learners and instructor. With the focus on learner-centered pedagogy, it can be a challenge to provide timely and relevant guidance to individual learners according to their levels of understanding. One of the options available is to collect reflections from learners after each lesson to extract relevant feedback so that doubts or questions can be addressed in a timely manner. In this paper, we derived an approach to automate the identification of doubts from students’ informal reflections through features analysis, word representation and machine learning. Using reflections as a feedback mechanism and aligning it to the weekly course content can pave the way to a promising approach for learner-centered teaching and personalized learning.

Software architecture remains a difficult subject for learners to grasp and for educators to teach given its level of abstraction. On the other hand, case-based learning (CBL) is a popular teaching approach used across disciplines especially in business, medicine and law where students work in groups apply their knowledge to solve real-world case studies, or scenarios using their reasoning skills and existing theoretical knowledge. In this paper, we provide how we apply case-based learning to address the challenge in teaching a postgraduate software architecture course. Our learners are postgraduate students taking a master's program in software engineering. We first describe our design of case-based learning for our software architecture course. We then analyse the survey ratings and learners' profile to evaluate the effectiveness of the proposed case-based design. These data are gathered from 9 class runs over a period of 8 years. Our analysis results show the effectiveness of our case-based design and significant relationships between this effectiveness to the learners' years of working experiences and the number of learners. Key contributions in this paper are our proposed case-based design for software architecture and the analysis findings.

Software architectures present high-level views of systems, enabling developers to abstract away the unnecessary details and focus on the overall big picture. Designing a software architecture is an essential skill in software engineering and adult learners are seeking this skill to further progress in their career. With the technology revolution and advancements in this rapidly changing world, the proportion of adult learners attending courses for continuing education are increasing. Their learning objectives are no longer to obtain good grades but the practical skills to enable them to perform better in their work and advance in their career. Teaching software architecture to upskill these adult learners requires contending with the problem of how to express this level of abstraction practically and also make the learning realistic. We provide here our seven years' history of teaching software architecture of information systems to these adult learners as a part of a public continued education program. We describe our key challenges encountered at these levels and go on to analyze the results of the course design we have taken over the years. We also compare our teaching methods based on problem-based and case-based learning and examine their effectiveness for adult learners. We hope this discussion can help educators design and improve software architecture curriculum and support education researchers in investigating pedagogical approaches and tools to better support adult learners.

This paper shares our insights on exploring the experiential learning model and risk management process to design an undergraduate software architecture course. The key challenge for undergraduate students to appreciate software architecture design is usually their limited experience in the software industry. In software architecture, the high-level design principles are heuristics lacking the absoluteness of first principles which for inexperienced undergraduate students, this is a frustrating divergence from what they used to value. From an educator's perspective, teaching software architecture requires contending with the problem of how to express this level of abstraction practically and also make the learning realistic. In this paper, we propose a model adapting the concepts of experiential learning and risk management to design the course on software architecture. The lesson plan promotes active learning with activities to observe how different parts of an information systems architecture work, experience the impact of real software quality issues or risks arise, reflect on the root causes of these risks, conceptualize and subsequently implement the countermeasure to mitigate the risk. We divide the course into first session conducted based on the traditional lecture format and second session based on our proposed experiential risk learning model. We evaluate the feedback ratings of 128 undergraduate students of an information system program for the two sessions and perform sentiment analysis on their comments. We also generalize the applicability of our experiential risk learning model to courses in other domains of software engineering. The key contribution of this paper is the experiential risk learning model. We hope that this model alleviates the challenge to design a software architecture course for undergraduates and can be used as another teaching method for active learning.