The recent challenge in higher education is to convey the importance of understanding concepts over rote learning. This challenge has increased in complexity with the arrival of large language model (LLM) based chatbots. Students are increasingly looking to such AI based chatbots as “sources of wisdom” instead of utilizing the same as learning aids. Despite disclaimers by the LLM creators, many students turn to the chatbot for answers to almost all learning assignments. This research work explores the level to which the LLM responses can be utilized for student learning in technical education. By understanding the contradictions between student answers and the responses generated by the LLMs, this work explores the limitations of the LLM based environments towards providing acceptable answers for assessments - specifically within the computer science engineering domain. While numerous studies have concentrated on ChatGPT, it is essential to consider the diverse range of alternative chat-bots accessible online that students may also utilize. Therefore, this work considers 5 popular AI-based chatbots for the study. With the “prompt” being the prime factor that impacts the response from chat-bots, the responses of the chatbots were collected using 2 different prompting techniques. The chatbot responses were evaluated against actual student responses by multiple reviewers to gauge its effectiveness as appropriate student answers. Both students and all chatbots were given questions aligned with the Blooms taxonomy levels (BTL) 1 to 4 in three different subjects. Each of the courses included a diverse range of questions including text-based questions, mathematical problems, and programming questions. The results show that the chatbot responses were acceptable for low BT level questions but failed to answer convincingly when asked for an algorithm. Overall, the chatbot performance (across the tested LLMs) was below average when the question set covered the BTL range 1–4. However, since the answers up to BTL2 were acceptable, LLM based chatbot answers were able to barely pass 1–2 of the 3 subjects (with the best performers scoring near the pass mark). Based on these results, it is possible to conclude that LLM based chatbots cannot be depended on for higher order learning but can be used to aid students who are struggling to pass basic courses.

The shift towards online learning through websites has been instrumental in enhancing accessibility. However, while virtual labs have been promising in generating interactive simulations of science experiments, they fall short in delivering the dynamic and immersive experiences essential for effective learning, especially in computing education courses. Recognizing this challenge, the integration of artificial intelligence (AI) into virtual labs emerges as a promising solution to aid student learning. AI-augmented virtual labs can simulate real-world scenarios, providing students with hands-on experiences in a controlled and safe environment. This work explores the impact on student learning (with focus on user engagement, skill development, comprehension, retention, and overall experience) when integrating a virtual labs module with and without AI support. Student perspectives on the effectiveness and satisfaction with the learning modules were collected using surveys that probe not only the perceived improvement in learning outcomes, but also the subjective user experience and engagement levels. Additionally, user performance indicators, based on assessments/evaluations and discussions (within the virtual labs, with and without AI integration), serve as a measure of the students' skills development, comprehension, and knowledge retention. This approach provides an objective metric to evaluate the efficacy of each learning approach. Based on data from surveys, performance indicators, and feedback analysis, the study aims to draw comprehensive conclusions and suggest potential avenues of further inquiry, regarding the effectiveness of integrating virtual labs both with AI. This integration not only enhances the learning experience, but also fosters critical thinking and problem-solving skills in students. This innovative approach offers a distinctive combination of interactivity, adaptability, and versatility. The goal of this study is not to argue for AI supported virtual lab-based learning approaches, but rather, to give empirical information about its usefulness. The findings of this study will contribute to the ongoing discussion on innovative educational practices by providing useful insights into the possible benefits and challenges of adopting hybrid learning approaches powered by AI.

It is well known that anti-malware scanners depend on malware signatures to identify malware. However, even minor modifications to malware code structure results in a change in the malware signature thus enabling the variant to evade detection by scanners. Therefore, there exists the need for a proactively generated malware variant dataset to aid detection of such diverse variants by automated antivirus scanners. This paper proposes and demonstrates a generic assembly source code based framework that facilitates any evolutionary algorithm to generate diverse and potential variants of an input malware, while retaining its maliciousness, yet capable of evading antivirus scanners. Generic code transformation functions and a novelty search supported quality metric have been proposed as components of the framework to be used respectively as variation operators and fitness function, for evolutionary algorithms. The results demonstrate the effectiveness of the framework in generating diverse variants and the generated variants have been shown to evade over 98% of popular antivirus scanners. The malware variants evolved by the framework can serve as antigens to assist malware analysis engines to improve their malware detection algorithms.