Large language models (LLMs) are quickly being adopted in a wide range of learning experiences, especially via ubiquitous and broadly accessible chat interfaces like ChatGPT. This type of interface is readily available to students and teachers around the world. Coding education is an interesting test case, both because LLMs have strong performance on coding tasks, and because LLM-powered support tools are rapidly becoming part of the workflow of professional software engineers. To help understand the impact of generic LLM use on coding education, we conducted a large-scale randomized control trial with 5,831 students from 146 countries in an online coding class in which we provided some students with access to a chat interface with GPT-4. Under some assumptions, we estimate positive benefits on exam performance for adopters, the students who used the tool, but over all students, the advertisement of GPT-4 led to a significant average decrease in exam participation. We observe similar decreases in other forms of course engagement. However, this decrease is modulated by the student's country of origin. Offering access to LLMs to students from low human development index countries increased their exam participation rate on average. Our results suggest there may be promising benefits to using LLMs in an introductory coding class, but also potential harms for engagement, which makes their longer term impact on student success unclear. Our work highlights the need for additional investigations to help understand the potential impact of future adoption and integration of LLMs into classrooms.

Forming social relationships is critical to student success and well-being, but is one of the first aspects to be neglected in the design of massive online courses. We present our experience deploying an in-course networking tool that enabled 1,600+ learners and teachers in a massive online CS1 course to form 2,000+ connections with other individuals. We discuss how social preferences and networking goals vary by demographics, economic factors, course goals, and course role. Contrary to usual online social behavior, users in our network sent more out-group requests than a random baseline by role (2.04x), gender (1.1x), and developing vs. developed country (1.07x). We highlight differences between developing vs. developed country users: developing country users send 2.5x requests and make, on average, 1.78x as many connections as those from developed countries. From a randomized control trial we find that random recommendations increase the volume of sent requests by 44.48% and promote cross-group requests across developing vs. developed countries (+28.9%), age (+15.1%), and gender (+8.6%). Ultimately we show that integrating socialization as a core feature of online CS1 classrooms can help support people from all backgrounds in achieving their diverse educational goals, which often extend well beyond improving coding proficiency.

One-on-one help from a teacher is highly impactful for students, yet extremely challenging to support in massive online courses (MOOCs). In this work, we present TeachNow: a novel system that lets volunteer teachers from anywhere in the world instantly provide 1:1 help sessions to students in MOOCs, without any scheduling or coordination overhead. TeachNow works by quickly finding an online student to help and putting them in a collaborative working session with the teacher. The spontaneous, on-demand nature of TeachNow gives teachers the flexibility to help whenever their schedule allows.

We propose and carry-out a novel method of formative assessment called Assessment via Teaching (AVT), in which learners demonstrate their understanding of CS1 topics by tutoring more novice students. AVT has powerful benefits over traditional forms of assessment: it is centered around service to others and is highly rewarding for the learners who teach. Moreover, teaching greatly improves the learners' own understanding of the material and has a huge positive impact on novices, who receive free 1:1 tutoring. Lastly, this form of assessment is naturally difficult to cheat---a critical property for assessments in the era of large-language models. We use AVT in a randomised control trial with learners in a CS1 course at an R1 university. The learners provide tutoring sessions to more novice students taking a lagged online version of the same course. We show that learners who do an AVT session before the course exam performed 20 to 30 percentage points better than the class average on several questions. Moreover, compared to students who did a practice exam, the AVT learners enjoyed their experience more and were twice as likely to study for their teaching session. We believe AVT is a scalable and uplifting method for formative assessment that could one day replace traditional exams.

Teaching students how to write code that is elegant, reusable, and comprehensible is a fundamental part of CS1 education. However, providing this "style feedback" in a timely manner has proven difficult to scale. In this paper, we present our experience deploying a novel, real-time style feedback tool in Code in Place, a large-scale online CS1 course. Our tool is based on the latest breakthroughs in large-language models (LLMs) and was carefully designed to be safe and helpful for students. We used our Real-Time Style Feedback tool (RTSF) in a class with over 8,000 diverse students from across the globe and ran a randomized control trial to understand its benefits. We show that students who received style feedback in real-time were five times more likely to view and engage with their feedback compared to students who received delayed feedback. Moreover, those who viewed feedback were more likely to make significant style-related edits to their code, with over 79% of these edits directly incorporating their feedback. We also discuss the practicality and dangers of LLM-based tools for feedback, investigating the quality of the feedback generated, LLM limitations, and techniques for consistency, standardization, and safeguarding against demographic bias, all of which are crucial for a tool utilized by students.

We show how any PAC learning algorithm that works under the uniform distribution can be transformed, in a blackbox fashion, into one that works under an arbitrary and unknown distribution ‍D. The efficiency of our transformation scales with the inherent complexity of ‍D, running in (n, (md)d) time for distributions over n whose pmfs are computed by depth-d decision trees, where m is the sample complexity of the original algorithm. For monotone distributions our transformation uses only samples from ‍D, and for general ones it uses subcube conditioning samples.

We initiate the study of a fundamental question concerning adversarial noise models in statistical problems where the algorithm receives i.i.d. draws from a distribution $\mathcal{D}$. The definitions of these adversaries specify the {\sl type} of allowable corruptions (noise model) as well as {\sl when} these corruptions can be made (adaptivity); the latter differentiates between oblivious adversaries that can only corrupt the distribution $\mathcal{D}$ and adaptive adversaries that can have their corruptions depend on the specific sample $S$ that is drawn from $\mathcal{D}$. We investigate whether oblivious adversaries are effectively equivalent to adaptive adversaries, across all noise models studied in the literature, under a unifying framework that we introduce. Specifically, can the behavior of an algorithm $\mathcal{A}$ in the presence of oblivious adversaries always be well-approximated by that of an algorithm $\mathcal{A}’$ in the presence of adaptive adversaries? Our first result shows that this is indeed the case for the broad class of {\sl statistical query} algorithms, under all reasonable noise models. We then show that in the specific case of {\sl additive noise}, this equivalence holds for {\sl all} algorithms. Finally, we map out an approach towards proving this statement in its fullest generality, for all algorithms and under all reasonable noise models.

Using the framework of boosting, we prove that all impurity-based decision tree learning algorithms, including the classic ID3, C4.5, and CART, are highly noise tolerant. Our guarantees hold under the strongest noise model of nasty noise, and we provide near-matching upper and lower bounds on the allowable noise rate. We further show that these algorithms, which are simple and have long been central to everyday machine learning, enjoy provable guarantees in the noisy setting that are unmatched by existing algorithms in the theoretical literature on decision tree learning. Taken together, our results add to an ongoing line of research that seeks to place the empirical success of these practical decision tree algorithms on firm theoretical footing.

The Internet of Things (IoT) is revolutionising nearly every aspect of modern life, playing an ever greater role in both industrial and domestic sectors. The increasing frequency of cyber-incidents is a consequence of the pervasiveness of IoT. Threats are becoming more sophisticated, with attackers using new attacks or modifying existing ones. Security teams must deal with a diverse and complex threat landscape that is constantly evolving. Traditional security solutions cannot protect such systems adequately and so researchers have begun to use Machine Learning algorithms to discover effective defence systems. In this paper, we investigate how one approach from the domain of evolutionary computation - grammatical evolution - can be used to identify cyberattacks in IoT environments. The experiments were conducted on up-to-date datasets and compared with state-of-the-art algorithms. The potential application of evolutionary computation-based approaches to detect unknown attacks is also examined and discussed.

Data center networks (DCNs) are the backbone of many cloud and Internet services. They are vulnerable to link failures, that occur on a daily basis, with a high frequency. Service disruption due to link failure may incur financial losses, compliance breaches and reputation damage. Performance metrics such as packet loss and routing flaps are negatively affected by these failure events. We propose a new Bayesian learning approach towards adaptive path allocation that aims to improve DCN performance by reducing both packet loss and routing flaps ratios. The proposed approach incorporates historical information about link failure and usage probabilities into its allocation procedure, and updates this information on-the-fly during DCN operational time. We evaluate the proposed framework using an experimental platform built with the POX controller and the Mininet emulator. Compared with a benchmark shortest path algorithm, the results show that the proposed methods perform better in terms of reducing the packet loss and routing flaps.

Could it be the case that the number of people who want to teach computer science, and have the potential, is roughly proportional to the number of people who want to learn? During the time of COVID-19 we offered a free CS1 class to people around the world. Well-aware of the high drop-out rates reported in many massive open-access online courses (MOOCs), we augmented our course with a scalable, human-centered solution: section leading. Section leaders teach small, weekly interactive learning sessions. We hypothesize that the personalized attention adds a sense of responsibility for both student and teacher which drives learning. We recruited over 900 volunteer section leaders and more than 10,000 students in the class. To our knowledge this is the largest group of section leaders in a single CS1 course offering and the most small group interactions. The completion rate in our class was more than 10 times that usually reported for similar MOOCs. Additionally, 99% of the volunteer section leaders taught through the entire span of the course, showing the potential for large scale volunteer-driven education, and the benefit that teachers themselves derive. We also discovered the potential for replication of this model, as 34% of students in a representative-sample survey indicated they would serve as section leaders for a future offering of the course. This level of participation would be more than sufficient to field additional offerings of the course sustainably. We believe this is an intriguing case study of a model for significantly scaling human-centric CS education for all.

Chart-based visual acuity measurements are used by billions of people to diagnose and guide treatment of vision impairment. However, the ubiquitous eye exam has no mechanism for reasoning about uncertainty and as such, suffers from a well-documented reproducibility problem. In this paper we make two core contributions. First, we uncover a new parametric probabilistic model of visual acuity response based on detailed measurements of patients with eye disease. Then, we present an adaptive, digital eye exam using modern artificial intelligence techniques which substantially reduces acuity exam error over existing approaches, while also introducing the novel ability to model its own uncertainty and incorporate prior beliefs. Using standard evaluation metrics, we estimate a 74% reduction in prediction error compared to the ubiquitous chart-based eye exam and up to 67% reduction compared to the previous best digital exam. For patients with eye disease, the novel ability to finely measure acuity from home could be a crucial part in early diagnosis. We provide a web implementation of our algorithm for anyone in the world to use. The insights in this paper also provide interesting implications for the field of psychometric Item Response Theory.

Recently, the rapid development of automotive industries has given rise to large multidimensional datasets both in the production sites and after-sale services. Fault diagnostic systems are one of the services that the automotive industries provide. As a consequence of the rapid development of cars features, traditional rule-based diagnostic systems became very limited. Therefore, more sophisticated AI approaches need to be investigated towards more efficient solutions. In this paper, we focus on utilising deep learning so as to build a diagnostic system that is able to estimate the required services in an efficient and effective way. We propose a new model, called Deep Symptoms-Based Model Deep-SBM, as an approach to predict a wide range of faults by relying on the deep learning technique. The new proposed model is validated through a set of experiments in order to demonstrate how the underlying model runs and its impact on improving the overall performance metrics. We have applied the Deep-SBM on a real historical diagnostic data provided by Cognitran Ltd. The performance of the Deep-SBM was compared against the state-of-the-art approaches and better result has been reported in terms of accuracy, precision, recall, and F-Score. Based on the obtained results, some further directions are suggested in this context. The final goal is having fault prediction data collected online relying on IoT.

Failure incidents result in temporarily preventing the network from delivering services properly. Such a deterioration in services called service unavailability. The traditional fault management techniques, i.e. protection and restoration, are inevitably concerned with service unavailability due to the convergence time that is required to achieve the recovery when a failure occurs. However, with the global view feature of software-defined networking a failure prediction is becoming attainable, which in turn reduces the service interruptions that originated by failures. In this paper, we propose a proactive restoration technique that reconfigure the vulnerable routes which are likely to be affected if the predicted failure indeed occurs. The proposed approach allocates the alternative routes based on the probability of failure. Experimental evaluation on real-world and synthetic topologies demonstrates that the network service availability can be improved with the proposed technique to reach up to 97%. Based on the obtained results, further directions are suggested towards achieving further advances in this research area.

A crucial requirement for the network service provider is to satisfy the Service Level Agreements (SLA) that it has made with its customers. Coexisting network tenants may have agreed different SLAs, and thus, the service provider must be able to provide QoS differentiation in order to meet his contractual commitments. Current one-size-fits-all routing models are not appropriate for all network tenants if their individual SLA requirements are to be efficiently met. We propose a SDN-based multi-cost routing approach which allocates network resources based on a portfolio of tenant SLA, which achieves the goal of accommodating multiple tenants, given their SLAs. This routing approach allocates routes based on both the hop count and the probability of link failure. Experimental evaluation demonstrates that the assignment of network paths to tenants is prioritised according to the SLA class of the tenant. Differentiation between tenants who have different SLAs is achieved. Finally, we demonstrate how the routing model operates and how it impacts upon the provision of different levels of service.

In recent years, the emerging paradigm of software-defined networking has become a hot and thriving topic in both the industrial and academic sectors. Software-defined networking offers numerous benefits against legacy networking systems by simplifying the process of network management through reducing the cost of network configurations. Currently, data plane fault management is limited to two mechanisms: proactive and reactive. These fault management and recovery techniques are activated only after a failure occurrence and hence packet loss is highly likely to occur. This is due to convergence time where new network paths will need to be allocated in order to forward the affected traffic rather than drop it. Such convergence leads to temporary service disruption and unavailability. Practically, not only the speed of recovery mechanisms affects the convergence, but also the delay caused by the process of failure detection. In this paper, we define a new approach for data plane fault management in software-defined networks where the goal is to eliminate the convergence process altogether rather than accelerate the failure detection and recovery. We propose a new framework, called Smart Routing, which allows the network controller to receive forewarning signs on failures and hence avoid risky paths before the failure incidents occur. The proposed approach aims to decrease service disruption, which in turn increases network service availability. We validate our framework through a set of experiments that demonstrate how the underlying model runs and its impact on improving service availability. We take as example of the applicability of the new framework three types of topologies covering real and simulated networks.

Estimates of predictive uncertainty are important for accurate model-based planning and reinforcement learning. However, predictive uncertainties — especially ones derived from modern deep learning systems — can be inaccurate and impose a bottleneck on performance. This paper explores which uncertainties are needed for model-based reinforcement learning and argues that ideal uncertainties should be calibrated, i.e. their probabilities should match empirical frequencies of predicted events. We describe a simple way to augment any model-based reinforcement learning agent with a calibrated model and show that doing so consistently improves planning, sample complexity, and exploration. On the \textsc{HalfCheetah} MuJoCo task, our system achieves state-of-the-art performance using 50% fewer samples than the current leading approach. Our findings suggest that calibration can improve the performance of model-based reinforcement learning with minimal computational and implementation overhead.

I suspect that students learn more from our programming assignments than from our much worried-over lectures, with their slide transitions and attempts at live coding in lecture. A great assignment is deliberate about where the student hours go, concentrating the student's attention on material that is interesting and useful. The best assignments solve a problem that is topical and entertaining, providing motivation for the whole stack of work. Unfortunately, creating great programming assignments is both time consuming and error prone. The Nifty Assignments special session is all about promoting and sharing the ideas and ready-to-use materials of successful assignments.

In this paper, we introduce a new approach that computes the shortest-reliable end-to-end paths for centrally controlled networks like software-defined networks (SDNs). The proposed method aims to find the correlation between the routing mechanism and reliability with the purpose of decreasing the required time of backup path installation through reducing the number of required rules at the moment of failure towards guarantee the fast restoration of the affected path, hence leading to the reduction of the overhead on SDN network controller and the probability of the loss of packets. We also investigate the correlation between the network topology and its reliability and demonstrate the benefits from this relation through experiments using well-known SDN network simulation tools.