ABSTRACT Objectives Cookiecutter is a popular, mature open‐source Python library for automating the creation of customized projects from templates. This kind of scaffolding is useful for enabling reuse, encapsulating expertise, achieve uniformity, and facilitating project creation for a range of languages and domains, including microservices, web applications, and data science. Despite their success, there is a lack of descriptions of general‐purpose project templating tools such as Cookiecutter in the literature. The objective of the paper is to provide a description of Cookiecutter that is useful for researchers and practitioners. Methods Our work is informed by our own use of Cookiecutter in the context of an industrial project. We describe Cookiecutter with the help of four different research questions. The first two relate to how Cookiecutter works (RQ1) and how it is used (RQ2). The latter two are concerned with providing guidance to users of Cookiecutter (RQ3) and identifying challenges that they may face (RQ4). Result Our answer to question RQ1 provides a succinct, high‐level description of the structure of Cookiecutter templates and Cookiecutter's execution semantics. For question RQ2, we provide an analysis of the 100 most popular Cookiecutter templates on GitHub and descriptions of three applications of Cookiecutter in different domains. For question RQ3, we identify quality attributes for Cookiecutter projects together with best practice recommendations. For question RQ4, our discussion of challenges is structured around different lifecycle activities related to the overall management of Cookiecutter templates. The potential for research results in related areas such as software product lines, feature and variability modeling, and model‐driven engineering to help address these challenges is highlighted. Conclusion The paper provides a comprehensive discussion of Cookiecutter, a successful general‐purpose project templating tool with demonstrated industrial use. The discussion covers fundamental and practical aspects of Cookiecutter and thus targets practitioners as well as researchers interested in general‐purpose templating and Cookiecutter in particular.

One of the main challenges developers face in the use of continuous integration (CI) and deployment pipelines is the occurrence of intermittent job failures, which result from unexpected non-deterministic issues (e.g., flaky tests or infrastructure problems) rather than regular code-related errors such as bugs. Prior studies developed machine learning (ML) models trained on large datasets of job logs to classify job failures as either intermittent or regular. As an alternative to costly manual labeling of large datasets, the state-of-the-art (SOTA) approach leveraged a heuristic based on non-deterministic job reruns. However, this method mislabels intermittent job failures as regular in contexts where rerunning suspicious job failures is not an explicit policy, and therefore limits the SOTA's performance in practice. In fact, our manual analysis of 2,125 job failures from 5 industrial and 1 open-source projects reveals that, on average, 32 % of intermittent job failures are mislabeled as regular. To address these limitations, this paper introduces a novel approach to intermittent job failure detection using fewshot learning (FSL). Specifically, we fine-tune a small language model using a few number of manually labeled log examples to generate rich embeddings, which are then used to train an ML classification head. Our FSL-based approach achieves 70 - 88% F1-score with only 12 shots in all projects, outperforming the SOTA, which proved ineffective (34-52% F1-score) in 4 projects. Overall, this study underlines the importance of data quality over quantity and provides a more efficient and practical framework for the detection of intermittent job failures in organizations.

Code review is a fundamental practice in software engineering, ensuring code quality, fostering collaboration, and reducing defects. While research has extensively examined various aspects of this process, most studies assume that all code reviews follow a standardized evaluation workflow. However, our industrial partner, which uses Merge Requests (MRs) mechanism for code review, reports that this assumption does not always hold in practice. Many MRs serve alternative purposes beyond rigorous code evaluation. These MRs often bypass the standard review process, requiring minimal oversight. We refer to these cases as deviations, as they disrupt expected workflow patterns. For example, work-in-progress (WIP) MRs may be used as draft implementations without the intention of being reviewed, MRs with huge changes are often created for code rebase, and library updates typically involve dependency version changes that require minimal or no review effort. We hypothesize that overlooking MR deviations can lead to biased analytics and reduced reliability of machine learning (ML) models used to explain the code review process. This study addresses these challenges by first identifying MR deviations. Our findings show that deviations occur in up to 37.02 % of MRs across seven distinct categories. In addition, we develop a detection approach leveraging few-shot learning, achieving up to 91 % accuracy in identifying these deviations. Furthermore, we examine the impact of removing MR deviations on ML models predicting code review completion time. Removing deviations significantly enhances model performance in 53.33 % of cases, with improvements of up to 2.25 times. Additionally, their exclusion significantly impacts model interpretation, strongly altering overall feature importance rankings in 47 % of cases and top-k rankings in 60 %. Our contributions include: (1) a clear definition and categorization of MR deviations, (2) a novel AI-based detection method leveraging few-shot learning, and (3) an empirical analysis of their exclusion impact on ML models explaining code review completion time. Our approach helps practitioners streamline review workflows, allocate reviewer effort more effectively, and ensure more reliable insights from MR analytics.

Documentation comments are essential for maintainability, yet they are often missing or outdated. This is true not only for programs in general-purpose languages, but also for artifacts in other languages often found in software projects such as scripts or configuration files. To address this problem, we present an approach that uses Large Language Models (LLMs) to generate header comments (aka, ‘block comments’ or ‘doc-strings’) for elements of different languages in different documentation formats. Given a file in some language and a description of elements in the file to be documented and the documentation format to be used, the approach generates header comments for all undocumented elements in the file that is guaranteed to conform to the documentation format. We describe a prototype implementation and its integration into an industrial development pipeline. Feedback from our industrial partner, an LLM-as-judge evaluation, and the participants of a user study involving a broad range of languages indicates that the approach is viable, able to produce sufficiently high-quality documentation in general, and holds potential for improving industrial documentation practices across different programming languages and teams.

Intent-Based Networking (IBN) presents a paradigm shift for network management, by promising to align intents and business objectives with network operations–in an automated manner. However, its practical realization is challenging: 1) processing intents, i.e., translate, decompose and identify the logic to fulfill the intent, and 2) intent conformance, that is, considering dynamic networks, the logic should be adequately adapted to assure intents. To address the latter, intent assurance is tasked with continuous verification and validation, including taking the necessary actions to align the operational and target states. In this paper, we define an assurance framework that allows us to detect and act when intent drift occurs. To do so, we leverage AI-driven policies, generated by Large Language Models (LLMs) which can quickly learn the necessary in-context requirements, and assist with the fulfillment and assurance of intents.

Automated management requires decomposing high-level user requests, such as intents, to an abstraction that the system can understand and execute. This is challenging because even a simple intent requires performing a number of ordered steps. And the task of identifying and adapting these steps (as conditions change) requires a decomposition approach that cannot be exactly pre-defined beforehand. To tackle these challenges and support automated intent decomposition and execution, we explore the few-shot capability of Large Language Models (LLMs). We propose a pipeline that progressively decomposes intents by generating the required actions using a policy-based abstraction. This allows us to automate the policy execution by creating a closed control loop for the intent deployment. To do so, we generate and map the policies to APIs and form application management loops that perform the necessary monitoring, analysis, planning and execution. We evaluate our proposal with a use-case to fulfill and assure an application service chain of virtual network functions. Using our approach, we can generalize and generate the necessary steps to realize intents, thereby enabling intent automation for application management.

Managing complex infrastructures in multi-domain settings is time-consuming and error-prone. Intent-based infrastructure management is a means to simplify management by allowing users to specify intents, i.e., high-level statements in natural language, that are automatically realized by the system. However, providing intent-based multi-domain infrastructure management poses a number of challenges: 1) intent translation; 2) plan execution and parallelization; 3) incompatible cross-domain abstractions. To tackle these challenges, we propose AppleSeed, an intent-based infrastructure management system that enables an end-to-end intent-to-deployment pipeline. AppleSeed uses few-shot learning for training a Large Language Model (LLM) to translate intents into intermediate programs, which are processed by a just-in-time compiler and a materialization module to automatically generate parallelizable, domain-specific executable programs. We evaluate the system in two use cases: Deep Packet Inspection (DPI); and machine learning training and inferencing. Our system achieves efficient intent translation into an execution plan with an average 22.3x lines of code to intent word ratio. It also speeds up the execution of the management plan by 1.7-2.6 times with our JIT compilation for parallelized execution compared to sequential execution.

End-to-end delay is a critical attribute of quality of service (QoS) in application domains such as cloud computing and computer networks. This metric is particularly important in tandem service systems, where the end-to-end service is provided through a chain of services. Service-rate control is a common mechanism for providing QoS guarantees in service systems. In this paper, we introduce a reinforcement learning-based (RL-based) service-rate controller that provides probabilistic upper-bounds on the end-to-end delay of the system, while preventing the overuse of service resources. In order to have a general framework, we use queueing theory to model the service systems. However, we adopt an RL-based approach to avoid the limitations of queueing-theoretic methods. In particular, we use Deep Deterministic Policy Gradient (DDPG) to learn the service rates (action) as a function of the queue lengths (state) in tandem service systems. In contrast to existing RL-based methods that quantify their performance by the achieved overall reward, which could be hard to interpret or even misleading, our proposed controller provides explicit probabilistic guarantees on the end-to-end delay of the system. The evaluations are presented for a tandem queueing system with non-exponential inter-arrival and service times, the results of which validate our controller's capability in meeting QoS constraints.

Network traffic classification using machine learning techniques has been widely studied. Most existing schemes classify entire traffic flows, but there are major limitations to their practicality. At a network router, the packets need to be processed with minimum delay, so the classifier cannot wait until the end of the flow to make a decision. Furthermore, a complicated machine learning algorithm can be too computationally expensive to implement inside the router. In this paper, we introduce flow-packet hybrid traffic classification (FPHTC), where the router makes a decision per packet based on a routing policy that is designed through transferring the learned knowledge from a flow-based classifier residing outside the router. We analyze the generalization bound of FPHTC and show its advantage over regular packet-based traffic classification. We present experimental results using a real-world traffic dataset to illustrate the classification performance of FPHTC. We show that it is robust toward traffic pattern changes and can be deployed with limited computational resource.

Large datasets in machine learning often contain missing data, which necessitates the imputation of missing data values. In this work, we are motivated by network traffic classification, where traditional data imputation methods do not perform well. We recognize that no existing method directly accounts for classification accuracy during data imputation. Therefore, we propose a joint data imputation and data classification method, termed generative adversarial classification network (GACN), whose architecture contains a generator network, a discriminator network, and a classification network, which are iteratively optimized toward the ultimate objective of classification accuracy. For the scenario where some data samples are unlabeled, we further propose an extension termed semi-supervised GACN (SS-GACN), which is able to use the partially labeled data to improve classification accuracy. We conduct experiments with real-world network traffic data traces, which demonstrate that GACN and SS-GACN can more accurately impute data features that are more important for classification, and they outperform existing methods in terms of classification accuracy.

Malicious data manipulation reduces the effectiveness of machine learning techniques, which rely on accurate knowledge of the input data. Motivated by real-world applications in network flow classification, we address the problem of robust online learning with delayed feedback in the presence of malicious data generators that attempt to gain favorable classification outcome by manipulating the data features. We propose online algorithms termed ROLC-NC and ROLC-C when the malicious data generators are non-clairvoyant and clairvoyant, respectively. We derive regret bounds for both algorithms and show that they are sub-linear under mild conditions. We further evaluate the proposed algorithms in network flow classification via extensive experiments using real-world data traces. Our experimental results demonstrate that both algorithms can approach the performance of an optimal static offline classifier that is not under attack, while outperforming the same offline classifier when tested with a mixture of normal and manipulated data.

Malicious data manipulation reduces the effectiveness of machine learning techniques, which rely on accurate knowledge of the input data. Motivated by real-world applications in network flow classification, we address the problem of robust online learning with delayed feedback in the presence of malicious data generators that attempt to gain favorable classification outcome by manipulating the data features. When the feedback delay is static, we propose online algorithms termed ROLC-NC and ROLC-C when the malicious data generators are non-clairvoyant and clairvoyant, respectively. We then consider the dynamic delay case, for which we propose online algorithms termed ROLC-NC-D and ROLC-C-D when the malicious data generators are non-clairvoyant and clairvoyant, respectively. We derive regret bounds for these four algorithms and show that they are sub-linear under mild conditions. We further evaluate the proposed algorithms in network flow classification via extensive experiments using real-world data traces. Our experimental results demonstrate that the proposed algorithms can approach the performance of an optimal static offline classifier that is not under attack, while outperforming the same offline classifier when tested with a mixture of normal and manipulated data.

Ensuring quality of service (QoS) guarantees in service systems is a challenging task, particularly when the system is composed of more fine-grained services, such as service function chains. An important QoS metric in service systems is the end-to-end delay, which becomes even more important in delay-sensitive applications, where the jobs must be completed within a time deadline. Admission control is one way of providing end-to-end delay guarantee, where the controller accepts a job only if it has a high probability of meeting the deadline. In this paper, we propose a reinforcement learning-based admission controller that guarantees a probabilistic upper-bound on the end-to-end delay of the service system, while minimizes the probability of unnecessary rejections. Our controller only uses the queue length information of the network and requires no knowledge about the network topology or system parameters. Since long-term performance metrics are of great importance in service systems, we take an average-reward reinforcement learning approach, which is well suited to infinite horizon problems. Our evaluations verify that the proposed RL-based admission controller is capable of providing probabilistic bounds on the end-to-end delay of the network, without using system model information.

Network flow classification is essential to proper provisioning of Quality of Service (QoS). Conventional machine-learning based flow classification methods assume reliable knowledge of the flow features. However, in practice, malicious flow generators can manipulate the flow features to increase the likelihood of certain learning outcomes, e.g., in terms of the QoS requirement label. Training a classifier that is robust to such feature manipulation is imperative. In this work, we present a study on robust flow classification against malicious feature manipulation. We leverage a detailed system model to capture the relation between the classifier and malicious flow generators and propose a Stackelberggame based solution framework to train a robust classifier. We conduct extensive experimentation using real-world traces. For flows with manipulated features, the Stackelberg classifier trained by our solution framework significantly outperforms a non-robust classifier that is oblivious to manipulation, achieving accuracy close to that of the non-robust classifier on unmanipulated flows. Furthermore, the Stackelberg classifier on manipulated test flows is no worse than the non-robust classifier on unmanipulated flows.

Ensuring the conformance of a service system's end-to-end delay to service level agreement (SLA) constraints is a challenging task that requires statistical measures beyond the average delay. In this paper, we study the real-time prediction of the end-to-end delay distribution in systems with composite services such as service function chains. In order to have a general framework, we use queueing theory to model service systems, while also adopting a statistical learning approach to avoid the limitations of queueing-theoretic methods such as stationarity assumptions or other approximations that are often used to make the analysis mathematically tractable. Specifically, we use deep mixture density networks (MDN) to predict the end-to-end distribution of the delay given the network's state. As a result, our method is sufficiently general to be applied in different contexts and applications. Our evaluations show a good match between the learned distributions and the simulations, which suggest that the proposed method is a good candidate for providing probabilistic bounds on the end-to-end delay of more complex systems where simulations or theoretical methods are not applicable.

Motivated by interest in providing more efficient services in customer service systems, we use statistical learning methods and delay history information to predict the conditional distribution of the customers' waiting times in queueing systems. From the predicted distributions, descriptive statistics of the system such as mean, variance and percentiles of the waiting times can be obtained, which can be used for delay announcements, SLA conformance and better system management. We model the distributions by mixtures of Gaussians, parameters of which can be estimated using Mixture Density Networks. We use the extensions of the Lindley's equation for multi-server queues to generate our datasets. The evaluations show that exploiting more delay history information can result in much more accurate predictions under realistic time-varying arrival assumptions.

Network function visualization (NFV) has emerged as a promising paradigm in networking, where the hardware-based middleboxes are replaced with software-based virtualized entities typically running on the cloud to provide specific functionalities. By deploying NFV, network services become more adaptive and cost-effective. Many multicast services such as real-time multimedia streaming and intrusion detection require appropriate services chaining; however, NFVs placement in the network as well as traffic routing strategy to guarantee that the multicast flows traverse through the services chain before reaching the end user is still an open problem. In this paper, we present an algorithm to solve this problem.

We consider the minimization of electricity cost of a core network where each node has access to solar renewable energy and energy storage in a time-of-use pricing environment. Using an optimization based approach we demonstrate that expenditure on electricity can be reduced by 60% through an effective energy management policy. We also present a distributed, greedy energy management algorithm, which makes hourly electricity purchase and energy storage decisions at each of the nodes in the network and performs close to the optimal case. Finally, we measure the impact of parameters including solar panel size, energy storage capacity and storage charging rate as well as seasonal variations of solar energy on the service provider's expenditure on electricity.

Virtual machine (VM) migration occurs very frequently in cloud computing. VM Migration enables a running OS, including memory and storage to move from one physical host to another physical host. A particular case of interest is live migration where the process of migrating the full state from one OS to the other should happen continuously and without any connection disruption. In order to have a seamless VM migration process the system has to be able to resume network connectivity very quickly. Fast resumption has proved to be a challenging problem. In this paper, we present a scheme to efficiently to migrate VMs networking resources using SDN techniques to achieve fast network flow resumption on SDN. We formulate the problem by an integer programming problem, we prove its NP-completeness and we propose a heuristic algorithm to solve the problem. Software simulation and real testbed implementation are done to demonstrate the performance of the flow migration scheme.

Due to the time-varying nature of wireless networks, it is required to find robust optimal methods to control the behavior and performance of such networks; however, this is a challenging task since robustness metrics and QoS-based (Quality of service) constraints in a wireless environment are typically highly non-linear and non-convex. This paper explores the possibility of using graph theoretic metrics to provide robustness in a wireless network at the presence of a set of QoS constraints. In particular, we are interested in robust planning of a wireless network for a given demand matrix while preserving end-to-end delay for input demands below a given threshold set. To this end, we show that the upper bound of end-to-end round trip time between two nodes of a network can be approximated by point-to-point network criticality (or resistance distance) of the network. We construct a convex optimization problem to provide a delay-guaranteed jointly optimal allocation of transmit powers and link flows. We show that the solution provides a robust behavior, i.e. it is insensitive to the environmental changes such as wireless link disruption, this is expected because network criticality is a robustness metric. Our framework can be applied to a wide range of SINR (Signal to Interference plus Noise Ratio) values.

This paper proposes a robust Criticality-based Clustering Algorithm (CCA) for Vehicular Ad Hoc NETworks (VANETs) based on the concept of network criticality. Network criticality is a global metric on an undirected graph, that quantifies the robustness of the graph against environmental changes such as topology. In this paper, we localize the notion of network criticality and apply it to control cluster formation in the vehicular wireless network. We use the localized notion of node criticality together with a universal link measure, Link Expiration Time (LET), to derive a distributed multi-hop clustering algorithm for VANETs. Simulation results show that the proposed CCA forms robust cluster structures.

The continuously increasing complexity of communication networks and the increasing diversity and unpredictability of traffic demand has led to a consensus view that the automation of the management process is inevitable. Currently, network and service management techniques are mostly manual, requiring human intervention, and leading to slow response times, high costs, and customer dissatisfaction. In this paper we present AutoNet, a self-organizing management system for core networks where robustness to environmental changes, namely traffic shifts, topology changes, and community of interest is viewed as critical. A framework to design robust control strategies for autonomic networks is proposed. The requirements of the network are translated to graph-theoretic metrics and the management system attempts to automatically evolve to a stable and robust control point by optimizing these metrics. The management approach is inspired by ideas from evolutionary science where a metric, network criticality, measures the survival value or robustness of a particular network configuration. In our system, network criticality is a measure of the robustness of the network to environmental changes. The control system is designed to direct the evolution of the system state in the direction of increasing robustness. As an application of our framework, we propose a traffic engineering method in which different paths are ranked based on their robustness measure, and the best path is selected to route the flow. The choice of the path is in the direction of preserving the robustness of the network to the unforeseen changes in topology and traffic demands. Furthermore, we develop a method for capacity assignment to optimize the robustness of the network.

This paper reports on a probabilistic method for traffic engineering (specifically routing and resource allocation) in backbone networks, where the transport is the main service and robustness to the unexpected changes in network parameters is required. We analyze the network using the probabilistic betweenness of the network nodes (or links). The theoretical results lead to the definition of "criticality" for nodes and links. Link criticality is used as the main metric to model the risk of taking a specific path from a source to a destination node. Different paths will be ranked based on their criticality measure, and the best path will be selected to route the flow along the core network. The choice of the path is in the direction of preserving the robustness of the network to the unforeseen changes in topology and traffic demands. The proposed method is useful in situations like MPLS and Ethernet networks where path assignment is required.

This paper reports on a promising approach for solving problems found when multi protocol label switching (MPLS), soon to be a dominant protocol, is used in core network systems. Difficulty is found largely in LSP routing and traffic engineering approaches. While there are a number of online and offline proposals to establish the LSPs but no one is a complete solution considering all the aspects of routing plan from traffic engineering point of view. Our research takes a viewpoint inspired by the concept of "between-ness" from graph theory, from which we introduce notions of link and path criticality indexes. The basis of the work is finding the most critical paths which are mathematically defined based on the algebra of routing. We try to avoid running aggregated flows or commodities on the most critical paths for the short term, and plan increasing the bandwidth of the critical paths for future if possible. This approach shows promise in simulations have run on benchmark networks available from research literature.

Peer-to-Peer (P2P) systems have witnessed an explosive growth in popularity due to their desirable characteristics (robustness, scalability, availability). In this paper, we present an approach to bring these characteristics into the control plane of IP networks, which mainly relies on signaling protocols such as SIP to setup multimedia and instant messaging sessions. We present a structured P2P control plane based on modifications to the original Chord P2P topology, resulting in a hierarchical overlay of SIP peers that replaces traditional client-server paradigms in control plane signaling protocols. Implementations were used to study the performance of the proposed structured P2P control plane, and its suitability for use in IP networks.

Data and telephone service providers have started considering migration to an IP based environment. This paper presents an unstructured peer-to-peer approach to initiating and maintaining sessions over IP using session initiation protocol (SIP). The offered solution is completely modular and is compatible with traditional server-client based approaches using SIP. The peer-to-peer nature of our design allows it to be highly scalable and self managing with low maintenance costs, making it an attractive solution for service providers

Traditional telecommunications service providers are undergoing a transition to a shared infrastructure in which multiple services will be delivered by peer and server computers interconnected by IP networks. IP transport networks that can transfer packets according to differentiated levels of QoS, availability and price are a key element to generating revenue through a rich offering of services. Automated service and network management are essential to creating and maintaining a flexible and agile service delivery infrastructure that also has much lower operations expense than existing systems. In this paper we focus on the SLA-based IP packet transport service on a core network infrastructure and we argue that the above requirements can be met by a self-management system based on autonomic computing and virtual network concepts. We present a control and management system based on this approach.