Test case prioritization (TCP) is essential for improving testing efficiency in large-scale continuous integration (CI) environments by reducing feedback time and efficient resource usage. Machine learning (ML) has shown promise in enhancing TCP, however, demonstrating its effectiveness in production environments remains a challenge. Using the IBM Open Liberty dataset, we developed and validated an ML-based TCP framework, showing how we identified the best-performing model step by step-from feature extraction and model training to hyperparameter tuning. After validating the framework in a research setting, we deployed it in IBM's live production system. The practical implications of this study are as follows. The production results closely mirrored the research outcomes, with models trained on recent data consistently outperforming older models and non-prioritized approaches. Specifically, prioritized builds achieved a mean Average Percentage of Faults Detected (APFD) value 50% higher than that of non-prioritized builds, leading to a substantial improvement in early fault detection. The consistent improvement of models trained on newer data (M-2023) over those trained on older data (M-2022) underscores the importance of regular model updates in maintaining optimal performance. This paper comprehensively compares research and production data, illustrating how our ML-driven TCP framework ensures optimal performance and detailing the steps necessary for successful implementation in dynamic CI environments.

Locks play a crucial role in multi-threaded applications, offering an effective solution for synchronizing shared resources. Yet, mishandling locks and threads can result in contention, leading to performance deterioration and compromising the scalability of software applications. In this study, several machine learning models were evaluated on how well they could detect the Java lock contention anti-pattern that caused the lock contention fault based on run time performance data. We trained the machine learning models with performance data generated from the execution of eight Java lock contention anti-patterns and tested the prediction of the models against 30% of the training data as well as performance data from six applications in the Dacappo benchmark that exhibit lock contention. Our results show that we can accurately identify the lock contention anti-pattern based on runtime performance data with an accuracy close to 90%.

Continuous integration pipelines execute extensive automated test suites to validate new software builds. In this fast-paced development environment, delivering timely testing results to developers is critical to ensuring software quality. Test case prioritization (TCP) emerges as a pivotal solution, enabling the prioritization of fault-prone test cases for immediate attention. Recent advancements in machine learning have showcased promising results in TCP, offering the potential to revolutionize how we optimize testing workflows. Hyperparameter tuning plays a crucial role in enhancing the performance of ML models. However, there needs to be more work investigating the effects of hyperparameter tuning on TCP. Therefore, we explore how optimized hyperparameters influence the performance of various ML classifiers, focusing on the Average Percentage of Faults Detected (APFD) metric. Through empirical analysis of ten real-world, large-scale, diverse datasets, we conduct a grid search-based tuning with 885 hyperparameter combinations for four machine learning models. Our results provide model-specific insights and demonstrate an average 15% improvement in model performance with hyperparameter tuning compared to default settings. We further explain how hyperparameter tuning improves precision (max = 1), recall (max = 0.9633), F1-score (max = 0.9662), and influences APFD value (max = 0.9835), indicating a direct connection between tuning and prioritization performance. Hence, this study underscores the importance of hyperparameter tuning in optimizing failure prediction models and their direct impact on prioritization performance.

The continuous Integration (CI) process runs a large set of automated test cases to verify software builds. The testing phase in the CI systems has timing constraints to ensure software quality without significantly delaying the CI builds. Therefore, CI requires efficient testing techniques such as Test Case Prioritization (TCP) to run faulty test cases with priority. Recent research studies on TCP utilize different Machine Learning (ML) methods to adopt the dynamic and complex nature of CI. However, the performance of ML for TCP may decrease for a low volume of data and less failure rate, whereas using existing data with similar patterns from other domains can be valuable. We formulate this as a transfer learning (TL) problem. TL has proven to be beneficial for many real-world applications where source domains have plenty of data, but the target domains have a scarcity of it. Therefore, this research investigates leveraging the benefit of transfer learning for test case prioritization (TCP). However, only some industrial CI datasets are publicly available due to data privacy protection regulations. In such cases, model-based transfer learning is a potential solution to share knowledge among different projects without revealing data to other stakeholders. This paper applies TransBoost, a tree-kernel-based TL algorithm, to evaluate the TL approach for 24 study subjects and identify potential source datasets.

Locks are essential in multi-threaded applications as they provide a solution to synchronization of shared resources. However, improper management of locks and threads can lead to contention and surface as run-time performance degradation in the application. Nowadays, performance engineers use legacy tools and their experience to determine causes of lock contention but it takes significant expertise to use these tools. In this paper, a data clustering approach is presented to help identify lock contention faults. The classifier is trained leveraging run-time performance data acquired from a catalog of lock contention Java anti-patterns and code smells. The K-means unsupervised classifier algorithm was used to create the classification model and the results show that lock contentions can be classified into three clusters that can be identified into those caused by a) threads spending too much time inside the critical section, b) threads blocked because of high frequency access requests, and c) threads with a low contention. This classifier is intended to be used to help tailor recommendations to the developer based on the lock contention anti-patterns and type of lock contention.

While Application Programming Interfaces (APIs) allow easier reuse of existing functionality, developers might make mistakes in using these APIs (a.k.a. API misuses). If an API usage specification exists, then automatically detecting such misuses becomes feasible. Since manually encoding specifications is a tedious process, there has been a lot of research regarding pattern-based specification mining. However, while annotations are widely used in Java enterprise microservices frameworks, most of these pattern-based rule discovery techniques have not considered annotation-based API usage rules. In this industrial case study of MicroProfile, an open-source Java microservices framework developed by IBM and others, we investigate whether the idea of pattern-based discovery of rules can be applied to annotation-based API usages. We find that our pattern-based approach mines 23 candidate rules, among which 4 are fully valid specifications and 8 are partially valid specifications. Overall, our technique mines 12 valid rules, 10 of which are not even documented in the official MicroProfile documentation. To evaluate the usefulness of the mined rules, we scan MicroProfile client projects for violations. We find 100 violations of 5 rules in 16 projects. Our results suggest that the mined rules can be useful in detecting and preventing annotation-based API misuses.

This paper discusses the issue with scaling SIP/RTP services in a container orchestration cluster such as Kubernetes. In order to run SIP/RTP services in the Kubernetes platform, we require that state and affinity between the two ends, be maintained by the container orchestration cluster. Currently Kubernetes primarily supports stateless services like HTTP. The paper explains the challenge of the Kubernetes overlay network for SIP/RTP services by presenting four failure scenarios with the objective to validate the failure scenarios and present viable solutions. Two of the four failure scenarios have been validated and our approach to counter these failures are presented. One proposed solution is based on a SIP back-to-back user agent or proxy integrated with the Kubernetes environment and a second that leverages the Kubernetes Ingress architecture and services. A third probable solution discussed are the service meshes.