Runtime misconfiguration can lead to software performance degradation and even cause failure. It is usually caused by invalid parameter values set by users. Developers typically perform sanity checks during the configuration parsing stage to prevent invalid parameter values. However, we discovered that even valid values that pass these checks can also lead to unexpected severe consequences. Our study reveals the underlying reason: the value of runtime configuration parameters may interact with other constants and variables when propagated and used, altering its original effect on software behavior. Consequently, parameter values may no longer be valid when encountering complex runtime environments and workloads. Therefore, it is extremely challenging for users to properly configure the software before it starts running. This paper presents the first comprehensive and in-depth study (to the best of our knowledge) on how configuration affects software at runtime through the interaction with constants, and variables (PCV Interaction). Parameter values represent user intentions, constants embody developer knowledge, and variables are typically defined by the runtime environment and workload. This interaction essentially illustrates how different roles jointly determine software behavior. In this regard, we studied 705 configuration parameters from 10 large-scale Software systems. We reveal that a large portion of configuration parameters interact with constants/variables after parsing. We analyzed the interaction patterns and their effects on software runtime behavior. Furthermore, we highlighted the risks of PCV interaction and identified potential issues behind specific interaction patterns. Our findings expose the “double edge” of PCV interaction, providing new insights and motivating the development of new automated techniques to help users configure software appropriately and assist developers in designing better configurations.

Among the misconfigurations with increasing preva-lence and severity in recent years, workload-related misconfigu-rations, i.e. misconfigurations under certain workloads with valid configuration values, account for a significant portion. Since the runtime constraints of configuration parameters are influenced by workloads, piror researches could not handle workload-related misconfigurations at present. To solve the situation mentioned above, we conducted an empirical study on how configuration variables interact with other program variables, and summarized five handling type of the interactions happen in branch statements. Based on the study, we proposed WMWatcher to help system admins to prevent workload-related misconfigurations in production environment. WMWatcher infers the runtime constraints of configuration parameters under certain workload by instrumenting probes in source code and monitoring the corresponding status. The experiments on seven open-source software systems proved that WMWatcher could automatically instrument proper probes while bringing only 2.33% extra runtime overhead at most. And the case study demonstrates the effectiveness of WMWatcher in preventing workload-related misconfigurations in real-world scenarios.

Misconfigurations have become the dominant causes of software failures in recent years, drawing tremendous attention for their increasing prevalence and severity. Configuration constraints can preemptively avoid misconfiguration by defining the conditions that configuration options should satisfy. Documentation is the main source of configuration constraints, but it might be incomplete or inconsistent with the source code. In this regard, prior researches have focused on obtaining configuration constraints from software source code through static analysis. However, the difficulty in pointer analysis and context comprehension prevents them from collecting accurate and comprehensive constraints. In this paper, we observed that software logs often contain configuration constraints. We conducted an empirical study and summarized patterns of configuration-related log messages. Guided by the study, we designed and implemented ConfInLog, a static tool to infer configuration constraints from log messages. ConfInLog first selects configuration-related log messages from source code by using the summarized patterns, then infers constraints from log messages based on the summarized natural language patterns. To evaluate the effectiveness of ConfInLog, we applied our tool on seven popular open-source software systems. ConfInLog successfully inferred 22~163 constraints, in which 59.5%~ 61.6% could not be inferred by the state-of-the-art work. Finally, we submitted 67 documentation patches regarding the constraints inferred by ConfInLog. The constraints in 29 patches have been confirmed by the developers, among which 10 patches have been accepted.

As the scale and complexity of software expands, the issue of software performance is attracting increasing attention. The causes of performance problems mainly fall into two categories: software bugs and the resource contention among multiple software programs. Software bugs are usually caused by inefficient or unnecessary computation in source code. However, the performance problems caused by resource contention among multiple software programs are usually ignored by most researchers. Unlike software bugs, resource contention is not a bug; as a result, it is difficult to identify the concrete reason for a performance problem given that they share the same symptoms, such as long response time or low system throughput. In this paper, we investigate the performance problems caused by resource contention from a configuration perspective. By studying the response time distribution of software as the workload changes, we find that there is an inflection point of response time with the change of workload. Based on our observations, we design and implement a tool, Relax, to automatically detect and resolve resource contention. Relax combines resource request delay at the inflection point and the system resource usage rate to identify the performance problems caused by resource contention. Moreover, inspired by the congestion control algorithm in computer networks, Relax uses the square-increase and multiplicative-decrease method to adjust the resource-related configurations so as to resolve the contention. Our experiments show that Relax can effectively detect and resolve resource contention, and shorten the total software response time by 15.8% ~ 22.8%.

In recent years, misconfigurations have become one of the major causes of software system failures, resulting in numerous service outages. What is worse, misconfigurations are also costly to diagnose and troubleshoot. This remains a great challenge for sysadmins (system administrators) to detect, diagnose, or troubleshoot these misconfigurations. Unlike software bugs, misconfigurations are more vulnerable to sysadmins' mistakes. Developers and researchers are attempting to improve system reactions to misconfigurations to ease the burden of sysadmins' diagnoses. Such efforts would greatly benefit from the techniques that can comprehensively detect bad system reactions through injected misconfigurations. Unfortunately, few such studies have achieved the above goal in the past, primarily because they only relied on generic alterations and failed to find a way to systematically generate misconfigurations. In this paper, we study eight mature open-source and commercial software packages and summarize a fine-grained classification of option types. Based on this classification, we use Augmented Backus-Naur Form to summarize and extract syntactic and semantic constraints of each type. In order to generate comprehensive misconfigurations in the test systems, we propose misconfiguration generation methods for our constraints. We implement a tool named Configuration Vulnerability Detector (ConfVD) to conduct misconfiguration injection and further analyze the systems' reaction abilities to various misconfigurations. We carried out comprehensive analyses upon Apache Httpd, MySQL, PostgreSQL, and Yum. The results of our analysis show that our option classification covers 96% of 1582 options from the above-mentioned systems. Our constraints are more fine grained than previous works and their accuracy was found to be 91% (ascertained by manual verification). Our technique could improve generic alteration approaches without constraints, and we found that ConfVD could find nearly three times the bad reactions that were found by ConfErr. In total, we found 65 bad reactions from the systems being tested and our fine-grained constraints contributed 27.7% more bad reactions than techniques only using coarse-grained constraints.

Misconfigurations have become one of the major causes of software failures because of their increasing prevalence and severity. The complexity of configurations and users' lack of domain knowledge are the main reasons for massive misconfigurations. Users usually identify and diagnose misconfigurations by making a comparison against the conditions that configuration options should satisfy, which we refer to as configuration constraints; however, sometimes it is hard for users to accomplish this work. Some work has been done on obtaining configuration constraints, especially from source code; nevertheless, only part of the situation has been considered, such as if-statement code snippets, limiting its help in misconfiguration diagnosis. In order to better extract configuration constraints for users' guidance and misconfiguration diagnosis, we carried out a comprehensive manual study on the existence and variance of the configuration constraints in the source code of five different pieces of widely used open-source software. Three categories of findings are summarized based on our study, namely the general statistics, the general features of specific kinds of constraints, and the obstacles to the automatic extraction of configuration constraints. With these findings, we proposed several suggestions to maximize the automatic extraction of configuration constraints. The results show that our suggestions could improve the extraction of configuration constraints compared to existing methods.

Misconfigurations are not only prevalent, but also costly on diagnosing and troubleshooting. Unlike software bugs, misconfigurations are more vulnerable to users' mistakes. Improving system reaction to misconfigurations would ease the burden of users' diagnoses. Such effort can greatly benefit from a comprehensive study of system reaction ability towards misconfigurations based on errors injection method. Unfortunately, few such studies have achieved the above goal in the past, primarily because they fail to provide rich error types or only rely on generic alternations to generate misconfigurations. In this paper, we studied 8 mature opensource and commercial software and summarized a fine-grained classification of option types. On the basis of this classification, we could extract syntactic and semantic constraints of each type to generate misconfigurations. We implemented a tool named ConfTest to conduct misconfiguration injection and further analyze system reaction abilities to various of misconfigurations. We carried out comprehensive analyses upon 4 open-source software systems. Our evaluation results show that our option classification covers over 96% of 1582 options from Httpd, Yum, PostgreSQL and MySQL.Our constraint is more fined-grained and the accuracy is more than 90% of of real constraints through manual verification. We compared the capability in finding bad system reactions between ConfTest and ConfErr, showing that the ConfTest can find nearly 3 times the bad reactions found by ConfErr.

Misconfigurations have drawn tremendous attention for their increasing prevalence and severity, and the main causes are the complexity of configurations as well as the lack of domain knowledge for software. To diagnose misconfigurations, one typical approach is to find out the conditions that configuration options should satisfy, which we refer to as configuration constraints. Current researches only handled part of the situations of configuration constraints in source code, which provide only limited help for misconfiguration diagnosis. To better extract configuration constraints, we conduct a comprehensive manual study on the existence and variance of the configuration constraints in source code from five pieces of popular open-source software. We summarized several findings from different aspects, including the general statistics about configuration constraints, the general features for specific configurations, and the obstacles in extraction of configuration constraints. Based on the findings, we propose several suggestions to maximize the automation of constraints extraction.