Autonomous Driving Systems (ADSs) rely on Deep Neural Networks, allowing vehicles to navigate complex, open environments. However, the unpredictability of these scenarios highlights the need for rigorous system-level testing to ensure safety, a task usually performed with a simulator in the loop. Though one important goal of such testing is to detect safety violations, there are many undesirable system behaviors, that may not immediately lead to violations, that testing should also be focusing on, thus detecting more subtle problems and enabling a finer-grained analysis. This paper introduces Cooperative Co-evolutionary MEtamorphic test Generator for Autonomous systems (CoCoMEGA), a novel automated testing framework aimed at advancing system-level safety assessments of ADSs. CoCoMEGA combines Metamorphic Testing (MT) with a searchbased approach utilizing Cooperative Co-Evolutionary Algorithms (CCEA) to efficiently generate a diverse set of test cases. CoCoMEGA emphasizes the identification of test scenarios that present undesirable system behavior, that may eventually lead to safety violations, captured by Metamorphic Relations (MRs). When evaluated within the CARLA simulation environment on the Interfuser ADS, CoCoMEGA consistently outperforms baseline methods, demonstrating enhanced effectiveness and efficiency in generating severe, diverse MR violations and achieving broader exploration of the test space. Further expert assessments of these violations confirmed that most represent real safety risks, which validates their practical relevance. These results underscore CoCoMEGA as a promising, more scalable solution to the inherent challenges in ADS testing with a simulator in the loop. Future research directions may include extending the approach to additional simulation platforms, applying it to other complex systems, and exploring methods for further improving testing efficiency such as surrogate modeling.

Logs record the behavioral data of running programs and are typically generated by executing log statements. Software developers generally carry out logging practices with clear intentions and associated concerns (I&Cs). However, I&Cs may not be properly fulfilled in source code as log placement - specifically determination of a log statement's context and content - is often susceptible to an individual's profile and experience. Some industrial studies have been conducted to discern developers' main logging I&Cs and the way I&Cs are fulfilled. However, the findings are only based on the developers from a single company in each individual study and hence have limited generalizability. More importantly, there lacks a comprehensive and deep understanding of the relationships between developers' profiles and experiences and their logging practices from a wider perspective. To fill this significant gap, we conducted an empirical study using mixed methods comprising questionnaire surveys, semi-structured interviews, and code analyses with practitioners from a wide range of companies across a variety of industrial domains. Results reveal that while developers share common logging I&Cs and conduct logging practices mainly in the coding stage, their profiles and experiences profoundly influence their logging I&Cs and the way the I&Cs are fulfilled. These findings pave the way to facilitate the acceptance of important logging I&Cs and the adoption of good logging practices by developers

Continuity and steadiness are vital for services with massive users, which requires the anomalies of services should be detected and resolved in a timely manner. Our previous work proposed a tool, namelyImpAPTr (Impact Analysis based on Pruning Tree), to identify the combination of multiple dimensional attributes as the clues leading to the root cause of service anomalies. However,ImpAPTrapplies a threshold driven strategy, i.e., it needs to be triggered by a$\geq 0.05\%$drop of the success rate of the service calls (abbr.SRSC), which may face problems in an atypical yet pervasive situation in field application. For example, the combination of trivial anomalies (i.e., each causes a drop less than 0.05% toSRSC) can lead to a far more than 0.05% drop onSRSC. Besides, a suitable threshold is usually hard to be determined, etc. To address these problems, we propose a new method, namelyImpAPTr+in this paper to free the constraint of the 0.05% threshold. The basic idea is to involve time dimension and identify clues across multiple time intervals of data. We performed evaluation on three typical methods (i.e.,ImpAPTr+,R-AdtributorandSqueeze) with both production environment dataset and simulation dataset. The former dataset is directly retrieved from the service monitoring data inMeituan, one of the largest on-line service providers worldwide. The latter dataset is fabricated also using the monitoring data from the same company. The results indicate: (1)ImpAPTr+outperforms previous approaches to a large degree in terms of accuracy. (2) BothImpAPTr+andR-Adtributorare able to find proper clues within seconds. (3)ImpAPTr+tends to find proper clues with shorter time intervals (i.e., less data), which implies that the method is more suitable for near real-time monitoring scenarios.

The microservice architecture has been commonly adopted by large scale software systems exemplified by a wide range of online services. Service monitoring through anomaly detection and root cause analysis (RCA) is crucial for these microservice systems to provide stable and continued services. However, compared with monolithic systems, software systems based on the layered microservice architecture are inherently complex and commonly involve entities at different levels of granularity. Therefore, for effective service monitoring, these systems have a special requirement of multi-granular RCA. Furthermore, as a large proportion of anomalies in microservice systems pertain to problematic code, to timely troubleshoot these anomalies, these systems have another special requirement of RCA at the finest code-level. Microservice systems rely on telemetry data to perform service monitoring and RCA of service anomalies. The majority of existing RCA approaches are only based on a single type of telemetry data and as a result can only support uni-granular RCA at either application-level or service-level. Although there are attempts to combine metric and tracing data in RCA, their objective is to improve RCA's efficiency or accuracy rather than to support multi-granular RCA. In this article, we propose a new RCA solutionTrinityRCLthat is able to localize the root causes of anomalies at multiple levels of granularity including application-level, service-level, host-level, and metric-level, with the unique capability of code-level localization by harnessing all three types of telemetry data to construct a causal graph representing the intricate, dynamic, and nondeterministic relationships among the various entities related to the anomalies. By implementing and deployingTrinityRCLin a real production environment, we evaluateTrinityRCLagainst two baseline methods and the results show thatTrinityRCLhas a significant performance advantage in terms of accuracy at the same level of granularity with comparable efficiency and is particularly effective to support large-scale systems with massive telemetry data.

Background:Logging practices provide the ability to record valuable runtime information of software systems to support operations tasks such as service monitoring and troubleshooting. However, current logging practices face common challenges. On the one hand, although the importance of logging practices has been broadly recognized, most of them are still conducted in an arbitrary or ad-hoc manner, ending up with questionable or inadequate support to perform these tasks. On the other hand, considerable research effort has been carried out on logging practices, however, few of the proposed techniques or methods have been widely adopted in industry.Objective:This study aims to establish a comprehensive understanding of the research state of logging practices, with a focus on unveiling possible problems and gaps which further shed light on the potential future research directions.Method:We carried out a systematic mapping study on logging practices with 56 primary studies.Results:This study provides a holistic report of the existing research on logging practices by systematically synthesizing and analyzing the focus and inter-relationship of the existing research in terms of issues, research topics and solution approaches. Using3W1H—Why to log,Where to log,What to logandHow well is the logging—as the categorization standard, we find that: (1) the best known issues in logging practices have been repeatedly investigated; (2) the issues are often studied separately without considering their intricate relationships; (3) theWhere and Whatquestions have attracted the majority of research attention while little research effort has been made on theWhyandHow wellquestions; and (4) the relationships between issues, research topics, and approaches regarding logging practices appear many-to-many, which indicates a lack of profound understanding of the issues in practice and how they should be appropriately tackled.Conclusions:This study indicates a need to advance the state of research on logging practices. For example, more research effort should be invested onwhy to logto set the anchor of logging practices as well as onhow well is the loggingto close the loop. In addition, a holistic process perspective should be taken into account in both the research and the adoption related to logging practices.

Background: Logs provide crucial information to understand the dynamic behavior of software systems in modern software development and maintenance. Usually, logs are produced by log statements which will be triggered and executed under certain conditions. However, current studies paid very limited attention to developers' Intentions and Concerns (I&C) on logging practice, leading uncertainty that whether the developers' I&C are properly reflected by log statements and questionable capability to capture the expected information of system behaviors in logs. Objective: This study aims to reveal the status of developers' I&C on logging practice and more importantly, how the I&C are properly reflected in software source code in real-world software development. Method: We collected evidence from two sources of a series of interviews and source code analysis which are conducted in a big-data company, followed by consolidation and analysis of the evidence. Results: Major gaps and inconsistencies have been identified between the developers' I&C and real log statements in source code. Many code snippets contained no log statements that the interviewees claimed to have inserted. Conclusion: Developers' original I&C towards logging practice are usually poorly realized, which inevitably impacted the motivation and purpose to conduct this practice.

Logs are the execution results of logging statements in software systems after being triggered by various events, which is able to capture the dynamic behavior of software systems during runtime and provide important information for software analysis, e.g., issue tracking, performance monitoring, etc. Obviously, to meet this purpose, the quality of the logs is critical, which requires appropriately placement of logging statements. Existing research on this topic reveals that where to log? and what to log? are two most concerns when conducting logging practice in software development, which mainly relies on developers' personal skills, expertise and preference, rendering several problems impacting the quality of the logs inevitably. One of the reasons leading to this phenomenon might be that several recognized best practices(strategies as well) are easily neglected by software developers. Especially in those software projects with relatively large number of participants. To address this issue, we designed and implemented a plug-in tool (i.e., JLLAR) based on the Intellij IDEA, which applied machine learning technology to identify and create a set of rules reflecting commonly recognized logging practices. Based on this rule set, JLLAR can be used to scan existing source code to identify issues regarding the placement of logging statements. Moreover, JLLAR also provides automatic code completion and semi code completion (i.e., to provide recommendations) regarding logging practice to support software developers during coding.

Knowledge management nowadays usually focuses on the choice among some models or methodologies as a whole, but not on some specific, quantitative contributions of particular goals of the organization. Such a simplification misses some important chances for knowledge integration and transformation. What's worse, this simplification depresses the motivation of team members to accumulate and use the knowledge. In this paper, we propose a knowledge management framework which features in its goal-driven philosophy to manage project development, organize the knowledge and effectively integrate the knowledge management process into the development process. This method helps software project teams comprehensively and systematically identify and track knowledge management goals as far as possible. With a common framework, an organization is able to exchange knowledge and expertise within itself, which helps to glue the company together; while at the same time ensures that knowledge is shared over time so that the company benefits from past experience. Team members come to a common understanding on how to accumulate knowledge by establishing goals and corresponding solutions to meet the goals, and this consensus and clear vision on knowledge management motivates members to create knowledge and reduce the "gulf" between knowledge creation and application. It was successfully applied in several projects of different companies. The framework helps them establish an initial knowledge and experience repository. Software engineers are able to have more information available than they could understand and apply.

Background: Logging practice is a critical activity in software development, which aims to offer significant information to understand the runtime behavior of software systems and support better software maintenance. There have been many relevant studies dedicated to logging practice in software engineering recently, yet it lacks a systematic understanding to the adoption state of logging practice in industry and research progress in academia. Objective: This study aims to synthesize relevant studies on the logging practice and portray a big picture of logging practice in software engineering so as to understand current adoption status and identify research opportunities. Method: We carried out a systematic review on the relevant studies on logging practice in software engineering. Results: Our study identified 41 primary studies relevant to logging practice. Typical findings are: (1) Logging practice attracts broad interests among researchers in many concrete research areas. (2) Logging practice occurred in many development types, among which the development of fault tolerance systems is the most adopted type. (3) Many challenges exist in current logging practice in software engineering, e.g., tradeoff between logging overhead and analysis cost, where and what to log, balance between enough logging and system performance, etc. Conclusion: Results show that logging practice plays a vital role in various applications for diverse purposes. However, there are many challenges and problems to be solved. Therefore, various novel techniques are necessary to guide developers conducting logging practice and improve the performance and efficiency of logging practice.

As an emerging approach to support fast delivery of software features with reliable quality, DevOps attracts more and more practitioners and shows the potential to become one of the mainstream approach for software development and operation. Many universities begin to offer DevOps related courses to the students majored in software engineering and computer science. However, as a critical part of a DevOps course, the project practicing using DevOps might cast big challenges for teachers, compared to traditional project practicing. For example, the more frequent than ever delivery in DevOps practicing will inevitably increase the workload vastly for teachers to conduct effective evaluation. In this paper, we introduce a web based system (DevOpsEnvy) to support the management and monitoring of student teams practicing DevOps. By integrating several popular open source tools, this system provides students with features such as group management, project status monitoring and student performance data analysis, etc. Meanwhile, DevOpsEnvy system also provides teachers with sufficient evidence to perform evaluation. Our preliminary trial in Nanjing University revealed several advantages of DevOpsEnvy system.