Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently lead to hidden degradation, known as “gray failure”, for AI workloads, significantly affecting end-to-end performance and concealing performance issues, which complicates root cause analysis for failures and regressions. We introduce SuperBench, a proactive validation system for AI infrastructure that mitigates hidden degradation caused by hardware redundancies and enhances overall reliability. SuperBench features a comprehensive benchmark suite, capable of evaluating individual hardware components and representing most real AI workloads. It comprises a Validator that learns benchmark criteria to pinpoint defective components clearly. Additionally, SuperBench incorporates a Selector to balance validation time and issue-related penalties, enabling optimal timing for validation execution with a tailored subset of benchmarks. Through testbed evaluation and simulation, we demonstrate that SuperBench can increase the mean time between incidents by up to 22.61×. SuperBench has been successfully deployed in Azure production, validating hundreds of thousands of GPUs every year.

Background: Practitioners would like to take action based on software metrics, as long as they find them reliable. Existing literature explores how metrics can be made reliable, but remains unclear if there are other conditions necessary for a metric to be actionable. Context & Method: In the context of a European H2020 Project, we conducted a multiple case study to study metrics' use in four companies, and identified instances where these metrics influenced actions. We used an online questionnaire to enquire about the project participants' views on actionable metrics. Next, we invited one participant from each company to elaborate on the identified metrics' use for taking actions and the questionnaire responses (N=17). Result: We learned that a metric that is practical, contextual, and exhibits high data quality characteristics is actionable. Even a non-actionable metric can be useful, but an actionable metric mostly requires interpretation. However, the more these metrics are simple and reflect the software development context accurately, the less interpretation required to infer actionable information from the metric. Company size and project characteristics can also influence the type of metric that can be actionable. Conclusion: This exploration of industry's views on actionable metrics help characterize actionable metrics in practical terms. This awareness of what characteristics constitute an actionable metric can facilitate their definition and development right from the start of a software metrics program.

Existing literature proposes success factors for establishing metrics programs. However, very few studies focus on factors that could ensure long-term use of metrics, and even fewer studies investigate such factors in the context of Agile Software Development (ASD). Motivated by this knowledge gap, we aim to identify success factors for operationalizing metrics in ASD, particularly, factors that could help in the long-term use of metrics. We conducted a multiple case study, where we operationalized process metrics at two software-intensive companies using ASD. We learned that data availability and development process are the two fundamental success factors for process metrics operationalization, albeit less prominent in literature. Companies prefer iterative and incremental operationalization of stable and functional process metrics, which is analogous to the agile way of working. Metrics trustworthiness plays a key role in successful operationalization of process metrics, and is potentially vital to ensuring their long-term use. By comparing the identified success factors with the existing literature, we conclude that success factors concerning data availability, development process, and metrics trustworthiness warrant greater attention, especially to maximize the chances of long-term use of process metrics.

Significant strides are being made in research towards security and privacy concerns associated with connected cars. Being a relatively novel research domain, it is crucial to examine the state of the art to formulate future research strategies. The objective of the paper is to aggregate the research progress made in identifying security and privacy concerns in connected cars, and the concomitant solutions. Using systematic mapping study, we reviewed the state of the art to address the research objective. Results indicate that in the last couple of years, there has been an exponential increase in investigation into this research topic. Compromised vehicular communication has received more attention than any other concern, and the popularity has only increased in recent years. A secure system design and infrastructure is the most common solution for all the identified concerns in this paper, but securing communication is the most popular one. Real-world applicability of the devised solutions is debatable, as most researchers have preferred simulations to empirical evaluation. Overall, insecure system design or infrastructure warrants more attention from solution point of view. In this context, the principles of Prevention by Design and Privacy by Design can be among the most effective strategies.

Software engineering, especially design and requirements engineering, is intensely creative. However, practitioners and researchers appear to perceive creativity differently, hindering knowledge transfer. To explore and understand these perceptual differences, this paper combines a systematic mapping study of SE research literature with an interview study of practitioners. The subsequent analysis of 84 primary studies and 17 semi-structured interviews reveal some agreement (e.g. creativity is a process that produces novel and useful ideas). However, it also reveals important differences in the way creativity is conceptualized, measured and improved. These differences undermine evidence-based techniques to enhance and measure creativity in SE research and practice.