Large Language Models (LLM) empower many modern software systems, and are required to be highly accurate and reliable. Evaluating LLM poses challenges due to the high costs of manual labeling and of validation of labeled data.This study investigates the suitability of probabilistic operational testing for effective and efficient evaluation of LLM, focusing on a case study with DistilBERT. To this aim, we adopt an existing framework (DeepSample) for Deep Neural Network (DNN) testing and adapt it to the LLM domain by introducing auxiliary variables tailored to LLM and classification tasks.Through a comprehensive evaluation, we demonstrate how sampling-based operational testing can yield reliable LLM accuracy estimates and effectively expose failures, or, under testing budget constraints, it can find a trade off between accuracy estimation and failure exposure. The experimental results, using DistilBERT on three sentiment analysis datasets, show that sampling-based methods can provide cost effective and reliable operational accuracy assessment for LLM. These findings offer practical insights for testers and help address critical gaps in current LLM evaluation practices.

Regression testing enhances software reliability by detecting regressions in new versions. Regression test suites often lack awareness of real-world product/service usage, potentially leading to undetected faults and ineffective testing scenarios. We propose LogTest, a transformer-based approach that learns from event logs and system traces to automatically generate service invocation sequences that mimic observed system behaviors. These sequences enhance regression test suites by exposing past real execution patterns. A preliminary experimentation on a realistic benchmark demonstrates its potential.

Performance testing is an essential activity in the engineering of microservice applications to identify deviations from the specified ranges of relevant metrics and to analyse resources usage. It demands for high automation to fit within the short microservices development-operation cycles. Engineers are often interested in identifying critical workloads - ideally, in the “minimal“ load configurations causing tests to expose performance issues. Triggering performance issues is challenging, requiring proper workload characterization and test design. We present the microWave framework for learning-based automated generation of critical performance testing workloads for microservices. The framework can harness various learning strategies: we analyze a Deep Neural Network, a Large Language Model and a Causal Reasoning strategy. We evaluate them experimentally on four subjects, using a random approach and a manually-crafted ground truth as baselines. The results show that the strategies exhibit different behavior depending on the data they learn from. When inferring from past executions data including performance issues, the causal model performs better. The random predictor is preferable when no data is available; however, it is more costly as it requires more tests. The results allow to draw practical recommendations for testers on how to select the most suitable strategy depending on the needs.

In a recent study, Reinforcement Learning (RL) used in combination with many-objective search, has been shown to outperform alternative techniques (random search and many-objective search) for online testing of Deep Neural Network-enabled systems. The empirical evaluation of these techniques was conducted on a state-of-the-art Autonomous Driving System (ADS). This work is a replication and extension of that empirical study. Our replication shows that RL does not outperform pure random test generation in a comparison conducted under the same settings of the original study, but with no confounding factor coming from the way collisions are measured. Our extension aims at eliminating some of the possible reasons for the poor performance of RL observed in our replication: (1) the presence of reward components providing contrasting feedback to the RL agent; (2) the usage of an RL algorithm (Q-learning) which requires discretization of an intrinsically continuous state space. Results show that our new RL agent is able to converge to an effective policy that outperforms random search. Results also highlight other possible improvements, which open to further investigations on how to best leverage RL for online ADS testing.

Context: Software Quality Assurance (SQA) is a fundamental part of software engineering to ensure stakeholders that software products work as expected after release in operation. Machine Learning (ML) has proven to be able to boost SQA activities and contribute to the development of quality software systems. In this context, Causal Reasoning is gaining increasing interest as a methodology to go beyond a purely data-driven approach by exploiting the use of causality for more effective SQA strategies. Objective: Provide a broad and detailed overview of the use of causal reasoning for SQA activities, in order to support researchers to access this research field, identifying room for application, main challenges and research opportunities. Methods: A systematic review of the scientific literature on causal reasoning for SQA. The study has found, classified, and analyzed 86 articles, according to established guidelines for software engineering secondary studies. Results: Results highlight the primary areas within SQA where causal reasoning has been applied, the predominant methodologies used, and the level of maturity of the proposed solutions. Fault localization is the activity where causal reasoning is more exploited, especially in the web services/microservices domain, but other tasks like testing are rapidly gaining popularity. Both causal inference and causal discovery are exploited, with the Pearl’s graphical formulation of causality being preferred, likely due to its intuitiveness. Tools to favor their application are appearing at a fast pace — most of them after 2021. Conclusions: The findings show that causal reasoning is a valuable means for SQA tasks with respect to multiple quality attributes , especially during V&V, evolution and maintenance to ensure reliability, while it is not yet fully exploited for phases like requirements engineering and design. We give a picture of the current landscape, pointing out exciting possibilities for future research.

In numerous contexts, ranging from systems safety assessment to finance and medical diagnosis, a relevant causal inference task is to predict unseen rare events—the so-called black swans . These are plausible, high-impact, but unexpected events for whose prediction a probabilistic-based causal inference falls short. For instance, a safety analyst needs to hypothesize potential rare co-causes that could lead to an accident, so as to manage the most unexpected failures besides the more obvious ones. Given an effect, we use abduction to support the generation of a plausible set of explanatory hypotheses for its causes. We present a generative evolutionary strategy—called Evolutionary Abduction (EVA)—for automating abductive inference by repeatedly constructing hypothetical cause-effect instances, and then automatically assessing their plausibility as well as their novelty with respect to already known instances—a mechanism mimicking the human reasoning employed whenever we need to select the best candidates from a set of hypotheses. Experiments with four datasets confirm that EVA can construct new and realistic multiple-cause hypotheses for a given effect. EVA outperforms alternative strategies based on probabilistic-based causal inference as well as state-of-the-art evolutionary algorithms, generating closer-to-real instances in most settings and datasets.

Evaluating the performance of Microservices Architectures (MSA) is essential to ensure their proper functioning and meet end-user satisfaction. For MSA performance analysts, one of the most challenging tasks is to determine the cause of any deviation of relevant metrics from the specified range.

Deep Neural Networks (DNN) are core components for classification and regression tasks of many software systems. Companies incur in high costs for testing DNN with datasets representative of the inputs expected in operation, as these need to be manually labelled. The challenge is to select a representative set of test inputs as small as possible to reduce the labelling cost, while sufficing to yield unbiased high-confidence estimates of the expected DNN accuracy. At the same time, testers are interested in exposing as many DNN mispredictions as possible to improve the DNN, ending up in the need for techniques pursuing a threefold aim: small dataset size, trustworthy estimates, mispredictions exposure.

With the expansion of cloud computing and data centers, the need has arisen to tackle their environmental impact. The increasing adoption of microservice architectures, while offering scalability and flexibility, poses new challenges in the effective management of systems’ energy consumption.This study analyzes experimentally the effectiveness, with respect to energy consumption, of algorithms for Anomaly Detection (AD) and Root Cause Analysis (RCA) for (containerized) microservices systems. The study analyzes five AD and three RCA algorithms. Metrics to assess the effectiveness of AD algorithms are Precision, Recall, and F-Score. For RCA algorithms, the chose metric is Precision at level k. Two subjects of different complexity are used: Sock Shop and UNI-Cloud. Experiments use a cross-over paired comparison design, involving multiple randomized runs for robust measures.The experiments show that AD algorithms exhibit a relatively moderate performance. The mean adjusted Precision for Sock Shop is 61.5%, while it is 75% for the best-performing algorithms (BIRCH, KNN, and SVM) on UNI-Cloud. The Recall and F-Score for UNI-Cloud, for the same algorithms, are 75%, while for Sock Shop KNN yields the best outcome at roughly 45%. MicroRCA and RCD emerge as the top-performing algorithms for RCA.We found that the effectiveness of AD algorithms is strongly influenced by anomaly thresholds, emphasizing the importance of careful tuning such algorithms. RCA algorithms reveal promising results, particularly RCD and MicroRCA, which showed robust performance. However, challenges remain, as seen with the ϵ-diagnosis algorithm, suggesting the need for further refinement.For DevOps engineers, the findings highlight the need to carefully select and tune AD and RCA algorithms for energy, and to take into account system topology and monitoring configurations.

Microservice Architectures (MSA) are nowadays largely adopted by companies in several domains to provide on-demand services. The reliability of microservices is fundamental to avoid failures compromising the business functionalities. MSA automated testing is possible thanks to well-defined service interfaces specified in open formats like OpenAPI/Swagger. To support automated MSA functional and non-functional testing, we define a framework that: (i) generates test cases with valid and invalid inputs, and executes and monitors tests; (ii) provides coverage and failure information not only on edge, but also on internal microservices; (iii) has the novel feature of identifying causal relations in observed chains of microservices failures. We abstract the testing process of MSA, present the MacroHive framework and its causal inference engine, compare it experimentally to state-of-the-art tools, and discuss its benefits in the MSA testing process. MacroHive exhibits performance comparable to advanced existing tools in terms of edge-level coverage. However, MacroHive has a better failure rate and provides the unique advantages of giving insights about internal coverage and failures, and of inferring causality in failure chains, evidencing microservices to be improved to increase the whole MSA reliability.

Microservice-based systems are usually developed according to agile practices like DevOps, which enables rapid and frequent releases to promptly react and adapt to changes. Monitoring is a key enabler for these systems, as they allow to continuously get feedback from the field and support timely and tailored decisions for a quality-driven evolution. In the realm of monitoring tools available for microservices in the DevOps-driven development practice, each with different features, assumptions, and performance, selecting a suitable tool is an as much difficult as impactful task. This article presents the results of a systematic study of the grey literature we performed to identify, classify and analyze the available monitoring tools for DevOps and microservices. We selected and examined a list of 71 monitoring tools, drawing a map of their characteristics, limitations, assumptions, and open challenges, meant to be useful to both researchers and practitioners working in this area. Results are publicly available and replicable. Editor's note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Testing Autonomous Driving Systems (ADS) is essential for safe development of self-driving cars. For thorough and realistic testing, ADS are usually embedded in a simulator and tested in interaction with the simulated environment. However, their high complexity and the multiple safety requirements lead to costly and ineffective testing. Recent techniques exploit many-objective strategies and ML to efficiently search the huge input space. Despite the indubitable advances, the need for smartening the search keep being pressing. This article presents CART ( CAusal-Reasoning-driven Testing ), a new technique that formulates testing as a causal reasoning task. Learning causation, unlike correlation, allows assessing the effect of actively changing an input on the output, net of possible confounding variables. CART first infers the causal relations between test inputs and outputs, then looks for promising tests by querying the learnt model. Only tests suggested by the model are run on the simulator. An extensive empirical evaluation, using Pylot as ADS and CARLA as simulator, compares CART with state-of-the-art algorithms used recently on ADS. CART shows a significant gain in exposing more safety violations and does so more efficiently. More broadly, the work opens to a wider exploitation of causal learning beside (or on top of) ML for testing-related tasks.

In the railway domain, rolling stock maintenance affects service operation time and efficiency. Minimizing train unavailability is essential for reducing capital loss and operational costs. To this aim, prediction of failures of rolling stock equipment is crucial to proactively trigger proper maintenance activities. Indeed, predictive maintenance is a golden example of the digital transformation within Industry 4.0, which affects several engineering processes in the railway domain. Nowadays, it may leverage artificial intelligence and machine learning algorithms to forecast failures and schedule the optimal time for maintenance actions. Generally, rail systems deteriorate gradually over time or fail directly, leading to data that vary extremely slowly. Indeed, ML approaches for predictive maintenance should consider this type of data to accurately predict and forecast failures. This paper proposes a methodology based on Long Short-Term Memory deep learning algorithms for predictive maintenance of railway rolling stock equipment. The methodology allows us to properly learn long-term dependencies for gradually changing data, and both predicting and forecasting failures of rail equipment. In the framework of an academic-industrial partnership, the methodology is experimented on a train traction converter cooling system, demonstrating its applicability and benefits. The results show that it outperforms state-of-the-art methods, reaching a failure prediction and forecasting accuracy over 99%, with a false alarm rate of ∼0.4% and a mean absolute error in the order of 10−4, respectively.

Abstract To enter the production stage, in DevOps practices candidate software releases have to pass quality gates, where they are assessed to meet established target values for key indicators of interest. We believe software reliability should be an important such indicator, as it greatly contributes to the end‐user satisfaction. We propose DevOpRET , an approach for reliability testing as part of the acceptance testing stage in DevOps. DevOpRET relies on operational‐profile–based testing, a common reliability assessment technique. DevOpRET leverages usage and failure data monitored in operations to continuously refine its estimate. We evaluate accuracy and efficiency of DevOpRET through controlled experiments with a real‐world open source platform and with a microservice architectures benchmark. The results show that DevOpRET provides accurate and efficient estimates of the true reliability over subsequent DevOps cycles.

Continuous quality assurance for extra-functional properties of modern software systems is today a big challenge as their complexity is constantly increasing to satisfy market demands. This is the case of microservice systems. They provide high control on the scale of operation by means of fine-grained service decomposition, but this demands careful consideration of the relations between performance of individual microservices and service failures.

Microservices Architectures (MSA) have found large adoption in companies delivering online services, often in conjunction with agile development practices. Microservices are distributed, independent and polyglot entities – all features favouring black-box testing. However, for real-scale MSA, a pure black-box strategy may not be able to exercise the system to properly cover the interactions involving internal microservices.We propose a grey-box strategy (MACROHIVE) for automated testing and monitoring of (internal) microservices interactions. It uses combinatorial testing to generate valid and invalid tests from microservices specification. Tests execution and monitoring are automated by a service mesh infrastructure. MACROHIVE runs the tests and traces the interactions among microservices, to report about internal coverage and failing behaviour.MACROHIVE is experimented on TrainTicket, an open-source MSA benchmark. It performs comparably to state-of-the-art techniques in terms of edge-level coverage, but exposes internal failures undetected by black-box testing, gives detailed internal coverage information, and requires fewer tests.

Deep Neural Networks (DNN) are typically tested for accuracy relying on a set of unlabelled real world data (operational dataset), from which a subset is selected, manually labelled and used as test suite. This subset is required to be small (due to manual labelling cost) yet to faithfully represent the operational context, with the resulting test suite containing roughly the same proportion of examples causing misprediction (i.e., failing test cases) as the operational dataset. However, while testing to estimate accuracy, it is desirable to also learn as much as possible from the failing tests in the operational dataset, since they inform about possible bugs of the DNN. A smart sampling strategy may allow to intentionally include in the test suite many examples causing misprediction, thus providing this way more valuable inputs for DNN improvement while preserving the ability to get trustworthy unbiased estimates. This paper presents a test selection technique (DeepEST) that actively looks for failing test cases in the operational dataset of a DNN, with the goal of assessing the DNN expected accuracy by a small and "informative" test suite (namely with a high number of mispredictions) for subsequent DNN improvement. Experiments with five subjects, combining four DNN models and three datasets, are described. The results show that DeepEST provides DNN accuracy estimates with precision close to (and often better than) those of existing sampling-based DNN testing techniques, while detecting from 5 to 30 times more mispredictions, with the same test suite size.

We present an adaptive software testing strategy for test case allocation, selection and generation, based on the combined use of operational profile and coverage spectrum, aimed at achieving high delivered reliability of the program under test. Operational profile-based testing is a black-box technique considered well suited when reliability is a major concern, as it selects the test cases having the largest impact on failure probability in operation. Coverage spectrum is a characterization of a program's behavior in terms of the code entities (e.g., branches, statements, functions) that are covered as the program executes. The proposed strategy - named covrel+ - complements operational profile information with white-box coverage measures, so as to adaptively select/generate the most effective test cases for improving reliability as testing proceeds. We assess covrel+ through experiments with subjects commonly used in software testing research, comparing results with traditional operational testing. The results show that exploiting operational and coverage data in an integrated adaptive way allows generally to outperform operational testing at achieving a given reliability target, or at detecting faults under the same testing budget, and that covrel+ has greater ability than operational testing in detecting hard-to-detect faults.

In Continuous Integration (CI), regression testing is constrained by the time between commits. This demands for careful selection and/or prioritization of test cases within test suites too large to be run entirely. To this aim, some Machine Learning (ML) techniques have been proposed, as an alternative to deterministic approaches. Two broad strategies for ML-based prioritization are learning-to-rank and what we call ranking-to-learn (i.e., reinforcement learning). Various ML algorithms can be applied in each strategy. In this paper we introduce ten of such algorithms for adoption in CI practices, and perform a comprehensive study comparing them against each other using subjects from the Apache Commons project. We analyze the influence of several features of the code under test and of the test process. The results allow to draw criteria to support testers in selecting and tuning the technique that best fits their context.

Summary Microservice architectures (MSA) is an emerging software architectural paradigm for service‐oriented applications, well‐suited for dynamic contexts requiring loosely coupled independent services, frequent software releases and decentralized governance. A key problem in the engineering of MSA applications is the estimate of their reliability, which is difficult to perform prior to release due frequent releases/service upgrades, dynamic service interactions, and changes in the way customers use the applications. This paper presents an in vivo testing method, named EMART, to faithfully assess the reliability of an MSA application in operation. EMART is based on an adaptive sampling strategy, leveraging monitoring data about microservices usage and failure/success of user demands. We present results of evaluation of estimation accuracy, confidence and efficiency, through a set of controlled experiments with publicly available subjects. © 2019 John Wiley & Sons, Ltd.

Likely system invariants model properties that hold in operating conditions of a computing system. Invariants may be mined offline from training datasets, or inferred during execution. Scientific work has shown that invariants' mining techniques support several activities, including capacity planning and detection of failures, anomalies and violations of Service Level Agreements. However their practical application by operation engineers is still a challenge. We aim to fill this gap through an empirical analysis of three major techniques for mining invariants in cloud-based utility computing systems: clustering, association rules, and decision list. The experiments use independent datasets from real-world systems: a Google cluster, whose traces are publicly available, and a Software-as-a-Service platform used by various companies worldwide. We assess the techniques in two invariants' applications, namely executions characterization and anomaly detection, using the metrics of coverage, recall and precision. A sensitivity analysis is performed. Experimental results allow inferring practical usage implications, showing that relatively few invariants characterize the majority of operating conditions, that precision and recall may drop significantly when trying to achieve a large coverage, and that techniques exhibit similar precision, though the supervised one a higher recall. Finally, we propose a general heuristic for selecting likely invariants from a dataset.

One of the biggest emerging challenges for telco operators is to dynamically create new services while maintaining the network at its optimal performance/revenue break point. To this end, operators are moving to a much leaner cloud-based Software Defined Network (SDN) infrastructure to achieve a truly programmable network fabric. While cloud services can be provisioned in seconds and tested in-production, service provisioning in SDNs still lasts many weeks and requires substantial manual effort. A large part of this service creation time can be attributed to testing and tuning the control plane. In this paper we present SCP-CLUB (SDN Control Plane CLoUd-based Benchmarking), a platform for in-production performance testing of telco operator SDNs, offering a level of automation as available in deploying cloud services. Telco cloud SDN performance testing with SCP-CLUB focuses on the analysis of how design choices in the cloud and SDN control planes influence SLA metrics like throughput and latency. We describe the SCP-CLUB architecture and its performance testing support capabilities. Experiments are performed on an SDN telco cloud built to demonstrate SCP-CLUB under production load conditions.

Microservices are gaining popularity as an architectural paradigm for service-oriented applications, especially suited for highly dynamic contexts requiring loosely-coupled independent services, frequent software releases, decentralized governance and data management. Because of the high flexibility and evolvability characterizing microservice architectures (MSAs), it is difficult to estimate their reliability at design time, as it changes continuously due to the services' upgrades and/or to the way applications are used by customers. This paper presents a testing method for on-demand reliability estimation of microservice applications in their operational phase. The method allows to faithfully assess, upon request, the reliability of a MSA-based application under a scarce testing budget, at any time when it is in operation, and exploit field data about microservice usage and failing/successful demands. A new in-vivo testing algorithm is developed based on an adaptive web sampling strategy, named Microservice Adaptive Reliability Testing (MART). The method is evaluated by simulation, as well as by experimentation on an example application based on the Netflix Open Source Software MSA stack, with encouraging results in terms of estimation accuracy and, especially, efficiency.

A relevant objective of software reliability assessment is to get unbiased estimates with an acceptable trade-off between the number of tests required and the variance of the estimate. A low variance is desirable to increase the confidence in the estimate, but too many tests may be required by conventional reliability assessment testing techniques based solely on the operational profile. This article presents probabilistic sampling-based testing, a new technique using unequal probability sampling to exploit auxiliary information about the software under test so as to assess reliability unbiasedly and efficiently. The technique expedites the assessment process assuming the availability of some prior belief about input regions failure proneness. The evaluation by simulation and experimentally shows promising results in terms of estimate accuracy and efficiency.

Testing resource allocation is the problem of planning the assignment of resources to testing activities of software components so as to achieve a target goal under given constraints. Existing methods build on software reliability growth models (SRGMs), aiming at maximizing reliability given time/cost constraints, or at minimizing cost given quality/time constraints. We formulate it as a multiobjective debug-aware and robust optimization problem under uncertainty of data, advancing the state-of-the-art in the following ways. Multiobjective optimization produces a set of solutions, allowing to evaluate alternative tradeoffs among reliability, cost, and release time. Debug awareness relaxes the traditional assumptions of SRGMs-in particular the very unrealistic immediate repair of detected faults-and incorporates the bug assignment activity. Robustness provides solutions valid in spite of a degree of uncertainty on input parameters. We show results with a real-world case study.

We introduce covrel, an adaptive software testing approach based on the combined use of operational profile and coverage spectrum, with the ultimate goal of improving the delivered reliability of the program under test. Operational profile-based testing is a black-box technique that selects test cases having the largest impact on failure probability in operation, as such, it is considered well suited when reliability is a major concern. Program spectrum is a characterization of a program's behavior in terms of the code entities (e.g., branches, statements, functions) that are covered as the program executes. The driving idea of covrel is to complement operational profile information with white-box coverage measures based on count spectra, so as to dynamically select the most effective test cases for reliability improvement. In particular, we bias operational profile-based test selection towards those entities covered less frequently. We assess the approach by experiments with 18 versions from 4 subjects commonly used in software testing research, comparing results with traditional operational and coverage testing. Results show that exploiting operational and coverage data in a combined adaptive way actually pays in terms of reliability improvement, with covrel overcoming conventional operational testing in more than 80% of the cases.

Summary Software reliability growth models support the prediction/assessment of product quality, release time, and testing/debugging cost. Several software reliability growth model extensions take into account the bug correction process. However, their estimates may be significantly inaccurate when debugging fails to fully fit modelling assumptions. This paper proposes debugging‐workflow‐aware software reliability growth method (DWA‐SRGM), a method for reliability growth analysis leveraging the debugging data usually managed by companies in bug tracking systems. On the basis of a characterization of the debugging workflow within the software project under consideration (in terms of bug features and treatment phases), DWA‐SRGM pinpoints the factors impacting the estimates and to spot bottlenecks, thus supporting process improvement decisions. Two industrial case studies are presented, a customer relationship management system and an enterprise resource planning system, whose defects span a period of about 17 and 13 months, respectively. DWA‐SRGM revealed effective to obtain more realistic estimates and to capitalize on the awareness of critical factors for improving debugging.

The papers in this special issue on security and dependability of cloud systems and services. Service-based cloud computing systems are used nowadays in many business- and mission-critical scenarios. As the service-oriented paradigm increasingly spreads in a wide range of application fields, including big data, cloud storage, mobile cloud computing, and sensor cloud, there is a growing need for sound methodologies, algorithms and techniques for building services in which companies, organizations and citizens can trust and rely upon. Security and dependability are therefore becoming more and more relevant concerns for such systems, whose complexity, heterogeneity, and fast-changing dynamics bring difficult challenges to the research and industry communities.

This is the second part of the special issue on security and dependability of cloud systems and services, which was organized to solicit novel results in these important and closely related research areas. Indeed, security and dependability are increasingly important concerns for cloud systems and services, due to their spread in a large variety of application fields, including business- and mission-critical scenarios. This demands for innovative methodologies, algorithms and techniques for building services in which companies, organizations and citizens can trust and rely upon.

Assessing reliability of software programs during validation is a challenging task for engineers. The assessment is not only required to be unbiased, but it needs to provide tight variance (hence, tight confidence interval) with as few test cases as possible. Statistical sampling is a theoretically sound approach for reliability testing, but it is often impractical in its current form, because of too many test cases required to achieve desired confidence levels, especially when the software has few residual faults inside. We claim that the potential of statistical sampling methods is largely underestimated. This paper presents an adaptive sampling-based testing (AST) strategy for reliability assessment. A two-stage conceptual framework is defined, where adaptiveness is included to uncover residual faults earlier, while various sampling-based techniques are proposed to improve the efficiency (in terms of variance-test cases tradeoff) by better exploiting the information available to tester. An empirical study is conducted to assess the AST performance and compare the proposed sampling techniques to each other on real programs.

The idea of turning data centers executing scientific batch jobs into private clouds is as attractive as troubling. Cloud platforms may help both in limiting power consumption and in implementing fault tolerance strategies. However, there is also the fear that performance may worsen, and that the electricity required for longer job duration and fault tolerance implementation may overcome the saved one. In this paper, we present the consumability analysis for assessing the impact of cloud and fault tolerance tunings on scientific processing systems. The analysis considers performance, consumption, and dependability aspects, jointly. The aim is to pinpoint if, for a given system, there is a setting where consumption and job failure rate decrease, while performance is not affected. Applied to the scientific data center at our University, the analysis allowed us to find the proper selection of virtual machines' configuration, consolidation strategy, and fault tolerance tuning.

Testing software to assess or improve reliability presents several practical challenges. Conventional operational testing is a fundamental strategy that simulates the real usage of the system in order to expose failures with the highest occurrence probability. However, practitioners find it unsuitable for assessing/achieving very high reliability levels; also, they do not see the adoption of a “real” usage profile estimate as a sensible idea, being it a source of non-quantifiable uncertainty. Oppositely, debug testing aims to expose as many failures as possible, but regardless of their impact on runtime reliability. These strategies are used either to assess or to improve reliability, but cannot improve and assess reliability in the same testing session. This article proposes Reliability Assessment and Improvement (RELAI) testing, a new technique thought to improve the delivered reliability by an adaptive testing scheme, while providing, at the same time, a continuous assessment of reliability attained through testing and fault removal. The technique also quantifies the impact of a partial knowledge of the operational profile. RELAI is positively evaluated on four software applications compared, in separate experiments, with techniques conceived either for reliability improvement or for reliability assessment, demonstrating substantial improvements in both cases.

Energy efficiency of large processing systems is usually assessed as the relation between a performance and a power consumption metric, neglecting malfunction. Execution failures have a tangible cost in terms of wasted energy, however. They are often managed through fault tolerance mechanisms, which in turn consume electricity. We introduce the consumability attribute for batch processing systems, encompassing performance, consumption, and dependability aspects altogether. We propose a metric for its quantification and a methodology for its analysis. Using a real 500-node batch system as a case study, we show that consumability is representative of both efficiency and effectiveness, and we show the usefulness of the proposed metric and the suitability of the proposed methodology.

Model-Driven Engineering (MDE) promises to enhance system development by reducing development time, and increasing productivity and quality. MDE is gaining popularity in several industry sectors, and is attractive also for critical systems where they can reduce efforts and costs for verification and validation (V&V), and can ease certification. Incorporating model-driven techniques into a legacy well-proven development cycle is not simply a matter of placing models and transformations in the design and implementation phases. We present the experience in the model-driven design and V&V of a safety-critical system in the railway domain, namely the Prolan Block, a railway interlocking system manufactured by the Hungarian company Prolan Co., required to be CENELEC SIL-4 compliant. The experience has been carried out in an industrial-academic partnership within the EU project CECRIS. We discuss the challenges and the lessons learnt in this pilot project of introducing MD design and testing techniques into the company's traditional V-model process.

The practice of defect analysis is recognized as an essential task for software process measurement, yet its effective application in the industrial development of large-scale software systems raises several challenges. We report the results of a study conducted at SELEX ES – a large system integrator leader in the market of software-intensive mission-critical systems. The article describes the defect analysis approach that we tailored to evaluate the software development process with respect to the quality of produced software and its relation with the required effort. Three key phases of the process were addressed, regarding the software implementation, the testing phase and the prerelease defect fixing activity, over a set of six computer software configuration items developed from 2009 to 2012 for the naval and maritime domain product line. The analysis highlighted efficiency bottlenecks in each of the monitored phases, providing company engineers with insights about room for process improvement. The implemented approach, the observed phenomena and the inferred conclusions are of support to practitioners coping with systems, development models and industrial environments similar to the considered one. Copyright © 2014 John Wiley & Sons, Ltd.

Revealing anomalies at the operating system (OS) level to support online diagnosis activities of complex software systems is a promising approach when traditional detection mechanisms (e.g., based on event logs, probes and heartbeats) are inadequate or cannot be applied. In this paper we propose a configurable detection framework to reveal anomalies in the OS behavior, related to system misbehaviors. The detector is based on online statistical analyses techniques, and it is designed for systems that operate under variable and non-stationary conditions. The framework is evaluated to detect the activation of software faults in a complex distributed system for Air Traffic Management (ATM). Results of experiments with two different OSs, namely Linux Red Hat EL5 and Windows Server 2008, show that the detector is effective for mission-critical systems. The framework can be configured to select the monitored indicators so as to tune the level of intrusivity. A sensitivity analysis of the detector parameters is carried out to show their impact on the performance and to give to practitioners guidelines for its field tuning.

We investigate the dependence of software failure reproducibility on the environment in which the software is executed. The existence of such dependence is ascertained in literature, but so far it is not fully characterized. In this paper we pinpoint some of the environmental components that can affect the reproducibility of a failure and show this influence through an experimental campaign conducted on the My SQL Server software system. The set of failures of interest is drawn from My SQL's failure reports database and an experiment is designed for each of these failures. The experiments expose the influence of disk usage and level of concurrency on My SQL failure reproducibility. Furthermore, the results show that high levels of usage of these factors increase the probabilities of failure reproducibility.

Software aging is a phenomenon plaguing many long-running complex software systems, which exhibit performance degradation or an increasing failure rate. Several strategies based on the proactive rejuvenation of the software state have been proposed to counteract software aging and prevent failures. This survey article provides an overview of studies on Software Aging and Rejuvenation (SAR) that have appeared in major journals and conference proceedings, with respect to the statistical approaches that have been used to forecast software aging phenomena and to plan rejuvenation, the kind of systems and aging effects that have been studied, and the techniques that have been proposed to rejuvenate complex software systems. The analysis is useful to identify key results from SAR research, and it is leveraged in this article to highlight trends and open issues.

In a large Infrastructure-as-a-Service (IaaS) cloud, component failures are quite common. Such failures may lead to occasional system downtime and eventual violation of Service Level Agreements (SLAs) on the cloud service availability. The availability analysis of the underlying infrastructure is useful to the service provider to design a system capable of providing a defined SLA, as well as to evaluate the capabilities of an existing one. This paper presents a scalable, stochastic model-driven approach to quantify the availability of a large-scale IaaS cloud, where failures are typically dealt with through migration of physical machines among three pools: hot (running), warm (turned on, but not ready), and cold (turned off). Since monolithic models do not scale for large systems, we use an interacting Markov chain based approach to demonstrate the reduction in the complexity of analysis and the solution time. The three pools are modeled by interacting sub-models. Dependencies among them are resolved using fixed-point iteration, for which existence of a solution is proved. The analytic-numeric solutions obtained from the proposed approach and from the monolithic model are compared. We show that the errors introduced by interacting sub-models are insignificant and that our approach can handle very large size IaaS clouds. The simulative solution is also considered for the proposed model, and solution time of the methods are compared.

Cyber attacks have become surprisingly sophisticated over the past fifteen years. While early infections mostly targeted individual machines, recent threats leverage the widespread network connectivity to develop complex and highly coordinated attacks involving several distributed nodes [1]. Attackers are currently targeting very diverse domains, e.g., e-commerce systems, corporate networks, datacenter facilities and industrial systems, to achieve a variety of objectives, which range from credentials compromise to sabotage of physical devices, by means of smarter and smarter worms and rootkits. Stuxnet is a recent worm that well emphasizes the strong technical advances achieved by the attackers' community. It was discovered in July 2010 and firstly affected Iranian nuclear plants [2]. Stuxnet compromises the regular behavior of the supervisory control and data acquisition (SCADA) system by reprogramming the code of programmable logic controllers (PLC). Once compromised, PLCs can progressively destroy a device (e.g., components of a centrifuge, such as the case of the Iranian plant) by sending malicious control signals. Stuxnet combines a relevant number of challenging features: it exploits zero-days vulnerabilities of the Windows OS to affect the nodes connected to the PLC; it propagates either locally (e.g., by means of USB sticks) or remotely (e.g., via shared folders or the print spooler vulnerability); it is able to modify its behavior during the progression of the attack, and communicates with a remote command and control server. More importantly, Stuxnet can remain undetected for many months [3] because it is able to fool the SCADA system by emulating regular monitoring signals.

This work presents a method to combine testing techniques adaptively during the testing process. It intends to mitigate the sources of uncertainty of software testing processes, by learning from past experience and, at the same time, adapting the technique selection to the current testing session. The method is based on machine learning strategies. It uses offline strategies to take historical information into account about the techniques performance collected in past testing sessions; then, online strategies are used to adapt the selection of test cases to the data observed as the testing proceeds. Experimental results show that techniques performance can be accurately characterized from features of the past testing sessions, by means of machine learning algorithms, and that integrating this result into the online algorithm allows improving the fault detection effectiveness with respect to single testing techniques, as well as to their random combination.

Mandelbugs are faults that are triggered by complex conditions, such as interaction with hardware and other software, and timing or ordering of events. These faults are considerably difficult to detect with traditional testing techniques, since it can be challenging to control their complex triggering conditions in a testing environment. Therefore, it is necessary to adopt specific verification and/or fault-tolerance strategies for dealing with them in a cost-effective way. In this paper, we investigate how to predict the location of Mandelbugs in complex software systems, in order to focus V&V activities and fault tolerance mechanisms in those modules where Mandelbugs are most likely present. In the context of an industrial complex software system, we empirically analyze Mandelbugs, and investigate an approach for Mandelbug prediction based on a set of novel software complexity metrics. Results show that Mandelbugs account for a noticeable share of faults, and that the proposed approach can predict Mandelbug-prone modules with greater accuracy than the sole adoption of traditional software metrics.

Continuous or high availability is a key requirement for many modern IT systems. Computer operating systems play an important role in IT systems availability. Due to the complexity of their architecture, they are prone to suffer failures due to several types of software faults. Software aging causes a nonnegligible fraction of these failures. It leads to an accumulation of errors with time, increasing the system failure rate. This phenomenon can be accompanied by performance degradation and eventually system hang or even crash. As a countermeasure, software rejuvenation entails stopping the system, cleaning its internal state, and resuming its operation. This process usually incurs downtime. For an operating system, the downtime impacts any application running on top of it. Several solutions have been developed to speed up the boot time of operating systems in order to reduce the downtime overhead. We present a study of two fast OS reboot techniques for rejuvenation of Linux-based operating systems, namely Kexec and Phase-based reboot. The study measures the performance penalty they introduce and the gain in reduction of downtime overhead. The results reveal that the Kexec and Phase-based reboot have no statistically significant impact in terms of performance penalty from the user perspective. However, they may require extra resource (e.g., CPU) usage. The downtime overhead reduction, compared with normal Linux and VM reboots, is 77% and 79% in Kexec and Phase-based reboot, respectively.

SUMMARY In this work, a software ageing analysis of Java‐based software systems is conducted. The JVM is the core layer in Java‐based systems, and its dependability greatly affects the overall system quality. Starting from an experimental campaign on a real‐world test bed, this work isolates the contribution of the JVM to the overall ageing trend, and identifies, through statistical methods, which workload parameters are the most relevant to ageing dynamics. Results revealed the presence of several ageing dynamics in the JVM, including (i) a throughput loss trend mainly dependent on the execution unit, (ii) a slow memory depletion drift due to the just‐in‐time‐compiler activity and (iii) a fast memory depletion drift caused by dynamics inside the garbage collector. The outlined procedure and obtained results are useful in order to (i) identify the presence of ageing phenomena, (ii) perform online ageing detection and time‐to‐exhaustion prediction and (iii) define optimal rejuvenation techniques. Copyright © 2011 John Wiley & Sons, Ltd.

This paper addresses the challenge of reliability-driven testing, i.e., of testing software systems with the specific objective of increasing its operational reliability. We first examined the most relevant approach oriented toward this goal, namely operational testing. The main issues that in the past hindered its wide-scale adoption and practical application are first discussed, followed by the analysis of its performance under different conditions and configurations. Then, a new approach conceived to overcome the limits of operational testing in delivering high reliability is proposed. The two testing strategies are evaluated probabilistically, and by simulation. Results report on the performance of operational testing when several involved parameters are taken into account, and on the effectiveness of the new proposed approach in achieving better reliability. At a higher level, the findings of the paper also suggest that a different view of the testing for reliability improvement concept may help to devise new testing approaches for high-reliability, demanding systems.

This study investigates software aging effects caused by the activation of concurrency bugs in a wellknown database management system (DBMS), namely MySQL. Experiments with different workloads are performed in order to reproduce the most likely conditions for concurrency bugs activation. Besides the typical aging effects observed in many operational systems (i.e., a gradual degradation over time), results highlight that both available resources and DBMS performance (e.g. service rate, service time, and connection latency) can decrease with time in a hard-to-predict way. We observed that, due to the activation of concurrency bug, the DBMS enters a degraded state in which: i) the estimation of Time-To-Failure (TTF) by means of memory depletion trend analysis is highly inaccurate, and ii) the failure rate does not depend on the instantaneous and/or mean accumulated work. Results suggest that, in such cases, finer-grained indicators and/or different techniques need to be taken into account for properly preventing failures.

Software faults are recognized to be among the main responsible for system failures in many application domains. Event logs play a key role to support the analysis of failures occurring under real workload conditions. Nevertheless, field experience suggests that event logs may be inaccurate at reporting software failures or they fail to provide accurate support for understanding their causes. This paper analyzes the factors that determine accurate detection of software failures through event logs. The study is based on a data set of 17,387 experiments where failures have been induced by means of software fault injection into three systems. Analysis reveals that the reporting ability of logs collected during the experiments, is not influenced by the type of fault that is activated at runtime. More importantly, analysis demonstrates that, despite the considered systems adopt very similar detection mechanisms, the ability of logs at reporting a given type of failure changes significantly across the systems. A closer inspection of collected logs reveals that characteristics, such as system architecture, placement of the logging instructions and specific supports provided by the execution environment, significantly increase accuracy of logs at runtime.

The phenomenon of software aging is increasingly recognized as a relevant problem of long-running systems. Numerous experiments have been carried out in the last decade to empirically analyze software aging. Such experiments, besides highlighting the relevance of the phenomenon, have shown that aging is tightly related to the applied workload. However, due to the differences among the experimented applications and among the experimental conditions, results of past studies are not comparable to each other. This prevent from drawing general conclusions (e.g., about the aging-workload relationship), and from comparing systems from the aging perspective. In this paper, we propose a procedure to carry out aging experiments in different applications for: i) assessing aging trend of the individual systems, as well as assessing differences among them (i.e., obtaining comparable results), ii) inferring workload-aging relationships from experiments performed on different applications, by highlighting the most relevant workload parameters. The procedure is applied, through a set of long-running experiments, to three real-scale software applications, namely Apache Web Server, James Mail Server, and CARDAMOM, a middleware for the development of air traffic control (ATC) systems.

Publish/subscribe middleware is being increasingly used to devise large-scale critical systems. Although several reliable publish/subscribe solutions have been proposed, none of them properly address the problem of assuring message dissemination even if network omissions happen without breaking any temporal constraints. In order to fill this gap, we have investigated how to guarantee a resilient and timely event dissemination despite of message losses. The contribution of this paper is on proposing a FEC approach, where encoding functionality is placed at the root and on a subset of interior nodes in the multicast tree, combined to a gossiping algorithm. Simulation-based experiments demonstrate that the proposed approach allows all the interested subscribers to receive all the published messages and the adopted resiliency mean does not affect the timeliness of the multicast protocol.

Software systems running continuously for a long time tend to show degrading performance and an increasing failure occurrence rate, due to error conditions that accrue over time and eventually lead the system to failure. This phenomenon is usually referred to as Software Aging. Several long-running mission and safety critical applications have been reported to experience catastrophic aging-related failures. Software aging sources (i.e., aging-related bugs) may be hidden in several layers of a complex software system, ranging from the Operating System (OS) to the user application level. This paper presents a software aging analysis at the Operating System level, investigating software aging sources inside the Linux kernel. Linux is increasingly being employed in critical scenarios; this analysis intends to shed light on its behaviour from the aging perspective. The study is based on an experimental campaign designed to investigate the kernel internal behaviour over long running executions. By means of a kernel tracing tool specifically developed for this study, we collected relevant parameters of several kernel subsystems. Statistical analysis of collected data allowed us to confirm the presence of aging sources in Linux and to relate the observed aging dynamics to the monitored subsystems behaviour. The analysis output allowed us to infer potential sources of aging in the kernel subsystems.

With software systems increasingly being employed in critical contexts, assuring high reliability levels for large, complex systems can incur huge verification costs. Existing standards usually assign predefined risk levels to components in the design phase, to provide some guidelines for the verification. It is a rough-grained assignment that does not consider the costs and does not provide sufficient modeling basis to let engineers quantitatively optimize resources usage. Software reliability allocation models partially address such issues, but they usually make so many assumptions on the input parameters that their application is difficult in practice. In this paper, we try to reduce this gap, proposing a reliability and testing resources allocation model that is able to provide solutions at various levels of detail, depending upon the information the engineer has about the system. The model aims to quantitatively identify the most critical components of software architecture in order to best assign the testing resources to them. A tool for the solution of the model is also developed. The model is applied to an empirical case study, a program developed for the European Space Agency, to verify model's prediction abilities and evaluate the impact of the parameter estimation errors on the prediction accuracy.

As the incidence of faults in real Wireless Sensor Networks (WSNs) increases, fault injection is starting to be adopted to verify and validate their design choices. Following this recent trend, this paper presents a tool, named AVR-INJECT, designed to automate the fault injection, and analysis of results, on WSN nodes. The tool emulates the injection of hardware faults, such as bit flips, acting via software at the assembly level. This allows to attain simplicity, while preserving the low level of abstraction needed to inject such faults. The potential of the tool is shown by using it to perform a large number of fault injection experiments, which allow to study the reaction to faults of real WSN software.

We propose a fault injection framework to assess hang detection facilities within the Linux operating system (OS). The novelty of the framework consists in the adoption of a more representative fault load than existing ones, and in the effectiveness in terms of number of hang failures produced; representativeness is supported by a field data study on the Linux OS. Using the proposed fault injection framework, along with realistic workloads, we find that the Linux OS is unable to detect hangs in several cases. We experience a relative coverage of 75%. To improve detection facilities, we propose a simple yet effective hang detector, which periodically tests OS liveness, as perceived by applications, by means of I/O system calls; it is shown that this approach can improve relative coverage up to 94%. The hang detector can be deployed on any Linux system, with an acceptable overhead.

Abstract Location‐aware computing is a form of context‐aware mobile computing that refers to the ability of providing users with services that depend on their position. Locating the user terminal, often called positioning, is essential in this form of computing. Towards this aim, several technologies exist, ranging from personal area networking, to indoor, outdoor, and up to geographic area systems. Developers of location‐aware software applications have to face with a number of design choices, that typically depend on the chosen technology. This work addresses the problem of easing the development of pull location‐aware applications, by allowing uniform access to multiple heterogeneous positioning systems. Towards this aim, the paper proposes an approach to structure location‐aware mobile computing systems in a way independent of positioning technologies. The approach consists in structuring the system into a layered architecture, that provides application developers with a standard Java Application Programming Interface (JSR‐179 API), and encapsulates location data management and technology‐specific positioning subsystems into lower layers with clear interfaces. In order to demonstrate the proposed approach we present the development of HyLocSys. It is an open hybrid software architecture designed to support indoor/outdoor applications, which allows the uniform (combined or separate) use of several positioning technologies. HyLocSys uses a hybrid data model, which allows the integration of different location information representations (using symbolic and geometric coordinates). Moreover, it allows support to handset‐ and infrastructure‐based positioning approaches while respecting the privacy of the user. The paper presents a prototypal implementation of HyLocSys for heterogeneous scenarios. It has been implemented and tested on several platforms and mobile devices. Copyright © 2009 John Wiley & Sons, Ltd.

Event-driven architectures (EDAs) are widely used to make distributed mission critical software systems more- efficient and scalable. In the context of EDAs, data distribution service (DDS) is a recent standard by the object management group that offers a rich support for quality- of-service and balances predictable behavior and implementation efficiency. The DDS specification does not outline how messages are delivered, so several architectures are nowadays available. This paper focuses on performance assessment of OMG DDS-compliant middleware technologies. It provides three contributions to the study of evaluating the performance of DDS implementations: 1) describe the challenges to be addressed; 2) propose possible solutions; 3) define a representative workload scenario for evaluating the performance and scalability of DDS platforms. At the end of the paper, a case study of DDS performance assessment, performed with the proposed benchmark, is presented.

This work presents a measurement-based dependability evaluation of the Bluetooth data communication channel, i.e., the Baseband layer. The main contribution is the definition of the Baseband's error/recovery model according to the Markov chains formalism. The model is derived by analyzing field data, which are collected via a commercial air sniffer deployed over real- world Bluetooth piconets. The model is parametric and actual values for its parameters are estimated by analyzing the field data. The paper also proposes the evaluation of dependability statistics (e.g., the error and failure times distributions, and the availability estimate), and the study of the failing behavior of the Bluetooth communication channel under Wi-Fi interferences.

This paper proposes a flexible framework for dependability modeling and assessing of Wireless Sensor Networks (WSNs). The framework takes into account network related aspects (topology, routing, network traffic) as well as hardware/software characteristics of nodes (type of sensors, running applications, power consumption). It is composed of two basic elements: i) a parametric Stochastic Activity Networks (SAN) failure model, reproducing WSN failure behavior as inferred from a detailed Failure Mode Effect Analysis (FMEA), and ii) an external library reproducing network behavior on behalf of the SAN model. This library specializes the SAN model by feeding it with quantitative parameters obtained by simulation or by experimental campaigns; it is also in charge of updating the network state in response to failure events during the simulation (e.g., routing tree updated due to node failures). The framework is thus suited to evaluate the dependability of several WSNs, with different topologies, routing algorithms, hardware/software platforms, without requiring any changes to its structure. The use of the external library makes the model simpler, decoupling the network behavior from the failure behavior. Simulation experiments are discussed that provide a quantitative evaluation of WSN dependability for a sample scenario: results show how the proposed framework supports WSN developers to find proper cost-reliability trade-offs for the system being deployed.

In this work we investigate software aging phenomena inside the Java Virtual Machine (JVM). Starting from an experimental campaign on real world testbeds, this work isolates the contribution of the JVM to the overall aging trend, and identifies, through statistical methods, which workload parameters are more relevant to aging dynamics. Experimental results show that the Sun Hotpost JVM experiences software aging phenomena. A consistent memory depletion trend (up to 50 KB/min) has been observed during periods of low garbage collector activity; the Just-In-Time compiler is also responsible for a lighter, but not negligible, memory depletion trend; finally, a consistent throughput loss (up to 24 KB/min) has been observed.

Abstract There is an increasing demand for middleware for nomadic computing applications. Owing to the inherent characteristics of such environments, these platforms have to address two fundamental issues: (i) device disconnections and the limitations of wireless networks may force users to experience short periods of service unavailability; and (ii) the complexity to design and develop next‐generation mobile computing applications. This paper proposes the Esperanto Broker (EB), a communication platform that addresses mobility issues via an integrated approach, i.e. at data‐link, network, and middleware levels. Decoupling interactions are achieved via a tuple‐space underlying infrastructure. To support developers with advanced services, the EB enhances the distributed objects computing model providing the abstraction for the communication paradigms standardized by the W3C. Esperanto applications can be modeled as sets of objects that are distributed over mobile devices, which communicate via remote method invocations (RMIs). RMIs natively implement pull and push models, in both one‐to‐one and one‐to‐many multiplicity. The paper focuses on the EB design issues, essential aspects of the implementation, and performance evaluations of the implemented prototype. Copyright © 2006 John Wiley & Sons, Ltd.

This work presents a failure data analysis campaign on Bluetooth personal area networks (PANs) conducted on two kind of heterogeneous testbeds (working for more than one year). The obtained results reveal how failures distribution is characterized and suggest how to improve the dependability of Bluetooth PANs. Specifically, we define the failure model and we then identify the most effective recovery actions and masking strategies that can be adopted for each failure. We then integrate the discovered recovery actions and masking strategies in our testbeds, improving the availability and the reliability of 3.64% (up to 36.6%) and 202% (referred to the mean time to failure), respectively

This paper presents a failure analysis of the Java Virtual Machine providing useful insights into the nature of reported failures and to improve the understanding of its dependability aspects. Failure data is extracted from publicly available bug databases, where developers and users of Java applications usually submit failures/bugs. Presented results clearly indicate that much more efforts have still to be done in order to improve the dependability of the JVM. In particular, the conducted analysis revealed that i) builtin error detection mechanism are characterized by a low coverage; ii) the JVM does not achieve the same levels of dependability across different platforms iii) developers have to pursue a tradeoff between performance and reliability. Finally, code fragments reproducing failures submitted in bug database are injected into Java Applications. Preliminary results show that often these faults could be removed changing the environment of the JVM.

A monitoring infrastructure is a key component in each task aimed at evaluating the dependability of a system. This paper presents a monitoring infrastructure for the Java Virtual Machine (JVM), which is starting to be employed in mission and safety critical application, often with real-time requirements. This infrastructure, named JVMMon, collects data about both the state and the failures of the monitored Virtual Machine. The state of the JVM was defined according to the Java Virtual Machine specification. JVMMon is constituted by three components: a monitoring agent which collects data from the monitored VM; a local monitor daemon that receives data from such agent and updates the state of the JVM; a data collector, which stores events and state snapshots in a database. The impact on the performance of the JVM has been evaluated running the SPEC JVM98 benchmark suite

Summary form only given. The most challenging issues in nomadic computing environments arise from the combination of heterogeneity, dynamism, context-awareness, and mobility. Driven by these issues, this paper presents a new middleware infrastructure, named ESPERANTO, to support the integration of diverse nomadic computing domains. This middleware aims to glue the emerging heterogeneous nomadic computing technologies and service oriented architectures.

The widespread use of mobile and wireless computing platforms is leading to a growing interest on dependability issues. Several research studies have been conducted on dependability of mobile environments, but none of them attempted to identify system bottlenecks and to quantify dependability measures. This paper proposes a distributed automated infrastructure for monitoring and collecting spontaneous failures of the Bluetooth infrastructure, which is nowadays more and more recognized as an enabler for mobile systems. Information sources for failure data are presented, and preliminary experimental results are discussed.

In this paper we investigate dependability issues of next-generation distributed multimedia applications. Examples of such applications are autonomous vehicle control, tele-medicine, and audio/video control. For these applications the quality of the delivered multimedia data is a critical factor. According to the ITU-T (working group SG 12), the quality of a multimedia service as perceived by end-users is defined by three parameters: delay, delay variation, and information loss. It is paramount to formalize the concept of a failure from the user's perspective. This paper defines the correctness of a multimedia service as a function of temporal distributions of the user-related parameters. It proposes a strategy for modeling and detecting failures of the considered applications. In particular, the detection process is based on error filtering functions. We show that the combination of threshold-based mechanisms is suitable for implementing an efficient detection strategy. We also evaluate the effectiveness of the proposed mechanism both by simulations and by experiments performed on a prototype. Such a prototype is tested with respect to a case study application, consisting of distributed remote-control based on RTP/RTCP standard streaming protocols.

The provisioning of multimedia services with guaranteed Quality-of-Service (QoS) is currently an important research issue in computer engineering, especially in the networking and information management areas. In this paper, we concentrate on the QoS-based provisioning of discovery and delivery services of multimedia resources in the educational application domain. We present a generic model for QoS-aware information management systems, able to support guarantee of the QoS in search and delivery of information objects over a network infrastructure (such as the Internet or an inter-organizations network). The model identifies actors and components involved, and their role. The metadata models we analyse are specialized for the education domain. For this application domain, we also propose QoS extensions to standard metadata models for profiling users, information services and resources, capable of supporting learning resources discovery and personalized delivery with QoS guarantee.

CSP and Petri Nets are powerful formalisms for the specification and the analysis of concurrent systems. We present an approach to their integration to take advantage of both formalisms. In particular the GSPN class is used to address dependability and real-time aspects. In this paper an algorithmic transformation from a trace-based specification of a concurrent system to a Petri Net model is described. Causal dependencies between behaviours of the system components are introduced in the net model through the definition of external assumptions. The steps of the integration are illustrated by applying them to an unmanned transportation problem.

No abstract available.

Abstract We describe the organization of a general purpose data archival system for Write‐Once, Read‐Many (WORM) optical disks. The system has been designed for large‐scale and long‐term data storage and retrieval. The archival system is independent of the operating system, flat, self‐consistent, does not use any write cache on magnetic disk, and allows the exploitation of auxiliary information on magnetic disk, which can be rebuilt immediately in case of a crash, to speed up file retrieval. A library in C language, called pODLIB, has been implemented as a portable interface to the archival system.