Monitoring is a key feature to enhance systems with the capability to anticipate, detect, predict, and mitigate failures, while providing Quality of Service (QoS) monitoring and Service Level Agreements (SLAs) guarantee. Monitoring frameworks can serve these purposes by deploying probes according to many possible patterns that have different features, for instance in terms of efficiency and privacy. So far, these probe deployment patterns have not been systematically defined, analyzed and assessed. Thus, engineers who design and configure their monitoring systems have to take decisions only based on partial knowledge and personal experience. This paper addresses this knowledge gap, by presenting a systematic analysis of 11 probe deployment patterns, their known uses, and implementations. We assess these patterns qualitatively, and quantitatively using both VMs and containers. Results show the targets have negligible resource consumption (e.g., less than 1% CPU usage), while the probe holder consumption is mainly significant in relation to memory consumption, reaching up to 10 GiB in our experiments. Our findings suggest that reusing probes and holders among users can generally enhance efficiency and scalability when direct access to the monitored target is not an option. We generate a set of best practices that can assist engineers in configuring their monitoring systems. Finally, we showcase the application of certain patterns through three practical usage scenarios, which feature diverse technologies and requirements.

Many automatic Web testing techniques generate test cases by analyzing the GUI of the Web applications under test, aiming to exercise sequences of actions that are similar to the ones that testers could manually execute. However, the efficiency of the test generation process is severely limited by the cost of analyzing the content of the GUI screens after executing each action. In this paper, we introduce an inference component, Sibilla, which accumulates knowledge about the behavior of the GUI after each action. Sibillaenables the test generators to reuse the results computed for GUI screens that recur multiple times during the test generation process, thus improving the efficiency of Web testing techniques.We experimented Sibillawith Web testing techniques based on three different GUI exploration strategies (Random, Depth-first, and Q-learning) and nine target systems, observing reductions from 22% to 96% of the test generation time.

The generation of syntactically and semantically valid input data, able to exercise functionalities imposing constraints on the validity of the inputs, is a key challenge in automatic GUI (Graphical User Interface) testing.Existing test case generation techniques often rely on manually curated catalogs of values, although they might require significant effort to be created and maintained, and could hardly scale to applications with several input forms. Alternatively, it is possible to extract values from external data sources, such as the Web or publicly available knowledge bases. However, external sources are unlikely to provide the domain-specific and application-specific data that are often required to thoroughly exercise applications.This paper proposes DBINPUTS, a novel approach that automatically identifies domain-specific and application-specific inputs to effectively fulfill the validity constraints present in the tested GUI screens. The approach exploits syntactic and semantic similarities between the identifiers of the input fields shown on GUI screens and those of the tables of the target GUI application database, and extracts valid inputs from such database, automatically resolving the mismatch between the user interface and the database schema. DBINPUTS can properly cope with system testing and maintenance testing efforts, since databases are naturally and inexpensively available in those phases.Our experiments with 4 Web applications and 11 Mobile apps provide evidence that DBINPUTS can outperform techniques like random input selection and LINK, a competing approach for searching inputs from knowledge bases, in both Web and Mobile domains.

The advent of edge devices dedicated to machine learning tasks enabled the execution of AI-based applications that efficiently process and classify the data acquired by the resource-constrained devices populating the Internet of Things. The proliferation of such applications (e.g., critical monitoring in smart cities) demands new strategies to make these systems also sustainable from an energetic point of view. In this paper, we present an energy-aware approach for the design and deployment of self-adaptive AI-based applications that can balance application objectives (e.g., accuracy in object detection and frames processing rate) with energy consumption. We address the problem of determining the set of configurations that can be used to self-adapt the system with a meta-heuristic search procedure that only needs a small number of empirical samples. The final set of configurations are selected using weighted gray relational analysis, and mapped to the operation modes of the self-adaptive application. We validate our approach on an AI-based application for pedestrian detection. Results show that our self-adaptive application can outperform non-adaptive baseline configurations by saving up to 81% of energy while loosing only between 2% and 6 % in accuracy.

Abstract Microservice‐based applications consist of multiple services that can evolve independently. When a service must be updated, it is first tested with in‐house regression test suites. However, the test suites that are executed are usually designed without the exact knowledge about how the services will be accessed and used in the field; therefore, they may easily miss relevant test scenarios, failing to prevent the deployment of faulty services. To address this problem, we introduce ExVivoMicroTest, an approach that analyzes the execution of deployed services at run‐time in the field, in order to generate test cases for future versions of the same services. ExVivoMicroTest implements lightweight monitoring and tracing capabilities, to inexpensively record executions that can be later turned into regression test cases that capture how services are used in the field. To prevent accumulating an excessive number of test cases, ExVivoMicroTest uses a test coverage model that can discriminate the recorded executions between the ones that are worth to be turned into test cases and the ones that should be discarded. The resulting test cases use a mocked environment that fully isolates the service under test from the rest of the system to faithfully reply interactions. We assessed ExVivoMicroTest with the PiggyMetrics and Train Ticket open source microservice applications and studied how different configurations of the monitoring and tracing logic impact on the capability to generate test cases.

Cloud services must be continuously monitored to guarantee that misbehaviors can be timely revealed, compensated, and fixed. While simple applications can be easily monitored and controlled, monitoring non-trivial cloud systems with dynamic behavior requires the operators to be able to rapidly adapt the set of collected indicators. Although the currently available monitoring frameworks are equipped with a rich set of probes to virtually collect any indicator, they do not provide the automation capabilities required to quickly and easily change (i.e., deploy and undeploy) the probes used to monitor a target system. Indeed, changing the collected indicators beyond standard platform-level indicators can be an error-prone and expensive process, which often requires manual intervention. This article presents a Monitoring-as-a-Service framework that provides the capability toautomaticallydeploy and undeploy arbitrary probes based on a user-provided set of indicators to be collected. The life-cycle of the probes is fully governed by the framework, including the detection and resolution of theerroneous statesat deployment time. The framework can be used jointly withexisting monitoring technologies, without requiring the adoption of a specific probing technology. We experimented our framework with cloud systems based on containers and virtual machines, obtaining evidence of the efficiency and effectiveness of the proposed solution.

Hierarchical Temporal Memory (HTM) is an unsupervised learning algorithm inspired by the features of the neocortex that can be used to continuously process stream data and detect anomalies, without requiring a large amount of data for training nor requiring labeled data. HTM is also able to continuously learn from samples, providing a model that is always up-to-date with respect to observations.These characteristics make HTM particularly suitable for supporting online failure prediction in cloud systems, which are systems with a dynamically changing behavior that must be monitored to anticipate problems. This paper presents the first systematic study that assesses HTM in the context of failure prediction.The results that we obtained considering 72 configurations of HTM applied to 12 different types of faults introduced in the Clearwater cloud system show that HTM can help to predict failures with sufficient effectiveness (F-measure = 0.76), representing an interesting practical alternative to (semi-)supervised algorithms.

A key challenge in automatic Web testing is the generation of syntactically and semantically valid input values that can exercise the many functionalities that impose constraints on the validity of the inputs. Existing test case generation techniques either rely on manually curated catalogs of values, or extract values from external data sources, such as the Web or publicly available knowledge bases. Unfortunately, relying on manual effort is generally too expensive for most practical applications, while domain-specific and application-specific data can be hardly found either on the Web or in general purpose knowledge bases.

Mobile operating systems evolve quickly, frequently updating the APIs that app developers use to build their apps. Unfortunately, API updates do not always guarantee backward compatibility, causing apps to not longer work properly or even crash when running with an updated system. This paper presents FILO, a tool that assists Android developers in resolving backward compatibility issues introduced by API upgrades. FILO both suggests the method that needs to be modified in the app in order to adapt the app to an upgraded API, and reports key symptoms observed in the failed execution to facilitate the fixing activity. Results obtained with the analysis of 12 actual upgrade problems and the feedback produced by early tool adopters show that FILO can practically support Android developers. FILO can be downloaded from https://gitlab.com/learnERC/filo, and its video demonstration is available at https://youtu.be/WDvkKj-wnlQ.

Dealing with the evolution of operating systems is challenging for developers of mobile apps, who have to deal with frequent upgrades that often include backward incompatible changes of the underlying API framework. As a consequence of framework upgrades, apps may show misbehaviours and unexpected crashes once executed within an evolved environment. Identifying the portion of the app that must be modified to correctly execute on a newly released operating system can be challenging. Although incompatibilities are visibile at the level of the interactions between the app and its execution environment, the actual methods to be changed are often located in classes that do not directly interact with any external element. To facilitate debugging activities for problems introduced by backward incompatible upgrades of the operating system, this paper presents FILO, a technique that can recommend the method that must be changed to implement the fix from the analysis of a single failing execution. FILO can also select key symptomatic anomalous events that can help the developer understanding the reason of the failure and facilitate the implementation of the fix. Our evaluation with multiple known compatibility problems introduced by Android upgrades shows that FILO can effectively and efficiently identify the faulty methods in the apps.

Android apps must be able to deal with both stop events, which require immediately stopping the execution of the app without losing state information, and start events, which require resuming the execution of the app at the same point it was stopped. Support to these kinds of events must be explicitly implemented by developers who unfortunately often fail to implement the proper logic for saving and restoring the state of an app. As a consequence apps can lose data when moved to background and then back to foreground (e.g., to answer a call) or when the screen is simply rotated. These faults can be the cause of annoying usability issues and unexpected crashes. This paper presents a public benchmark of 110 data loss faults in Android apps that we systematically collected to facilitate research and experimentation with these problems. The benchmark is available on GitLab and includes the faulty apps, the fixed apps (when available), the test cases to automatically reproduce the problems, and additional information that may help researchers in their tasks.

In the last years supervised machine learning techniques are largely employed for automatic Human Activity Recognition (HAR) using inertial sensors, such as accelerometer and gyroscope. HAR has many applications in several domains such as, for example, healthcare, sport, and entertainment. Machine learning scientists made available to the community a plenty of labeled databases for benchmarking that, unfortunately, are not consistent, both syntactically (e.g., different sampling frequency) and semantically (e.g., labels with different meanings). Commonly, due to this inconsistency, scientists evaluate their progress on individual databases separately, which corresponds to training and testing using the same database. Coherent merging of existing databases would enable: 1) evaluation of generalization capabilities of methods across databases; 2) use of deep learning techniques that, unlike traditional ones, require much more labeled data for the training process. Moreover, the growth in the daily use of wearable devices will produce a big amount of inertial data which, if not correctly labeled, cannot be efficiently exploited for the study of automatic HAR. In this paper we propose a semi-automatic procedure to coherently merge existing databases based on signal and word similarity. Preliminary experiments demonstrates the effectiveness of the proposed procedure.

Cloud systems are large scalable distributed systems that must be carefully monitored to timely detect problems and anomalies. While a number of cloud monitoring frameworks are available, only a few solutions address the problem of adaptively and dynamically selecting the indicators that must be collected, based on the actual needs of the operator. Unfortunately, these solutions are either limited to infrastructure-level indicators or technology-specific, for instance, they are designed to work with OpenStack but not with other cloud platforms. This paper presents the VARYS monitoring framework, a technology-agnostic Monitoring-as-a-Service solution that can address KPI monitoring at all levels of the Cloud stack, including the application-level. Operators use VARYS to indicate their monitoring goals declaratively, letting the framework to perform all the operations necessary to achieve a requested monitoring configuration automatically. Interestingly, the VARYS architecture is general and extendable, and can thus be used to support increasingly more platforms and probing technologies.

The paper presents a layered architecture that improves software modularity and reduces computational and communication overhead for systems requiring data from sensors in order to perform domain-related elaborations (e.g., tracking and surveillance systems). Each layer manages hypotheses that are abductions related to objects modeling the ”real world” at a specific abstraction level, from raw data up to domain concepts. Each layer, by analyzing hypotheses coming from the lower layer, abduces new hypotheses regarding objects at a higher level of abstraction (e.g., from image blobs to identified people) and formulates timed previsions about objects. The failure of a prevision causes a hypothesis to flow up-stream. In turn, previsions can flow downstream, so that their verification is delegated to the lower layers. The proposed architectural patterns have been reified in a Java framework, which is being exploited in an experimental multi-camera tracking system.