The orchestration of application instances is critical for the efficient management of cloud computing platforms. Specifically, the serverless paradigm automates container spawning and de-spawning based on actual load, mitigating inefficiencies, such as over- and under-provisioning, that might compromise Service Level Objectives (SLOs). This dynamic behavior introduces significant challenges concerning initialization and termination latencies, which are exacerbated when enforcing real-time requirements in mixed-criticality systems. The existing literature already addresses key issues, such as reducing cold-start times and assuring real-time performance to deployed instances. However, container orchestration times remain an overlooked factor that can severely affect instance startup times, especially when the orchestrator is subject to intense workloads. In this paper, we present PREEMPT-FaaS, an orchestration controller that, unlike commonly adopted controllers, adopts a fixed-priority preemptive scheduling of requests to guarantee reduced orchestration times to high-priority and highly critical instances. We implemented PREEMPT-FaaS as a Rust custom controller for Kubernetes (K8s), along with a patch for Knative, a popular serverless platform built upon K8s. We perform an extensive experimental campaign of PREEMPT-FaaS, including the serving of AI workloads, such as, recurring neural networks and video analytics, showing up to ∼6× reduction of orchestration times under high load and improving end-to-end cold-start times of critical instances, with a consequent reduction of service-level latencies (up to ∼2 s reduction under stress at the 95th percentile).

Regression testing enhances software reliability by detecting regressions in new versions. Regression test suites often lack awareness of real-world product/service usage, potentially leading to undetected faults and ineffective testing scenarios. We propose LogTest, a transformer-based approach that learns from event logs and system traces to automatically generate service invocation sequences that mimic observed system behaviors. These sequences enhance regression test suites by exposing past real execution patterns. A preliminary experimentation on a realistic benchmark demonstrates its potential.

The paper presents Open-FARI, an open-source testbed for evaluating federated learning algorithms for anomaly detection in the railway Industrial Internet of Things domain. Open-FARI uses synthetic data generation modules trained from real train sensor data to generate realistic sensor data of a fleet of trains. Generated data encompass normal and anomalous data, enabling the evaluation of federated learning algorithms for anomaly detection. The paper addresses the lack of testbeds and datasets tailored to the railway domain, which represents an obstacle to research on Machine Learning-driven solutions in this domain.

The modeling and simulation of Internet of Things (IoT) and Industrial IoT (IIoT) systems allow practitioners to obtain valuable insights into the system's behavior before their actual deployment in the field. Early designing permits the analysis of the interactions among the involved entities, evaluating the effects of modifications, and understanding the impact of failures on the system. In particular, this is exacerbated in the context of IoT/IIoT, which is characterized by multiple and heterogeneous subsystems, different processing levels, and communication protocols. In such a direction, recent innovations in IT devices have enabled the rail industry to gather information from Train Control and Monitoring Systems (TCMS) to check conditions constantly and prevent issues, thus improving relia-bility and safety and, in some cases, leading to cost-saving by optimizing maintenance resources. In such a scenario, this paper presents RailRED, a framework for simulating and prototyping a TCMS based on the Node-RED tool. In RaiIRED, the main TCMS subsystems are modeled using Node-RED flows, while the subsystem interconnections are performed through a low footprint and encrypted gateway based on the MQTT protocol. The proposal can also generate diagnostic data that mimic the behavior of a real-world TCMS. RailRED communication latency and its ability to generate diagnostic data have been analyzed, with the latter evaluated by using clusters of diagnostic events collected from a real-world TCMS running on a high-speed train.

The evolution of industrial environments makes the reconfigurability and flexibility key requirements to rapidly adapt to changeable market needs. Computing paradigms like Edge/Fog computing are able to provide the required flexibility and scalability while guaranteeing low latencies and response times. Orchestration systems play a key role in these environments, enforcing automatic management of resources and workloads’ lifecycle, and drastically reducing the need for manual interventions. However, they do not currently meet industrial non-functional requirements, such as real-timeliness, determinism, reliability, and support for mixed-criticality workloads. In this article, we present k4.0s, an orchestration system for Industry 4.0 (I4.0) environments, which enables the support for real-time and mixed-criticality workloads. We highlight through experiments the need for novel monitoring approaches and propose a workflow for selecting monitoring metrics, which depends on both workload requirements and hosting node guarantees. We introduce new abstractions for the components of a cluster in order to enable criticality-aware monitoring and orchestration of real-time industrial workloads. Finally, we design an orchestration system architecture that reflects the proposed model, introducing new components and prototyping a Kubernetes-based implementation, taking the first steps towards a fully I4.0-enabled orchestration system.

Real-time containers are a promising solution to reduce latencies in time-sensitive cloud systems. Recent efforts are emerging to extend their usage in industrial edge systems with mixed-criticality constraints. In these contexts, isolation becomes a major concern: a disturbance (such as timing faults or unexpected overloads) affecting a container must not impact the behavior of other containers deployed on the same hardware. In this paper, we propose a novel architectural solution to achieve isolation in real-time containers, based on real-time co-kernels, hierarchical scheduling, and time-division networking. The architecture has been implemented on Linux patched with the Xenomai co-kernel, extended with a new hierarchical scheduling policy, named SCHED_DS, and integrating the RTNet stack. Experimental results are promising in terms of overhead and latency compared to other Linux-based solutions. More importantly, the isolation of containers is guaranteed even in presence of severe co-located disturbances, such as faulty tasks (elapsing more time than declared) or high CPU, network, or I/O stress on the same machine.

This paper describes a study on log mining in the domain of microservices technologies. We focus on the detection of anomalies from logs, i.e., events requiring deeper inspection by analysts. Log mining is challenging in microservices systems due to the high number of heterogeneous logs. We present Micro2vec, a novel approach to mine numeric representations of computer logs without making assumptions on the format of underlying data and requiring no application knowledge; representations computed by Micro2vec are suited for anomaly detection. To cope with the lack of publicly-available datasets of labeled logs from production systems, we validate our approach by means of a mixture of direct measurements from logs, one-class classification experiments and generation of log variants. The study has been conducted in the context of a Clearwater IP Multimedia Subsystem setup consisting of microservices deployed in Docker containers, and on a real-world critical information system from the Air Traffic Control domain, which implements a communication model typically used with microservices.

Monitoring is a core practice in any software system. Trends in microservices systems exacerbate the role of monitoring and pose novel challenges to data sources being used for monitoring, such as event logs. Current deployments create a distinct log per microservice; moreover, composing microservices by different vendors exacerbates format and semantic heterogeneity of logs. Understanding and traversing the logs from different microservices demands for substantial cognitive work by human experts. This paper proposes a novel approach to accompany microservices logs with black box tracing to help practitioners in making informed decisions for troubleshooting. Our approach is based on the passive tracing of request-response messages of the REpresentational State Transfer (REST) communication model. Differently from many existing tools for microservices, our tracing is application transparent and non-intrusive. We present an implementation called MetroFunnel and conduct an assessment in the context of two case studies: a Clearwater IP Multimedia Subsystem (IMS) setup consisting of Docker microservices and a Kubernetes orchestrator deployment hosting tens of microservices. MetroFunnel allows making useful attributions in traversing the logs; more important, it reduces the size of collected monitoring data at negligible performance overhead with respect to traditional logs.

Monitoring is a core practice in any software system, and entails gathering a variety of data sources that pertain the execution of a given system. Trends in microservices systems exacerbate the role of monitoring. Microservices put forth reduced size, independency, flexibility and modularity principles, which well cope with ever-changing business environments. However, as real-world applications are decomposed, they can easily reach hundreds of microservices. This inherent complexity determines an increasing difficulty in debugging, monitoring and forensics, and poses novel challenges to monitoring data sources, such as event logs.

Real-time containers are a promising solution to reduce latencies in time-sensitive cloud systems. Recent efforts are emerging to extend their usage in industrial edge systems with mixed-criticality constraints. In these contexts, isolation becomes a major concern: a disturbance (such as timing faults or unexpected overloads) affecting a container must not impact the behavior of other containers deployed on the same hardware. In this paper, we propose a novel architectural solution to achieve isolation in real-time containers, based on real-time co-kernels, hierarchical scheduling, and time-division networking. The architecture has been implemented on Linux patched with the Xenomai co-kernel, extended with a new hierarchical scheduling policy, named SCHED_DS, and integrating the RTNet stack. Experimental results are promising in terms of overhead and latency compared to other Linux-based solutions. More importantly, the isolation of containers is guaranteed even in presence of severe co-located disturbances, such as faulty tasks (elapsing more time than declared) or high CPU, network, or I/O stress on the same machine.

Phishing attacks continue to pose a major threat for computer system defenders, often forming the first step in a multi-stage attack. There have been great strides made in phishing detection; however, some phishing emails appear to pass through filters by making simple structural and semantic changes to the messages. We tackle this problem through the use of a machine learning classifier operating on a large corpus of phishing and legitimate emails. We design SAFe-PC (Semi-Automated Feature generation for Phish Classification), a system to extract features, elevating some to higher level features, that are meant to defeat common phishing email detection strategies. To evaluate SAFe-PC , we collect a large corpus of phishing emails from the central IT organization at a tier-1 university. The execution of SAFe-PC on the dataset exposes hitherto unknown insights on phishing campaigns directed at university users. SAFe-PC detects more than 70 percent of the emails that had eluded our production deployment of Sophos, a state-of-the-art email filtering tool. It also outperforms SpamAssassin, a commonly used email filtering tool. We also developed an online version of SAFe-PC, that can be incrementally retrained with new samples. Its detection performance improves with time as new samples are collected, while the time to retrain the classifier stays constant.

Critical information systems strongly rely on event logging techniques to collect data, such as housekeeping/error events, execution traces and dumps of variables, into unstructured text logs. Event logs are the primary source to gain actionable intelligence from production systems. In spite of the recognized importance, system/application logs remain quite underutilized in security analytics when compared to conventional and structured data sources, such as audit traces, network flows and intrusion detection logs. This paper proposes a method to measure the occurrence of interesting activity (i.e., entries that should be followed up by analysts) within textual and heterogeneous runtime log streams. We use an entropy-based approach, which makes no assumptions on the structure of underlying log entries. Measurements have been done in a real-world Air Traffic Control information system through a data analytics framework. Experiments suggest that our entropy-based method represents a valuable complement to security analytics solutions.

Monitoring is a consolidated practice to characterize the dependability behavior of a software system. A variety of techniques, such as event logging and operating system probes, are currently used to generate monitoring data for troubleshooting and failure analysis. In spite of the importance of monitoring, whose role can be essential in critical software systems, there is a lack of studies addressing the assessment and the comparison of the techniques aiming to monitor the occurrence of failures during operations. This paper proposes a method to characterize the monitoring techniques implemented in a software system. The method is based on a fault injection approach and allows measuring 1) precision and recall of a monitoring technique and 2) the dissimilarity of the data it generates upon failures. The method has been used in two critical software systems implementing event logging, assertion checking, and source code instrumentation techniques. We analyzed a total of 3 844 failures. With respect to our data, we observed that the effectiveness of a technique is strongly affected by the system and type of failure, and that the combination of different techniques is potentially beneficial to increase the overall failure reporting ability. More important, our analysis revealed a number of practical implications to be taken into account when developing a monitoring technique.

Event logs are the first place where to find useful information about application failures. Event logs are available at different system levels, such as application, middleware and operating system. In this paper we analyze the failure reporting capability of event logs collected at different levels of an industrial system in the Air Traffic Control (ATC) domain. The study is based on a data set of 3,159 failures induced in the system by means of software fault injection. Results indicate that the reporting ability of event logs collected at a given level is strongly affected by the type of failure observed at runtime. For example, even if operating system logs catch almost all application crashes, they are strongly ineffective in face of silent and erratic failures in the considered system.

The analysis of monitoring data is extremely valuable for critical computer systems. It allows to gain insights into the failure behavior of a given system under real workload conditions, which is crucial to assure service continuity and downtime reduction. This paper proposes an experimental evaluation of different direct monitoring techniques, namely event logs, assertions, and source code instrumentation, that are widely used in the context of critical industrial systems. We inject 12,733 software faults in a real-world air traffic control (ATC) middleware system with the aim of analyzing the ability of mentioned techniques to produce information in case of failures. Experimental results indicate that each technique is able to cover a limited number of failure manifestations. Moreover, we observe that the quality of collected data to support failure diagnosis tasks strongly varies across the techniques considered in this study.