In explainable AI, many explanation methods generate similar yet diverging explanations for machine learning (ML) models. How fair is it then to explain ML model behaviour by such explanations? Arguably, one needs to judge whether those explanations are good at explaining ML model input/output behaviour. We here attempt to formalise ways to judge goodness of such explanations in terms of their correctness. For assessing correctness, one needs to have desirable properties of explanation correctness in mind, as well as was to measure satisfaction of those properties. We submit two high-level properties of soundness and completeness for assessing explanation correctness: explaining is sound if the model behaves the way the explanations say; explaining is complete if explanations can be given for model’s outputs on any inputs. We formulate soundness and completeness properties for three forms of explanations: feature importance, counterfactuals and rules. We further formalise multiple general metrics, at least one for each property and form of explanation, for quantitatively measuring satisfaction of soundness and completeness. We argue that explanations are correct in as much as various aspects of the different forms of explanations are met as quantified by those metrics. We hope that being able to assess correctness of ML model input/output behaviour explanations against formal properties and metrics is a substantial step towards fairly explaining ML-based inference.

A main challenge of using 5G network slices is to meet all the quality of service requirements of the slices (which are agreed with the customer in a service level agreement (SLA)), throughout the network slices' lifecycle. To avoid the penalty for violation, a proactive solution is presented, including predicting the SLA violation, explaining the violation cause, and then providing an adaptation to traffic. This work uses counterfactual (CF) explanations to 1) explain the main factors affecting the identified model's SLA violation prediction and 2) present modifications in the input values, which are required to configure the network traffic to avoid such a violation. We evaluate the CF explanation at two different levels where the generated CF instance is fed to the predictive model, and then actuation data are generated to evaluate the result in the real network. Our solution minimizes the violation up to 98%. This information can be utilized to reconfigure the system, either by humans or by the system automatically, to make it fully autonomous on the one hand and comply with the 'right to explanation' introduced by the European Union's General Data Protection Regulation on the other hand.

Robots with constrained hardware resources usually rely on Multi-access Edge Computing infrastructures to offload computationally expensive tasks to meet real-time and safety requirements. Offloading every task might not be the best option due to dynamic changes in the network conditions and can result in network congestion or failures. This work proposes a task offloading strategy for mobile robots in a Human-Robot Collaboration scenario that optimizes the edge resource usage and reduces network delays, leading to safety enhancement. The solution utilizes a Deep Reinforcement Learning (DRL) agent that observes safety and network metrics to dynamically decide at runtime if (i) a less accurate model should run on the robot; (ii) a more complex model should run on the edge; or (iii) the previous output should be reused through temporal coherence verification. Experiments are performed in a simulated warehouse where humans and robots have close interactions and safety needs are high. Our results show that the proposed DRL solution outperforms the baselines in several aspects. The edge is used only when the network performance is reliable, reducing the number of failures (up to 47 %). The latency is not only decreased (up to 68 %) but also adapted to the safety requirements (risk × latency reduced up to 48 %), avoiding unnecessary network congestion in safe situations and letting other devices use the network. Overall, the safety metrics get improved, such as the increased time in the safe zone by up to 3.1%.

Edge computing is projected to have profound implications in the coming decades, proposed to provide solutions for applications such as augmented reality, predictive functionalities, and collaborative Cyber-Physical Systems (CPS). For such applications, edge computing addresses the new computational needs, as well as privacy, availability, and real-time constraints, by providing local high-performance computing capabilities to deal with the limitations and constraints of cloud and embedded systems. Edge computing is today driven by strong market forces stemming from IT/cloud, telecom, and networking—with corresponding multiple interpretations of “edge computing” (e.g., device edge, network edge, distributed cloud). Considering the strong drivers for edge computing and the relative novelty of the field, it becomes important to understand the specific requirements and characteristics of edge-based CPS, and to ensure that research is guided adequately, e.g., avoiding specific gaps. Our interests lie in the applications of edge computing as part of CPS, where several properties (or attributes) of trustworthiness, including safety, security, and predictability/availability, are of particular concern, each facing challenges for the introduction of edge-based CPS. We present the results of a systematic mapping study, a kind of systematic literature survey, investigating the use of edge computing for CPS with a special emphasis on trustworthiness. The main contributions of this study are a detailed description of the current research efforts in edge-based CPS and the identification and discussion of trends and research gaps. The results show that the main body of research in edge-based CPS only to a very limited extent consider key attributes of system trustworthiness, despite many efforts referring to critical CPS and applications like intelligent transportation. More research and industrial efforts will be needed on aspects of trustworthiness of future edge-based CPS including their experimental evaluation. Such research needs to consider the multiple interrelated attributes of trustworthiness including safety, security, and predictability, and new methodologies and architectures to address them. It is further important to provide bridges and collaboration between edge computing and CPS disciplines.

Artificial Intelligence (AI) is implemented in various applications of telecommunication domain, ranging from managing the network, controlling a specific hardware function, preventing a failure, or troubleshooting a problem till automating the network slice management in 5G. The greater levels of autonomy increase the need for explainability of the decisions made by AI so that humans can understand them (e.g. the underlying data evidence and causal reasoning) consequently enabling trust. This paper presents first, the application of multiple global and local explainability methods with the main purpose to analyze the root-cause of Service Level Agreement violation prediction in a 5G network slicing setup by identifying important features contributing to the decision. Second, it performs a comparative analysis of the applied methods to analyze explainability of the predicted violation. Further, the global explainability results are validated using statistical Causal Dataframe method in order to improve the identified cause of the problem and thus validating the explainability.

Autonomous Vehicles (AV) are expected to bring considerable benefits to society, such as traffic optimization and accidents reduction. They rely heavily on advances in many Artificial Intelligence (AI) approaches and techniques. However, while some researchers in this field believe AI is the core element to enhance safety, others believe AI imposes new challenges to assure the safety of these new AI-based systems and applications. In this non-convergent context, this paper presents a systematic literature review to paint a clear picture of the state of the art of the literature in AI on AV safety. Based on an initial sample of 4870 retrieved papers, 59 studies were selected as the result of the selection criteria detailed in the paper. The shortlisted studies were then mapped into six categories to answer the proposed research questions. An AV system model was proposed and applied to orient the discussions about the SLR findings. As a main result, we have reinforced our preliminary observation about the necessity of considering a serious safety agenda for the future studies on AI-based AV systems.

This paper presents implementation details of AI methods used for safety in human robot collaborative scenarios, based on fuzzy logic and state-of-the-art deep learning-based perception. For semantic representation of the environment, a scene graph encodes the environment surrounding the robots with information from camera, which is then fed to risk management system for safety analysis purpose. Transfer learning effects have been observed when starting training based on weights pre-trained on Image Net and COCO data-sets. A fuzzy logic solution for risk analysis, evaluation and mitigation has been compared to a neuro-fuzzy approach. Experiments have been performed in a physically realistic 3D simulation of a warehouse environment to evaluate which configuration presents the best performance for robotic perception and risk mitigation.

This paper addresses the problem of reducing the number of messages needed to exchange state updates between the Cyber-Physical System (CPS) components that integrate with the rest of the CPS through Digital Twins in order to maintain uniform communication interface and carry out their tasks correctly and safely. The main contribution is a proposed architecture and the discussion of its suitability to support correct execution of complex tasks across the CPS. A new State Event Filtering component is presented to provide event-based communication among Digital Twins that are based on the Linked Data principles while keeping the fan-out limited to ensure the scalability of the architecture.

Collaborative robotics is recently taking an ever-increasing role in modern industrial environments like manufacturing, warehouses, mining, agriculture and others. This trend introduces a number of advantages, such as increased productivity and efficiency, but also new issues, such as new risks and hazards due to the elimination of barriers between humans and robots. In this paper we present risk assessment for an automated warehouse use case in which mobile robots and humans collaborate in a shared workspace to deliver products from the shelves to the conveyor belts. We provide definitions of specific human roles and perform risk assessment of human-robot collaboration in these scenarios and identify a list of hazards using Hazard Operability (HAZOP). Further, we present safety recommendations that will be used in risk reduction phase. We develop a simulated warehouse environment using V-REP simulator. The robots use cameras for perception and dynamically generate scene graphs for semantic representations of their surroundings. We present our initial results on the generated scene graphs. This representation will be employed in the risk assessment process to enable the use of contextual information of the robot's perceived environment, which will be further used during risk evaluation and mitigation phases and then on robots' actuation when needed.

Autonomous vehicle systems within an intelligent transportation systems (ITS) paradigm have attracted continuously increasing interest from both academia and the industry. Safety is a key characteristic; without safety, the dream of autonomous vehicles cannot come true. This paper focuses on vehicle safety aspects and proposes a new conceptual framework, supported by a consolidated, international normative risk management process and is based on a safety-critical architecture brought from an analogous transportation domain, to design and to implement an autonomous supervision and control system (SCS). This SCS approach, based on automotive cyber physical systems (ACPSs) and called autonomous ACPS, aims to minimize the probability of vehicle collision hazard by employing resilient actions at run-time, reducing risks related to potential loss of human lives in the automotive transportation domain.

The Multi-Resource Server (MRS) technique has been proposed to enable predictable execution of memory intensive real-time applications on COTS multi-core platforms. It uses resource reservation approaches in the context of CPU-bandwidth and memory-bus bandwidth reservations to bound the interference between the applications running on the same core as well as between the applications running on different cores. In this paper we present a complete composable local and global schedulability analysis for the Multi-Resource Server technique. Based on the proposed analysis, we further provide an experimental study that investigates the behaviour of the MRS and identifies the factors that contribute mostly on the overall system performance.

5G networks will be a key enabler for the Internet of Things by providing a platform to connect a massive number of devices with heterogeneous sets of network quality requirements. In this environment, 5G network operators will have to solve the complex challenge of managing network services for diverse customer sectors (such as automotive, health or energy) with different requirements throughout their lifecycle. In this paper, we present current state of our work on automating part of the network service lifecycle management using knowledge management- and decision support techniques. We also present our ongoing implementation steps for such management function in 5G networks.

In real-time embedded systems the components and components integration must satisfy both functional correctness and extra-functional correctness, such as satisfying timing properties. Deploying multiple real-time components on a physical node poses timing problems in components's integration. These timing problems during integration further effect predictability and reusability of real-time components. We propose a novel concept of runnable virtual node (RVN) whose interaction with the environment is bounded both by a functional and a temporal interface, and the validity of its internal temporal behaviour is preserved when integrated with other components or when reused in a new environment. Our realization of RVN exploits the latest techniques for hierarchical scheduling framework to achieve temporal isolation, and the principles from component-based software-engineering to achieve functional isolation. Proof-of-concept case studies executed on an AVR based 32-bit micro-controller demonstrates the preserving of real-time properties within components for predictable integration and reusability in a new environment without altering its temporal behaviour in both hierarchical scheduling and RVN contexts. We also take a step ahead towards expanding the performance preserving servers' concept for multicore platform on which the scheduling of real-time tasks is inherently unpredictable due to the contention for shared physical memory and caches. It results in proposing and implementation of a novel type of server, called Multi-Resource Server (MRS) which controls the access to both CPU and memory bandwidth resources such that the execution of real-time tasks become predictable. The MRS provides temporal isolation both between tasks running on the same core, as well as, between tasks running on different cores. Further, we provide the schedulability analysis for MRS to provide predictable performance when composing multiple components on a shared multi-core platform.

In this paper we present challenges that hinder the predictable integration and execution of real-time applications on multicore platforms. We investigate how shared resources, like CPU, memory-bus bandwidth, caches, and memory cause unpredictability and interference. We propose to adapt the traditional server-based scheduling approach on the multicore platforms with additional resource-reservations to control the shared access to such resources and present the multi-resource server as a solution such that the execution of real-time applications becomes predictable.

In this paper we present an implementation and demonstration of the Multi-Resource Server (MRS) which enables predictable execution of real-time applications on multi-core platforms. The MRS provides temporal isolation both between tasks running on the same core, as well as, between tasks running on different cores. The latter could, without MRS, interfere with each other due to contention on a shared memory bus. We demonstrate that MRS can be used to “encapsulate” legacy systems and to give them enough resources to fulfill their purpose. In our case study a legacy media-player is integrated with several resource-hungry tasks running at a different core. We show that without MRS the media-player starts to drop frames due to the interference from other tasks; while introduction of MRS alleviates this problem. Another part of our demonstration shows how traditional periodic real-time tasks can be kept schedulable even when tasks with high memory-demand are added to the system.

Multi-mode embedded real-time systems exhibit a specific behaviour for each mode, and upon a mode-change request the task-set and timing interfaces of the system need to be changed. This paper presents the implementation of a MultiMode Adaptive Hierarchical Scheduling Framework (MMAHSF) and provides a generic skeleton (framework) for a two-level adaptive hierarchical scheduling supporting multiple modes and multiple mode-change mechanisms on an open source real-time operating system (FreeRTOS). The MMAHSF enable application-specific implementations of mode-change protocols using a set of predefined mode-change mechanisms. The paper addresses different mode-change mechanisms at both global and local scheduling levels. It presents examples of mode-change protocols that are developed by composing together these mechanisms in multiple ways and provide the initial results of executing these protocols in the MMAHSF implementation on an AVR 32-bit board EVK1100.

In this paper we present our ongoing work on implementing the multi-resource server technology in the Linux operating system running on multi-core architectures. The multi-resource server is used to control the access to both CPU and memory bandwidth resources such that the execution of real-time tasks become predictable. We are targeting Legacy applications to be migrated from single to multi-core architectures. We investigate the available techniques and mechanisms that can support our multi-resource servers and we discuss the potential problems that needed to be tackled considering the requirements of legacy applications.

Memory contention is one of the largest sources of inter-core interference in statically partitioned multicore systems, and the contention reduces the overall performance of applications and causes unpredictable execution-times. A first step in achieving predictable execution is to accurately measure the amount of consumed memory bandwidth for each application. Such measurements can be used to track down bottlenecks, provide better partitioning among cores, and ultimately be used to arbitrate and police access to the memory bus. We propose to use hardware performance counters to continuously track the memory-bandwidth consumed by different applications executing in parallel. In this paper we describe ongoing efforts exploring suitable performance counters on core-level and on system-on-chip level for the 8-core Freescale P4080 processor. The aim is to accurately and efficiently track consumed memory bandwidth per application; with the final goal to use these measurements to improve predictability of multicore realtime software.

This paper presents the implementation of a Hierarchical Scheduling Framework (HSF) on an open source real-time operating system (FreeRTOS) to support the temporal isolation between a number of applications, on a single processor. The goal is to achieve predictable integration and reusability of independently developed components or applications. We present the initial results of the HSF implementation by running it on an AVR 32-bit board EVK1100. The paper addresses the fixed-priority preemptive scheduling at both global and local scheduling levels. It describes the detailed design of HSF with the emphasis of doing minimal changes to the underlying FreeRTOS kernel and keeping its API intact. Finally it provides (and compares) the results for the performance measures of idling and deferrable servers with respect to the overhead of the implementation.

In this paper we propose a message passing synchronization protocol for resource sharing among real-time applications on multi-core platforms where each application is allocated on a cluster of cores. In this protocol the resources that are only used within an application (local resources) are handled by shared memory synchronization while the resources shared cross applications (global resources) are accessed by means of message passing. In our protocol the global resources are safely accessed without requiring to lock the resources explicitly. The goal is to avoid resource locking using shared memory, since accessing shared memory in multi-cores is very time consuming, whereas message passing has the potential to be much more efficient in systems with deep memory hierarchies.