Cyber Physical Systems (CPSs) have a larger attack surface due to the integration of unprotected sensors and actuators into cyber infrastructure and hence a significant amount of research effort is devoted to address the problems of cyber attacks on these systems. In this article, we address the problem of discovering the signatures of a broad type of cyber attacks that can be launched by a remote attacker using malware on an operational CPS. Our aim is to efficiently detect and prevent such attacks at the boundary of cyber infrastructure and before the payloads can actually cause any damage to the system. In particular, we have considered a large dataset of an operational and popular CPS testbed, called SWaT (Secure Water Treatment), where a number of such cyber attacks have been launched and the network traces, without any specific evidence of such attacks, have been made public recently so that effective security solutions can be developed. We have proposed an effective method to analyze the traffic to discover the signatures of these cyber attacks. Our method has discovered an exact set of signatures based on the packets of Common Industrial Protocol (CIP) in EtherNet/Industrial Protocol stack (ENIP/CIP) of all “sensor reading distortion” and “actuator state alteration” attacks present in SWaT.A6_Dec2019 dataset for the first time in this article. Leveraging these signatures, we have proposed an algorithm that takes as input a network trace file containing ENIP/CIP packets and a set of signatures and automatically generates as output a graphical model of the cyber infrastructure of SWaT without using any background information and the path in the model that the signatures travel. Our analysis of computational time to execute the algorithm shows that the processing of raw network trace files, a step in the algorithm, consumes a considerable amount of time. Hence, we have developed a set of rules using the signatures and deployed them in Suricata, a well-known and well-adopted rules-based network intrusion detection system, to generate effective alert logs. We found that the rules in Suricata can produce alerts with zero false positives and false negatives in the SWaT.A6_Dec2019 dataset and in three other SWaT datasets for the two types of attacks.

In this paper, we investigate Non-Terrestrial Network (NTN)-empowered Multi-access Edge Computing (MEC) systems for emerging applications like telemedicine, industrial automation, and augmented reality, which demand significant computational resources from User Equipments (UEs). To address these issues, we consider that UEs can either process tasks locally or offload them to edge servers deployed on Unmanned Aerial Vehicles (UAVs) and Low Earth Orbit (LEO) satellites. The efficacy of such systems crucially depends on formulating an efficient computation offloading policy. We propose a multi-agent distributed deep reinforcement learning (MADDRL) approach for computation offloading in NTN-empowered MEC systems. Our approach utilizes two distributed DRL frameworks: Centralized Training and Decentralized Execution (CTDE), where training occurs on a central controller, and Independent Learners (IL), where training is performed locally on the UEs. In both frameworks, policies are executed by UEs in a decentralized manner. Simulation results show that the CTDE framework reduces the average cost, defined as the weighted sum of delay and energy consumption experienced by the UEs, by 11%, 42%, and 66% compared to IL, local, and random policies, respectively.

Providing a high Quality of Experience (QoE) for video streaming in 5G and beyond 5G(B5G) networks is challenging due to the dynamic nature of the underlying network conditions. Several Adaptive Bit Rate (ABR) algorithms have been developed to improve QoE, but most of them are designed based on fixed rules and unsuitable for a wide range of network conditions. Recently, Deep Reinforcement Learning (DRL) based Asynchronous Advantage Actor-Critic (A3C) methods have recently demonstrated promise in their ability to generalise to diverse network conditions, but they still have limitations. One specific issue with A3C methods is the lag between each actor’s behavior policy and central learner’s target policy. Consequently, suboptimal updates emerge when the behavior and target policies become out of synchronization. In this paper, we address the problems faced by vanilla-A3C by integrating the on-policybased multi-agent DRL method into the existing video streaming framework. Specifically, we propose a novel system for ABR generation- Proximal Policy optimization-based DRL for Adaptive Bit Rate streaming (PPO-ABR). Our proposed method improves the overall video QoE by maximizing sample efficiency using a clipped probability ratio between the new and the old policies on multiple epochs of minibatch updates. The experiments on real network traces demonstrate that PPO-ABR outperforms stateof-the-art methods for different QoE variants.

Adaptive bitrate (ABR) algorithms are used to adapt the video bitrate based on the network conditions to improve the overall video quality of experience (QoE). Recently, reinforcement learning (RL) and asynchronous advantage actor-critic (A3C) methods have been used to generate adaptive bit rate algorithms and they have been shown to improve the overall QoE as compared to fixed rule ABR algorithms. However, a common issue in the A3C methods is the lag between behaviour policy and target policy. As a result, the behaviour and the target policies are no longer synchronized which results in suboptimal updates. In this work, we present ALISA: An Actor-Learner Architecture with Importance Sampling for efficient learning in ABR algorithms. ALISA incorporates importance sampling weights to give more weightage to relevant experience to address the lag issues with the existing A3C methods. We present the design and implementation of ALISA, and compare its performance to state-of-the-art video rate adaptation algorithms including vanilla A3C implemented in the Pensieve framework and other fixed-rule schedulers like BB, BOLA, and RB. Our results show that ALISA improves average QoE by up to 25%-48% higher average QoE than Pensieve, and even more when compared to fixed-rule schedulers.

Intrusion detection systems (IDS) generate a large number of false alerts which makes it difficult to inspect true positives. Hence, alert prioritization plays a crucial role in deciding which alerts to investigate from an enormous number of alerts that are generated by IDS. Recently, deep reinforcement learning (DRL) based deep deterministic policy gradient (DDPG) off-policy method has shown to achieve better results for alert prioritization as compared to other state-of-the-art methods. However, DDPG is prone to the problem of overfitting. Additionally, it also has a poor exploration capability and hence it is not suitable for problems with a stochastic environment. To address these limitations, we present a soft actor-critic based DRL algorithm for alert prioritization (SAC-AP), an off-policy method, based on the maximum entropy reinforcement learning framework that aims to maximize the expected reward while also maximizing the entropy. Further, the interaction between an adversary and a defender is modeled as a zero-sum game and a double oracle framework is utilized to obtain the approximate mixed strategy Nash equilibrium (MSNE). SAC-AP finds robust alert investigation policies and computes pure strategy best response against opponent's mixed strategy. We present the overall design of SAC-AP and evaluate its performance as compared to other state-of-the art alert prioritization methods. We consider defender's loss, i.e., the defender's inability to investigate the alerts that are triggered due to attacks, as the performance metric. Our results show that SAC-AP achieves up to 30% decrease in defender's loss as compared to the DDPG based alert prioritization method and hence provides better protection against intrusions. Moreover, the benefits are even higher when SAC-AP is compared to other traditional alert prioritization methods including Uniform, GAIN, RIO and Suricata.

This paper presents NANCY, a system that generates adaptive bit rates (ABR) for video and adaptive network coding rates (ANCR) using reinforcement learning (RL) for video distribution over wireless networks. NANCY trains a neural network model with rewards formulated as quality of experience (QoE) metrics. It performs joint optimization in order to select: (i) adaptive bit rates for future video chunks to counter variations in available bandwidth and (ii) adaptive network coding rates to encode the video chunk slices to counter packet losses in wireless networks. We present the design and implementation of NANCY, and evaluate its performance compared to state-of-the-art video rate adaptation algorithms including Pensieve and robustMPC. Our results show that NANCY provides 29.91% and 60.34% higher average QoE than Pensieve and robustMPC, respectively.

In this paper, we present the functional design and experimental validation of network coding technology over hybrid networks including satellite links. We first describe our design framework based on a holistic modelling of (overlay) heterogeneous networking satellite scenarios. We then define different types of logical nodes depending on their encoding, re-encoding and decoding functionalities and whether or not the satellite (overlay) application designer has control over them. Nodes are assumed strategically chosen to recode, which may result in a small number of re-encoding nodes that suffice to optimize selected performance metrics. Our main contribution is a system-oriented functional design of network coding that enables flexible instantiation of different types of network codes via configurable network coding (C-NC) functions. Random or structured NC coefficients can be remotely or locally generated and a packet scheduler can forward packets according to different policies. The choice of coefficients and overall NC scheme depend on the SATCOM-specific performance target, namely delay or bandwidth constraints. Here, we present a preliminary design and experimental testebed validation for the case of delay constrained transmission. Our results show the practical benefits of re-encoding and performance tradeoffs of different network coding schemes. In particular, our results show the good structural properties and delay-reliability tradeoffs of our novel proposal of structured network codes using Pascal matrices due to the regenerative properties of the coding coefficients.

For the two-user interference channel, the Han-Kobayashi (HK) scheme is known to achieve optimal rates within one bit of the capacity for Gaussian codebooks. In this paper, we propose its practical implementation based on finite M-QAM constellations. For the very strong interference regime, we derive the interference level conditions and its geometrical interpretation. For the weak interference regime, we propose a joint private/common constellation design which utilizes the interference-free constellation spaces and induces the geometrical conditions for this regime. Our results show that for the very strong interference regime, the desired signal is transmitted interference-free over a larger range of channel values than in the Gaussian case. Furthermore, for the weak interference regime, the achievable rates of the proposed scheme are compared with 1) the theoretical limit of the HK scheme when Gaussian codebooks are used [8] and 2) the orthogonal rates. In particular, the proposed scheme achieves upto 150% increase in the transmission rates as compared to the orthogonal rates.