We address the problem of detecting adversarial attacks against cooperative multi-agent reinforcement learning with continuous action space. We propose a decentralized detector that relies solely on the local observations of the agents and makes use of a statistical characterization of the normal behavior of observable agents. The proposed detector utilizes deep neural networks to approximate the normal behavior of agents as parametric multivariate Gaussian distributions. Based on the predicted density functions, we define a normality score and provide a characterization of its mean and variance. This characterization allows us to employ a two-sided CUSUM procedure for detecting deviations of the normality score from its mean, serving as a detector of anomalous behavior in real-time. We evaluate our scheme on various multi-agent PettingZoo benchmarks against different state-of-the-art attack methods, and our results demonstrate the effectiveness of our method in detecting impactful adversarial attacks. Particularly, it outperforms the discrete counterpart by achieving AUC-ROC scores of over 0.95 against the most impactful attacks in all evaluated environments.

We consider the problem of detecting adversarial attacks against cooperative multi-agent reinforcement learning. We propose a decentralized scheme that allows agents to detect the abnormal behavior of one compromised agent. Our approach is based on a recurrent neural network (RNN) trained during cooperative learning to predict the action distribution of other agents based on local observations. The predicted distribution is used for computing a normality score for the agents, which allows the detection of the misbehavior of other agents. To explore the robustness of the proposed detection scheme, we formulate the worst-case attack against our scheme as a constrained reinforcement learning problem. We propose to compute an attack policy by optimizing the corresponding dual function using reinforcement learning. Extensive simulations on various multi-agent benchmarks show the effectiveness of the proposed detection scheme in detecting state-of-the-art attacks and in limiting the impact of undetectable attacks.

Precise time synchronization of Phasor Measurement Units (PMUs) is critical for monitoring and control of smart grids. Thus, time synchronization attacks (TSAs) against PMUs pose a severe threat to smart grid security. In this paper we present an approach for detecting TSAs based on the interaction between the time synchronization system and the power system. We develop a phasor measurement model and use it to derive an accurate closed form expression for the correlation between the frequency adjustments made by the PMU clock and the resulting change in the measured phase angle, without an attack. We then propose one model-based and three data-driven TSA detectors that exploit the change in correlation due to a TSA. Using extensive simulations, we evaluate the proposed detectors under different strategies for implementing TSAs, and show that the proposed detectors are superior to state-of-the-art clock frequency anomaly detection, especially for unstable clocks.

In this paper, we evaluate throughput performance of spatially distributed RFID tags where tags closer to the reader will be at an advantage of having the possibility of being captured more frequently by the reader. We provide an analysis of the optimal frame length during reader interrogation rounds and show that it depends on the probability of capture. Over successive rounds of Dynamic Frame Slotted ALOHA, DFSA, the reader tends to finalize interrogation of closer tags with stronger received power and leaves the reading field to be predominantly populated by farther ones with relatively weaker received power. As a result, it is anticipated that capture probability decreases after each DFSA round. Accordingly, a Linear Frame Length Stepping Algorithm LFLSA for frame length selection in each DFSA round is proposed. In the proposed scheme, the capture probability is linearly varied from its initial value of the first DFSA round until it reaches zero at the final rounds where all remaining tags are essentially received with equal power. Simulation results show that applying LFLSA introduces a (7% - 11%) gain in the achieved slot throughput depending on the RFID system environment when compared to other frame length setting that has been widely utilized in the literature that assumes 100% successful capture of any scenario of two-tags collision timeslots.

UHF EPC Gen2 RFID systems use Dynamic Frame Slotted ALOHA (DFSA) to manage reading of multiple RFID tags. In DFSA, the reader may receive short address messages (RN16) from one or more tags at any given timeslot. To enable RFID readers to read a significantly large population of tags within a small time interval, it is required to increase the underlying throughput by maximizing the percentage of timeslots in which the reader may receive RN16 messages correctly. Common convention in the literature presumes that readers have the ability to correctly decode all RN16 messages from single-tag timeslots as well as to resolve at least one RN16 message from two-tag collision timeslots. However, practically dependent on operating SNR conditions, the reader may incur RN16 decoding errors within such timeslots. As a result, attainable throughputs become inferior to theoretical limits expected in the corresponding literature. This paper analyzes the time throughput performance of RFID systems with imperfect RN16 decoding. The optimal DFSA frame length is derived and the corresponding throughput performance is evaluated. Results reveal that there exists a cutoff probability when the reader should stop attempting to resolve two-tag collisions and only confine its efforts in decoding single-tag timeslots.