Deep reinforcement learning (DRL) has achieved remarkable success in solving sequential decision-making problems across various domains. However, a critical challenge is sample inefficiency, especially in real-world environments with high-dimensional solution spaces due to continuous state and action spaces. Although off-policy actor-critic algorithms have been proposed to mitigate this issue, the gains in sample efficiency remain limited, as decision-making in these algorithms relies solely on the policy function that might not always yield optimal actions. To bridge the gap, we design a novel feedback-based decision-making mechanism (FADA) that incorporates a feedback mechanism into the actor-critic framework to enhance decision-making robustness. Specifically, FADA utilizes feedback from the value function (critic) to calibrate the decisions produced by the policy function (actor). More concretely, FADA comprises four integrated modules: a decision-space expansion module (DEM) to produce a pool of candidate actions, a critic-guided evaluation module (CGEM) that estimates the efficacy of the candidate actions, an adaptive selection module (ASM) that adaptively selects a set of elite actions based on the estimated efficacy and samples the final action, and an iterative refinement module (IRM) that improves the quality of elite actions. We evaluate our approach on multiple tasks in the DeepMind Control Suite, and the experimental results demonstrate a significant improvement in sample efficiency.

With the advent of IoT technology, smart building management has been transformed, leading to significant improvements in energy efficiency and occupant comfort. Indoor room temperature control is crucial as it affects both building performance and occupant quality of life. Nevertheless, strin-gent cybersecurity measures are required due to the increasing susceptibility to cyber attacks with more IoT links in smart buildings. Identifying and managing unusual temperature readings is essential to keep the system running smoothly, efficiently, and safely. By integrating classical control methods such as PID with anomaly detection and LSTM modeling, this approach enables proactive anomaly identification and accurate temperature fore-casts, rendering sustainable and resilient living conditions. This integration optimizes resource usage and mitigates cyber risks. This paper presents a holistic method that combines PID control, LSTM forecasting, and anomaly detection for smart building applications. The proposed integrated approach successfully addresses aberrant temperature variations and enhances building performance, as shown through experimental validation.

Smart cities have appeared as an essential paradigm, leveraging advanced technology and data-driven techniques to enhance residents' quality of life, drive economic growth, improve governance, and facilitate environmental sustainability. Promoting experimental testing of existing and emerging technologies in realistic smart-city-simulated environments is paramount for their development. OpenCyberCity, a smart city testbed developed at Virginia Commonwealth University, echoes this progress by incorporating smart functionalities such as smart buildings, traffic systems, manufacturing, data analytics, autonomous response systems, and microgrid infrastructure capabilities. This paper presents recent expansions and updates on the OpenCyberCity testbed, enabling further experimentation across various smart city domains, with the aim of improving energy conservation, transportation, building management, resilience, and sustainable infrastructure development.

Drug overdose is now the leading cause of death for those under 50 in the United States. Inadequate data present a challenge for city officials, which prevents them from investigating the scale of the opioid overdose crisis. Various factors need to be considered in the prediction model for estimating the level of drug consumption, type of drug, and the location of the affected area. The aim of this project is to investigate several prediction and analysis models for forecasting drug use and overdoses by considering diverse data obtained from different sources, including sewage-based drug epidemiology, healthcare data, social networks data mining, and police data. Such analysis will help to formulate more effective policies and programs to combat fatal opioid overdoses.

Unsignalized intersections are often sources of congestion and collisions. When human-driven vehicles arrive simultaneously, the drivers typically creep out into the inter-section or wave each other through to break stalemates. While intuitive for human drivers, this approach would be challenging for autonomous vehicles (AVs). Current AVs typically operate in isolation without explicitly communicating their intentions to others. In this paper, we propose an auction-based intersection management system (IMS) to determine a crossing schedule. Vehicles bid for crossing time using a cost function over different possible crossing times, and the IMS assigns crossing times that maximize social utility. We evaluate our system with an ambiguous crossing scenario and demonstrate its usefulness in determining socially-optimal crossing schedules.

As DDoS (Distributed Denial of Service) attacks constantly evolve and bombard businesses and organizations from time to time, DDoS mitigation cloud service is a popular solution to defend against DDoS attacks. Decision makers can select which services to deploy given the associated risk and the deployment cost. In this work, we establish a game-theoretic model to simulate the decision making of attackers and defenders under the context of DDoS attacks. We simulate the attacker/defender game under different scenarios and demonstrate that the efficacy of using external services is impacted by several factors including the resources of the organization, the potential damage and the attacker cost/reward. We find that under different scenarios, the Nash Equilibrium may vary drastically from no attack at all to definite attack. Our study can provide useful insights to decision makers and stakeholders on their DDoS defense strategy planning. To the best of our knowledge, this is the first game model to investigate the DDoS attack/defense strategy involving third-party services.

Safety risk assessment is an essential process to ensure a dependable cyber-physical system (CPS) design. Traditional risk assessment considers only physical failures. For modern CPSs, failures caused by cyber attacks are on the rise. The focus of latest research effort is on safety–security lifecycle integration and the expansion of modeling formalisms for risk assessment to incorporate security failures. The interaction between safety and security lifecycles and its impact on the overall system design, as well as the reliability loss resulting from ignoring security failures, are some of the overlooked research questions. This article addresses these research questions by presenting a new safety design method named cyber layer of protection analysis (CLOPA) that extends the existing layer of protection analysis (LOPA) framework to include failures caused by cyber attacks. The proposed method provides a rigorous mathematical formulation that expresses quantitatively the tradeoff between designing a highly reliable and a highly secure CPS. We further propose a co-design lifecycle process that integrates the safety and security risk assessment processes. We evaluate the proposed CLOPA approach and the integrated lifecycle on a practical case study of a process reactor controlled by an industrial control testbed and provide a comparison between the proposed CLOPA and current LOPA risk assessment practice.

One of the major trends in research on Self-Protecting Systems is to use a model of the system to be protected to predict its evolution. However, very often, devising the model requires special knowledge of mathematical frameworks, that prevents the adoption of this technique outside of the academic environment. Furthermore, some of the proposed approaches suffer from the curse of dimensionality, as their complexity is exponential in the size of the protected system. In this paper, we introduce a model-integrated approach for the design of Self-Protecting Systems, which automatically generates and solves Markov Decision Processes (MDPs) to obtain optimal defense strategies for systems under attack. MDPs are created in such a way that the size of the state space does not depend on the size of the system, but on the scope of the attack, which allows us to apply it to systems of arbitrary size.

This paper introduces an indirect adaptive fuzzy model predictive control strategy for a nonlinear rotational inverted pendulum with model uncertainties. In the first stage, a nonlinear prediction model is provided based on the fuzzy sets, and the model parameters are tuned through the adaption rules. In the second stage, the model predictive controller is designed based on the predicted inputs and outputs of the system. The control objective is to track the desired outputs with minimum error and to maintain closed-loop stability based on the Lyapunov theorem. Combining the adaptive Mamdani fuzzy model with the model predictive control method is proposed for the first time for the nonlinear inverted pendulum. Moreover, the proposed approach considers the disturbances predictions as part of the system inputs which have not been considered in the previous related works. Thus, more accurate predictions resistant to the parameters variations enhance the system performance using the proposed approach. A classical model predictive controller is also applied to the plant, and the results of the proposed strategy are compared with the results from the classical approach. Results proved that the proposed algorithm improves the control performance significantly with guaranteed stability and excellent tracking.

Intelligent transportation systems (ITSs) and other smart-city technologies are increasingly advancing in capability and complexity. While simulation environments continue to improve, their fidelity and ease of use can quickly degrade as newer systems become increasingly complex. To remedy this, we propose a hardware- and software-based traffic management system testbed as part of a larger smart-city testbed. It comprises a network of connected vehicles, a network of intersection controllers, a variety of control services, and data analytics services. The main goal of our testbed is to provide researchers and students with the means to develop novel traffic and vehicle control algorithms with higher fidelity than what can be achieved with simulation alone. Specifically, we are using the testbed to develop an integrated management system that combines model-based control and data analytics to improve the system performance over time. In this paper, we give a detailed description of each component within the testbed and discuss its current developmental state. Additionally, we present initial results and propose future work.

City infrastructure is deteriorating, traffic management systems are becoming increasingly inefficient due to volume, and resources are becoming scarce. In the era of information and analytics, the idea of smart cities has been increasingly proposed as a solution to inefficient public services and resource management. While some cities have had success with beginning to transform into smart cities, the process has revealed significant barriers. One of which is the communication infrastructure necessary to create an interconnected network of sensors, actuators, and analytics systems. This barrier is discussed, and a physical testbed for smart city research is proposed. The current progress of the testbed development is reported, and a plan for continued work is outlined.

The continuous increase in the quantity and sophistication of cyberattacks is making it more difficult and error prone for system administrators to handle the alerts generated by intrusion detection systems (IDSs). To deal with this problem, several intrusion response systems (IRSs) have been proposed lately. IRSs extend the IDSs by providing an automatic response to the detected attack. Such a response is usually selected either with a static attack-response mapping or by quantitatively evaluating all available responses, given a set of predefined criteria. In this article, we introduce a probabilistic model-based IRS built on the Markov decision process (MDP) framework. In contrast to most existing approaches to intrusion response, the proposed IRS effectively captures the dynamics of both the defended system and the attacker and is able to compose atomic response actions to plan optimal multiobjective long-term response policies to protect the system. We evaluate the effectiveness of the proposed IRS by showing that long-term response planning always outperforms short-term planning, and we conduct a thorough performance assessment to show that the proposed IRS can be adopted to protect large distributed systems at runtime.

Property-graphs are becoming popular for Intrusion Detection Systems (IDSs) because they allow to leverage distributed graph processing platforms in order to identify malicious network traffic patterns. However, a benchmark for studying their performance when operating on big data has not yet been reported. In general, benchmarking a system involves the execution of workloads on datasets, where both of them must be representative of the application of interest. However, few datasets containing real network traffic are openly available due to privacy concerns, which in turn could limit the scope and results of the benchmark. In this work, we build two synthetic data generators for benchmarking next generation IDSs by introducing the support for property-graphs in two well-known graph generation algorithms: Barabási-Albert and Kronecker. We run an extensive experimental evaluation using a publicly available dataset as seed for the data generation, and we show that the proposed approach is able to generate synthetic datasets with high veracity, while also exhibiting linear performance scalability.

The quantity and sophistication of cyber attacks have increased year by year, thus it is infeasible to manually process Intrusion Detection Systems (IDSs) alerts. Intrusion Response Systems (IRSs) extend IDSs by providing automatic protection mechanisms. The core of an IRS is its planning algorithm, in charge of selecting the best response action to counter the detected attacks. However, the planning algorithm has to be carefully designed and implemented in order to exhibit a low overhead and not to compromise the scalability of the protected system. In this paper we present the performance evaluation of an IRS based on Markov Decision Process (MDP), which leverages many-core co-processors. Such an IRS produces optimal long-term response policies evaluated according to a multi-criteria objective function. We show that, despite the complexity of the MDP modeling, the proposed IRS is able to protect large systems while introducing little to no overhead on the protected hosts.

The extensive use of virtualization in implementing cloud infrastructure brings unrivaled security concerns for cloud tenants or customers and introduces an additional layer that itself must be completely configured and secured. Intruders can exploit the large amount of cloud resources for their attacks. Most of the current security technologies do not provide the essential security features for cloud systems such as early warnings about future ongoing attacks, autonomic prevention actions, and risk measure. This paper discusses the integration of these three features to our Autonomic Cloud Intrusion Detection Framework (ACIDF). The early warnings are signaled through a new finite State Hidden Markov prediction model that captures the interaction between the attackers and cloud assets. The risk assessment model measures the potential impact of a threat on assets given its occurrence probability. The estimated risk of each security alert is updated dynamically as the alert is correlated to prior ones. This enables the adaptive risk metric to evaluate the cloud's overall security state. The prediction system raises early warnings about potential attacks to the autonomic component, controller. Thus, the controller can take proactive corrective actions before the attacks pose a serious security risk to the system. According to our experiments, both risk metric and prediction model have successfully signaled early warning alerts 39.6 minutes before the launching of the LLDDoS1.0 attack. This gives the system administrator or an autonomic controller ample time to take preventive measures.

Distributed computing systems (DCS) host a wide variety of enterprise applications in dynamic and uncertain operating environments. These applications require stringent reliability, availability, and quality of service (QoS) guarantee to maintain their service level agreements (SLAs). Due to the growing size, increasing complexity, and varying nature of applications hosted in DCS, development of a single autonomic performance management system is difficult to maintain the SLAs of all of these applications. Therefore, a customizable autonomic performance management system is introduced in this paper, by using model-integrated computing methodologies, which allow application domain architects to develop meta-models of each system, various system modules with attributes, their connectivity, constraints, and visualization aspects. Then, domain engineers can define the initial settings of the application, QoS objectives, system components' placement, and interaction among these components in a graphical domain specific modeling environment. This configurable performance management system facilitates reusability of the same components, algorithms, and application performance models in different deployment settings.

Cloud computing significantly increased the security threats because intruders can exploit the large amount of cloud resources for their attacks. However, most of the current security technologies do not provide early warnings about such attacks. This paper presents a Finite State Hidden Markov prediction model that uses an adaptive risk approach to predict multi-staged cloud attacks. The risk model measures the potential impact of a threat on assets given its occurrence probability. The attacks prediction model was integrated with our autonomous cloud intrusion detection framework (ACIDF) to raise early warnings about attacks to the controller so it can take proactive corrective actions before the attacks pose a serious security risk to the system. According to our experiments on DARPA 2000 dataset, the proposed prediction model has successfully fired the early warning alerts 39.6 minutes before the launching of the LLDDoS1.0 attack. This gives the auto response controller ample time to take preventive measures.

This paper introduces an autonomic model-based cyber security management approach for the Internet of Things (IoT) ecosystems. The approach aims at realizing a self-protecting system, which has the ability to autonomously estimate, detect, and react to cyber attacks at an early stage. Our approach integrates various model-based techniques including: 1) real-time estimation and baseline security controls to predict and eliminate potential cyber attacks; 2) data analysis to identify and classify attacks; and 3) a multicriteria optimization method to select the optimal active response for deploying countermeasures while maintaining system functions. The prototype framework has been developed with a master controller virtual machine, which can be configured for various platforms. Experimental results demonstrated the effectiveness of this proposed approach in protecting a Web-based application against known and unknown attacks with little or no human intervention.

This paper describes an integrated monitoring and control framework for managing performance of distributed enterprise systems.

Fault-tolerant frameworks for large scale computing clusters require sensor programs, which are executed periodically to facilitate performance and fault management. By construction, these clusters use general purpose operating systems such as Linux that are built for best average case performance and do not provide deterministic scheduling guarantees. Consequently, periodic applications show jitter in execution times relative to the expected execution time. Obtaining a deterministic schedule for periodic tasks in general purpose operating systems is difficult without using kernel-level modifications such as RTAI and RTLinux. However, due to performance and administrative issues kernel modification cannot be used in all scenarios. In this paper, we address the problem of jitter compensation for periodic tasks that cannot rely on modifying the operating system kernel. ; Towards that, (a) we present motivating examples; (b) we present a feedback controller based approach that runs in the user space and actively compensates periodic schedule based on past jitter; This approach is platform-agnostic i.e. it can be used in different operating systems without modification; and (c) we show through analysis and experiments that this approach is platform-agnostic i.e. it can be used in different operating systems without modification and also that it maintains a stable system with bounded total jitter.

This paper presents a conservative approximation method for the real-time verification of asynchronous event-driven distributed systems. This problem is known to be undecidable in the generic setting. The proposed approach is based on composable timed automata models that provide a sufficient condition to determine schedulability. We demonstrate the method on a real-time CORBA avionics design.

This paper introduces thecross-abstractionreal-timeanalysis(Carta) framework for the model-based functional verification and performance estimation of chip multiprocessors (CMPs) utilizing bus matrix (crossbar switch) interconnection networks. We argue that the inherent complexity in CMP designs requires the synergistic use of various models of computation to efficiently manage the tradeoffs between accuracy and complexity. Our approach builds on domain-specific modeling languages (DSMLs) driving an open-source tool-chain that provides a cross-abstraction bridge between the finite-state machine (FSM), discrete-event (DE), and timed automata (TA) models of computation, and utilizes multiple model checkers to analyze formal properties at the cycle-accurate and transaction-level abstractions. The cross-abstraction analysis exploits accuracy for functional verification, and achieves significant speedups for performance estimation with marginal accuracy loss. We demonstrate results on an industrial strength networking CMP design utilizing a bus matrix interconnection network. To the best of our knowledge, the Carta framework is the first model-based tool-chain that utilizes multiple abstractions and model checkers for the comprehensive and formal functional verification, performance estimation, and real-time verification of bus matrix-based CMP designs.

This paper addresses adaptive performance management of real-time computing systems. We consider a generic model-based predictive control approach that can be applied to a variety of computing applications in which the system performance must be tuned using a finite set of control inputs. The paper focuses on several key aspects affecting the application of this control technique to practical systems. In particular, we present techniques to enhance the speed of the control algorithm for real-time systems. Next we study the feasibility of the predictive control policy for a given system model and performance specification under uncertain operating conditions. The paper then introduces several measures to characterize the performance of the controller, and presents a generic tool for system modeling and automatic control synthesis. Finally, we present a case study involving a real-time computing system to demonstrate the applicability of the predictive control framework.

Key challenges in the performance estimation of distributed real-time embedded (DRE) systems include the systematic measurement of coverage by simulations, and the automated generation of directed test vectors. This paper investigates how DRE systems can be represented as discrete event systems (DES) in continuous time, and proposes an automated method for the performance evaluation of such systems. The proposed method also provides a way for the verification of dense time properties for a large class of DRE systems. This approach provides a formal executable model allowing to bridge the gap between simulations and formal verification. Our results show that the proposed DES-based evaluation method can achieve better coverage in large-scale DRE systems than alternative methods.

In this work, a discrete input receding horizon controller (DIRHC) is proposed, as motivated by the need to use solenoid valves to control the motion of a pneumatic hoping robot. The proposed controller structure is for a switching control system in which only a finite number of discrete-valued control inputs are available. The controller utilizes a model to predict system behavior along a finite forward horizon, establishes an optimization problem, and finds the optimal control sequence of discrete-valued inputs. Stability is addressed by adding a terminal equality constraint to the control formulation. A modified depth first search algorithm, named sorted depth first search (sDFS), is presented to improve the searching efficiency and then compared to an exhaustive search. The proposed search preserves the completeness of the exhaustive search, while significantly reducing time and space complexity. The approach is applied to a pneumatic hopping robot system where the motion control is re-formulated as an explicit energy regulation problem. The proposed DIRHC controller and sDFS algorithm are used to realize the motion specification by maintaining the system energy around desired levels. Simulation results demonstrate the effectiveness of the proposed method.

This paper develops a scalable online optimization framework for the autonomic performance management of distributed computing systems operating in a dynamic environment to satisfy desired quality-ofservice objectives. To efficiently solve the performance management problems of interest in a distributed setting, we develop a hierarchical structure where a highlevel limited-lookahead controller manages interactions between lower-level controllers using forecast operating and environment parameters. We develop the overall control structure, and as a case study, show how to efficiently manage the power consumed by a computer cluster. Using workload traces from the Soccer World Cup 98 web site, we show via simulations that the proposed method is scalable, has low run-time overhead, and adapts quickly to time-varying workload patterns.

Key challenges in distributed real-time embedded (DRE) system developments include safe composition of system components and mapping the functional specifications onto the target platform. Model-based verification techniques provide a way for the design-time analysis of DRE systems enabling rapid evaluation of design alternatives with respect to given performance measures before committing to a specific platform. This paper introduces a semantic domain for model-based analysis of a general class of DRE systems capturing their key time-based performance measures. We then utilize this semantic domain to develop a verification strategy for preemptive schedulability using available model checking tools. The proposed framework and verification strategy is demonstrated on a mission-critical avionics DRE system case study.

Dependable computer systems hosting critical commerce, transportation, and military applications, among others, must satisfy stringent quality-of-service (QoS) requirements. However, as these systems become increasingly complex, maintaining the desired QoS by manually tuning the numerous performance-related parameters are very difficult. This paper develops a generic online control framework to design self-managing computer systems. The proposed approach explores a limited region of the system state-space at each time step and decides the best control action accordingly. We present two case studies to demonstrate the practicality of the proposed control framework.

Distributed real-time embedded (DRB) systems often need to satisfy various time, resource and fault-tolerance constraints. To manage the complexity of scheduling these systems many methods use rate monotonic scheduling assuming a time-triggered architecture. This paper presents a method that captures the reactive behavior of complex time- and event-driven systems, can provide simulation runs and can provide exact characterization of timed properties of component-based DRE applications that use the publisher/subscriber communication pattern. We demonstrate our approach on real-time CORBA avionics applications.

In this paper we present a model-based approach for designing quality of service adaptive applications. We have developed a prototype distributed QoS modeling environment (DQME) that captures important elements of dynamic QoS adaptation at the model level. This modeling environment is designed independent of and can be integrated with, specific application domains to capture their QoS features and adaptation strategies. It combines the domain-specific modeling capability of the generic modeling environment with the QoS adaptation mechanisms of the quality objects middleware framework. DQME captures both the QoS and the functional concerns of distributed real-time embedded systems, and provides clear separation of these two. Integrated code-synthesis tools facilitate code generation and model refinement. We present a signal analyzer case study to demonstrate the use of the DQME modeling tool in real world applications.

In this paper, we present an approach for QoS management that can be applied to a general class of real-time distributed computation systems. In the proposed approach, a switching hybrid system model is used to represent the system. In this setting, the QoS specifications are transformed into set-point specifications and a limited-horizon online supervisory controller is used to move the system to the optimal operation point.

In this paper we present an approach for QoS management that can be applied for a general class of real-time distributed computation systems. In this paper, the QoS adaptation problem is formulated based on a utility function that measures the relative performance of the system. A limited-horizon online supervisory controller is used for this purpose. The online controller explores a limited region of the state-space of the system at each time step and decides the best action accordingly. The feasibility and accuracy of the online algorithm can be assessed at design time.

In this paper a modelling and analysis paradigm for multiprocess discrete event systems is presented within the formal language and automata settings. The proposed modelling structure features explicit representation of the system components as well as their interaction constraints. The model is then extended for hierarchical multilevel systems.

In this paper we propose a method to create discrete abstraction of state space behavior for continuous-time systems based on gradient analysis of the system dynamics. Then we describe how to use such a discrete model to design a supervisory controller for a given safety specification for the system. Finally we provide an entropy measure of nondeterminism, which can be used to evaluate the quality of the result discrete model as the degree of nondeterminism in that model.