The stochastic nature of wireless communication suggests a Markov Decision Process (MDP) as a formalism for identifying and evaluating spectrum control policies. However, in practice numerous factors influence the success or failure of a transmission, so that the applicability of particular MDP models to real spectrum management problems must itself be examined. This paper presents a series of model validation studies in which correspondence between an MDP model and a discrete-event simulation (DES) model is evaluated. We test several hypotheses that together provide a foundation and an exemplar for the idea of using MDPs to guide management of shared spectrum. We conclude that there is sufficient similarity between the performance predictions made by the MDP model and the DES model that MDPs can be used effectively to determine spectrum control policies.

Time utility functions can describe the complex timing constraints of real-time and cyber-physical systems. However, utility aware scheduling policy design is an open research problem. Previously we solved a Markov Decision Process formulation of the scheduling problem to derive value-optimal scheduling policies for systems with periodic real-time task sets and stochastic non-preemptive execution intervals. However, the complexity of computing solutions and their policy storage requirements necessitate the exploration of scalable solutions. In this paper we generalize the Utility Accrual Packet Scheduling Algorithm. We compare several heuristics to Markov Decision Process policy evaluation under soft and hard real-time conditions, different load conditions, and different classes of time utility functions. Based on these evaluations we present guidelines for which heuristics are best suited to particular scheduling criteria.

Open soft real-time systems, such as mobile robots, must respond adaptively to varying operating conditions, while balancing the need to perform multiple mission specific tasks against the requirement that those tasks complete in a timely manner. Setting and enforcing a utilization target for shared resources is a key mechanism for achieving this behavior. However, because of the uncertainty and non-preempt ability of some tasks, key assumptions of classical scheduling approaches do not hold. In previous work we presented foundational methods for generating task scheduling policies to enforce proportional resource utilization for open soft real-time systems with these properties. However, these methods scale exponentially in the number of tasks, limiting their practical applicability.In this paper, we present a novel parameterized scheduling policy that scales our technique to a much wider range of systems. These policies can represent geometric features of the scheduling policies produced by our earlier methods, but only require a number of parameters that is quadratic in the number of tasks. We provide empirical evidence that the best of these policies are competitive with exact solution methods in small problems, and significantly outperform heuristic methods in larger ones.

Abstract—Classical scheduling abstractions such as deadlines and priorities do not readily capture the complex timing semantics found in many real-time cyber-physical systems. Time utility functions provide a necessarily richer description of timing semantics, but designing utility-aware scheduling policies using them is an open research problem. In particular, scheduling design that optimizes expected utility accrual is needed for realtime cyber-physical domains. In this paper we design scheduling policies that optimize expected utility accrual for cyber-physical systems with periodic, non-preemptable tasks that run with stochastic duration. These policies are derived by solving a Markov Decision Process formulation of the scheduling problem. We use this formulation to demonstrate that our technique improves on existing heuristic utility accrual scheduling policies. I.

Real-time hybrid testing of civil structures represents agrand challenge in the emerging area of cyber-physical systems. Hybrid testing improves significantly on either purely numerical or purely empirical approaches by integrating physical structural components and computational models. Actuator dynamics, complex interactions among computers and physical components, and computation and communication delays all hamper the ability to conduct accurate tests. To address these challenges, this paper presents initial work towards a Cyber-physical Instrument for Real-time hybrid Structural Testing (CIRST). CIRST aims to provide two salient features: a highly configurable architecture for integrating computers and physical components; and system support for real-time operations in distributed hybrid testing. This paper presents the motivation of the CIRST architectureand preliminary test results from a proof-of-concept implementation that integrates a simple structural element and simulation model. CIRST will have broad impacts on thefields of both civil engineering and real-time computing.It will enable high-fidelity real-time hybrid testing of awide range of civil infrastructures, and will also providea high-impact cyber-physical application for the study andevaluation of real-time middleware.

Open soft real-time systems, such as mobile robots, experience unpredictable interactions with their environments and yet must respond both adaptively and with reasonable temporal predictability. New scheduling approaches are needed to address the demands of such systems, in which many of the assumptions made by traditional real-time scheduling theory do not hold. In previous work we established foundations for a scheduling policy design and verification approach for open soft real-time systems, that can use different decision models, e.g., a Markov decision process (MDP), to capture the nuances of their scheduling semantics.However, several important refinements to the preliminary techniques developed in that work are needed to make the approach applicable in practice. This paper makes three main contributions to the state of the art in scheduling open soft real-time systems: (1) it defines a novel representation of the scheduling state space that is both more compact and more expressive than the model defined in our previous work; (2) it exploits regular structure of that representation to allow efficient verification of properties involving both discrete and continuous system state variables under specific scheduling policies; and (3) it removes the unnecessary use of a time horizon in our previous approach, thus allowing the more precise specification and enforcement of a wider range of scheduling policies for open soft real-time systems.