We study service scheduling problems in a slotted system in which agents arrive with service requests according to a Bernoulli process and have to leave within two slots after arrival, service costs are quadratic in service rates, and there is also a waiting cost. We consider fixed waiting costs. We frame the problem as an average cost Markov decision process . While the studied system is a linear system with quadratic costs, it has state dependent control. Moreover, it also possesses a non-standard cost function structure rendering the optimization problem complex. Here, we characterize the optimal policy. We also consider a system in which the agents make scheduling decisions for their respective service requests keeping their own cost in view. We frame this scheduling problem as a stochastic game. Here, we provide Nash equilibrium.

We study content caching in a cellular network consisting of a base station with a cache. New contents arrive in the network according to a Poisson process and the contents stay for exponentially distributed times. At any given time, all the contents in the network have the same popularity or request rate. Also, contents’ request rates are time-varying, instantaneous request rates being a decreasing function of the number of contents in the network. Precaching contents at the base station incurs a cost. However, fetching contents from the server on being requested incurs an even higher cost. We formulate caching problem as a Markov decision process and derive the optimal caching policy. We also propose a Reinforcement Learning based algorithm that yields precaching decisions when system parameters are unknown. Numerical results show that the proposed algorithm’s time averaged cost is close to the optimal average cost.

We study service scheduling problems in a slotted system in which jobs arrive according to a Bernoulli process and have to leave within two slots after arrival, service costs are quadratic in service rates, and there is also a linear waiting cost. We illustrate how these systems can be used to model CPU speed scaling, EV charging at a charging center, load scheduling in smart grids etc. We frame the problems as average cost Markov decision processes. While the studied system is a linear system with quadratic costs, it has state dependent control and non-standard cost function structure, rendering the optimization problem complex. We obtain explicit optimal policies in the case when all the jobs are of same size. In particular, we show that the optimal policy is linear or piece-wise linear in the system state, depending on the system parameters. Further, when the job sizes can take two distinct values, we provide an algorithm that yields the optimal policy. We illustrate different forms of these policies via numerical examples.