The predominant use of wireless access networks is for media streaming applications. However, current access networks treat all packets identically, and lack the agility to determine which clients are most in need of service at a given time. Software reconfigurability of networking devices has seen wide adoption, and this in turn implies that agile control policies can be now instantiated on access networks. Exploiting such reconfigurability requires the design of a system that can enable a configuration, measure the impact on the application performance (Quality of Experience), and adaptively select a new configuration. Effectively, this feedback loop is a Markov Decision Process whose parameters are unknown. The goal of this work is to develop QFlow, a platform that instantiates this feedback loop, and instantiate a variety of control policies over it. We use the popular application of video streaming over YouTube as our use case. Our context is priority queueing, with the action space being that of determining which clients should be assigned to each queue at each decision period. We first develop policies based on model-based and model-free reinforcement learning. We then design an auction-based system under which clients place bids for priority service, as well as a more structured index-based policy. Through experiments, we show how these learning-based policies on QFlow are able to select the right clients for prioritization in a high-load scenario to outperform the best known solutions with over 25% improvement in QoE, and a perfect QoE score of 5 over 85% of the time.

We consider a general small-scale market for agent-to-agent resource sharing, in which each agent could either be a server (seller) or a client (buyer) in each time period. In every time period, a server has a certain amount of resources that any client could consume, and randomly gets matched with a client. Our target is to maximize the resource utilization in such an agent-to-agent market, where the agents are strategic. During each transaction, the server gets money and the client gets resources. Hence, trade ratio maximization implies efficiency maximization of our system. We model the proposed market system through a Mean Field Game approach and prove the existence of the Mean Field Equilibrium, which can achieve an almost 100% trade ratio. Finally, we carry out a simulation study motivated by an agent-to-agent computing market, and a case study on a proposed photovoltaic market, and show the designed market benefits both individuals and the system as a whole.

No abstract available.